Spectral analysis of distorted and autoencoder-reconstructed MNIST images in a kernel-induced feature space
DOI:
https://doi.org/10.61467/2007.1558.2026.v17i3.1457Keywords:
Convolutional Neural Networks, principal components analysis, Redes neuronales convolucionales, análisis de componentes principalesAbstract
In this paper, the unsupervised learning spectral methods Principal Component Analysis (PCA) and Kernel Principal Component Analysis (KPCA) were explored. PCA captured the maximum variance of MNIST images, including their distorted and autoencoder-reconstructed versions, and KPCA, that captured the maximum variance and nonlinear structures of the feature space where the same images were mapped. Accordingly, for KPCA the kernel functions: a polynomial of degree 5, a radial basis function (RBF), and a cosine kernel, were evaluated. A correlation analysis between the distortions level, the digit inclination and the anisotropic of pixel values of the same images and the PCA-derived components from KPCA was calculated. The distortion level and anisotropy parameters appear to have strong nonlinear correlation with PCA-derived components, especially for the polynomial and the RBF kernels. Conversely, for cosine kernel weaker correlations were observed. Moreover, the digit inclination parameter appears unrelated to the PCA-derived components for all three kernels.
Spanish-language metadata / Metadatos en español
Título en español:
Análisis espectral de imágenes MNIST distorsionadas y reconstruidas mediante autoencoders en un espacio de características inducido por kernel
Resumen:
En este artículo se analizaron los métodos espectrales de aprendizaje no supervisado: el Análisis de Componentes Principales (PCA) y el Análisis de Componentes Principales con Kernel (KPCA). El PCA capturó la máxima varianza de las imágenes MNIST, incluidas sus versiones distorsionadas y las reconstruidas mediante autoencodificadores, mientras que el KPCA capturó la máxima varianza y las estructuras no lineales del espacio de características en el que se mapearon las mismas imágenes. En consecuencia, para el KPCA se evaluaron las siguientes funciones del núcleo: un polinomio de grado 5, una función de base radial (RBF) y un núcleo coseno. Se calculó un análisis de correlación entre el nivel de distorsión, la inclinación de los dígitos y la anisotropía de los valores de los píxeles de las mismas imágenes, y los componentes derivados del PCA a partir del KPCA. El nivel de distorsión y los parámetros de anisotropía parecen presentar una fuerte correlación no lineal con los componentes derivados del PCA, especialmente en el caso de los núcleos polinómicos y RBF. Por el contrario, en el caso del núcleo coseno se observaron correlaciones más débiles. Además, el parámetro de inclinación de los dígitos no parece estar relacionado con los componentes derivados del PCA en ninguno de los tres núcleos.
Palabras Claves:
Redes neuronales convolucionales, análisis de componentes principales
Smart citations:
https://scite.ai/reports/10.61467/2007.1558.2026.v17i3.1457
Dimensions.
Open Alex.
References
Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828. https://doi.org/10.1109/TPAMI.2013.50
Bishop, C. M., & Bishop, H. (2024). Autoencoders. In C. M. Bishop & H. Bishop (Eds.), Deep Learning: Foundations and Concepts (pp. 563–579). Springer. https://doi.org/10.1007/978-3-031-45468-4_19
Burger, W., & Burge, M. J. (2022). Digital image processing: An algorithmic introduction (3rd ed.). Springer. https://doi.org/10.1007/978-3-031-05744-1
Castaneda-Diaz, R., Lopez-Betancur, D., Guerrero-Mendez, C., Gonzalez-Ramirez, E., Puma-Ttito, F., & Troncoso-Pacheco, R. (2026). Visual analysis of MNIST convolutional autoencoder reconstructions via linear dimensionality reduction. In L. Martínez-Villaseñor, R. A. Vázquez, G. Ochoa-Ruiz, M. Montes Rivera, S. Zapotecas-Martínez, M. L. Barrón-Estrada, E. Mezura-Montes, & A. Gómez-Chávez (Eds.), Advances in Computational Intelligence. MICAI 2025 International Workshops (Vol. 16264, pp. 352–367). Springer. https://doi.org/10.1007/978-3-032-17930-2_25
Castaneda-Diaz, R., Lopez-Betancur, D., Guerrero-Mendez, C., González-Ramírez, E., Gómez-Jiménez, S., & Puma-Ttito, F. (2025). Cleaning binary distortion on MNIST dataset. In L. Martínez-Villaseñor, G. Ochoa-Ruiz, M. Montes Rivera, M. L. Barrón-Estrada, & H. G. Acosta-Mesa (Eds.), Advances in Computational Intelligence. MICAI 2024 International Workshops (pp. 133–142). Springer. https://doi.org/10.1007/978-3-031-83879-8_11
Cayton, L. (2008). Algorithms for manifold learning (Technical Report CS2008-0923). University of California, San Diego.
de Winter, J. C. F., Gosling, S. D., & Potter, J. (2016). Comparing the Pearson and Spearman correlation coefficients across distributions and sample sizes: A tutorial using simulations and empirical data. Psychological Methods, 21(3), 273–290. https://doi.org/10.1037/met0000079
Delete this duplicate. Keep only: van der Maaten, L. J. P., & Hinton, G. E. (2008).
Deng, X., Yuan, M., & Sudjianto, A. (2007). A note on robust kernel principal component analysis. In J. S. Verducci, X. Shen, & J. Lafferty (Eds.), Prediction and Discovery (Contemporary Mathematics, Vol. 443, pp. 21–34). American Mathematical Society. https://doi.org/10.1090/conm/443/08552
Fefferman, C., Mitter, S., & Narayanan, H. (2016). Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4), 983–1049. https://doi.org/10.1090/jams/852
Fisher, R. A. (1919). XV.—The correlation between relatives on the supposition of Mendelian inheritance. Earth and Environmental Science Transactions of the Royal Society of Edinburgh, 52(2), 399–433. https://doi.org/10.1017/S0080456800012163
Ghojogh, B. (2021). Data reduction algorithms in machine learning and data science [Doctoral dissertation, University of Waterloo]. UWSpace. https://uwspace.uwaterloo.ca/items/b30d8515-09d1-4063-98dd-1d13a4dc4e56
Ghojogh, B., Crowley, M., Karray, F., & Ghodsi, A. (2023). Elements of Dimensionality Reduction and Manifold Learning. Springer. https://doi.org/10.1007/978-3-031-10602-6
Ghojogh, B., Samad, M. N., Mashhadi, S. A., Kapoor, T., Ali, W., Karray, F., & Crowley, M. (2019). Feature selection and feature extraction in pattern analysis: A literature review [Preprint]. arXiv. https://doi.org/10.48550/arXiv.1905.02845
Gonzalez, R. C., & Woods, R. E. (2018). Digital image processing (4th ed., global ed.). Pearson.
Gorban, A. N., Kégl, B., Wunsch, D. C., & Zinovyev, A. Y. (Eds.). (2008). Principal Manifolds for Data Visualization and Dimension Reduction (Vol. 58). Springer. https://doi.org/10.1007/978-3-540-73750-6
He, X., Yan, S., Hu, Y., Niyogi, P., & Zhang, H.-J. (2005). Face recognition using Laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3), 328–340. https://doi.org/10.1109/TPAMI.2005.55
Hira, Z. M., & Gillies, D. F. (2015). A review of feature selection and feature extraction methods applied on microarray data. Advances in Bioinformatics, 2015, Article 198363. https://doi.org/10.1155/2015/198363
Jansen, A., & Niyogi, P. (2013). Intrinsic spectral analysis. IEEE Transactions on Signal Processing, 61(7), 1698–1710. https://doi.org/10.1109/TSP.2013.2238931
Jolliffe, I. T. (2014). Principal component analysis. In Wiley StatsRef: Statistics Reference Online. Wiley. https://doi.org/10.1002/9781118445112.stat06472
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
McInnes, L., Healy, J., & Melville, J. (2018). UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction [Preprint]. arXiv. https://doi.org/10.48550/arXiv.1802.03426
Mittal, M., J, P. G., M S, G. P., Devadas, R. M., Ambreen, L., & Kumar, V. (2024). Dimensionality reduction using UMAP and TSNE technique. In 2024 Second International Conference on Advances in Information Technology (ICAIT) (Vol. 1, pp. 1–5). IEEE. https://doi.org/10.1109/ICAIT61638.2024.10690797
Rebekić, A., Lončarić, Z., Petrović, S., & Marić, S. (2015). Pearson’s or Spearman’s correlation coefficient—Which one to use? Poljoprivreda, 21(2), 47–54. https://doi.org/10.18047/poljo.21.2.8
Reverter, F., Vegas, E., & Oller, J. M. (2014). Kernel-PCA data integration with enhanced interpretability. BMC Systems Biology, 8(Suppl. 2), Article S6. https://doi.org/10.1186/1752-0509-8-S2-S6
Saul, L. K., Weinberger, K. Q., Sha, F., Ham, J., & Lee, D. D. (2006). Spectral methods for dimensionality reduction. In O. Chapelle, B. Schölkopf, & A. Zien (Eds.), Semi-supervised learning. MIT Press. https://doi.org/10.7551/mitpress/9780262033589.003.0016
Schölkopf, B., Mika, S., Burges, C. J. C., Knirsch, P., Müller, K.-R., Rätsch, G., & Smola, A. J. (1999). Input space versus feature space in kernel-based methods. IEEE Transactions on Neural Networks, 10(5), 1000–1017. https://doi.org/10.1109/72.788641
Schölkopf, B., Smola, A. J., & Müller, K.-R. (1999). Kernel principal component analysis. In B. Schölkopf, C. J. C. Burges, & A. J. Smola (Eds.), Advances in kernel methods—Support vector learning (pp. 327–352). MIT Press.
Schölkopf, B., Smola, A., & Müller, K.-R. (1997). Kernel principal component analysis. In W. Gerstner, A. Germond, M. Hasler, & J.-D. Nicoud (Eds.), Artificial Neural Networks—ICANN’97 (pp. 583–588). Springer. https://doi.org/10.1007/BFb0020217
Schölkopf, B., Smola, A., & Müller, K.-R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10(5), 1299–1319. https://doi.org/10.1162/089976698300017467
Shah, F. P., & Patel, V. (2016). A review on feature selection and feature extraction for text classification. In 2016 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET) (pp. 2264–2268). IEEE. https://doi.org/10.1109/WiSPNET.2016.7566545
Shawe-Taylor, J., & Cristianini, N. (2004). Kernel Methods for Pattern Analysis. Cambridge University Press. https://doi.org/10.1017/CBO9780511809682
Strange, H., & Zwiggelaar, R. (2014). Spectral dimensionality reduction. In H. Strange & R. Zwiggelaar (Eds.), Open Problems in Spectral Dimensionality Reduction (pp. 7–22). Springer. https://doi.org/10.1007/978-3-319-03943-5_2
van der Maaten, L. J. P., & Hinton, G. E. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9, 2579–2605. https://www.jmlr.org/papers/v9/vandermaaten08a.html
van Vliet, L. J., & Verbeek, P. W. (1995). Estimators for orientation and anisotropy in digitized images. In Proceedings of the First Annual Conference of the Advanced School for Computing and Imaging (pp. 442–450). ASCI.
Walpole, R. E., Myers, R. H., Myers, S. L., & Ye, K. (2012). Probabilidad y estadística para ingeniería y ciencias (9.ª ed.). Pearson.
Wang, Q. (2012). Kernel Principal Component Analysis and its Applications in Face Recognition and Active Shape Models [Preprint]. arXiv. https://doi.org/10.48550/arXiv.1207.3538
Yun, H. S., Jargal, A., Hyun, C. M., & Seo, J. K. (2023). Nonlinear representation and dimensionality reduction. In J. K. Seo (Ed.), Deep Learning and Medical Applications (pp. 1–49). Springer. https://doi.org/10.1007/978-981-99-1839-3_1
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 International Journal of Combinatorial Optimization Problems and Informatics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
