Bimodal biometric recognition system using Convolutional Neural Networks and fusion of deep audiovisual feature vectors

Authors

  • Juan Carlos Atenco Department of Electronics, National Institute of Astrophysics, Optics and Electronics
  • Juan Carlos Moreno Department of Electronics, National Institute of Astrophysics, Optics and Electronics
  • Juan Manuel Ramirez Department of Electronics, National Institute of Astrophysics, Optics and Electronics
  • Rene Arechiga New Mexico Tech
  • Pilar Gomez Department of Computer Science, National Institute of Astrophysics, Optics and Electronics
  • Rigoberto Fonseca School of Mathematical and Computational Sciences

DOI:

https://doi.org/10.61467/2007.1558.2024.v15i2.289

Keywords:

Multimodal biometrics, Speaker recognition, Face recognition, CNN, Audiovisual biometrics

Abstract

In recent years, interest has grown in the use biometric systems for identity authentication tasks in digital services, forensic and security applications. A unimodal system (employing a single biometric trait) with high performance is still vulnerable to falsification attacks such as spoofing. For this reason, research on multimodal biometrics (employing various biometric traits) has increased to reinforce security, increase recognition performance, and make false identity authentication more difficult. In this paper, we propose a bimodal system that combines speech and face modalities by concatenating their feature vectors, these vectors are extracted from two convolutional neural networks (CNN) and used for identity verification. The performance of unimodal CNNs was evaluated individually and compared to the bimodal system of concatenated vectors. A data augmentation scheme is used for both modalities to evaluate different operation conditions. Results were measured in terms of Equal Error Rate (EER).

Author Biographies

Juan Carlos Atenco, Department of Electronics, National Institute of Astrophysics, Optics and Electronics

JUAN CARLOS ATENCO-VAZQUEZ was bornin Puebla, Mexico, in 1991. He received the B.Sc. degree from the Puebla Institute of Technology(ITP), Mexico, and the M.Sc. degree from the Na-tional Institute of Astrophysics, Optics, and Elec-tronics (INAOE), Mexico. He is currently a Ph.D.student at the Electronics Department, INAOE, inMexico. His research interests include signal pro-cessing, biometric systems, embedding systems,neural networks and applications.

Juan Carlos Moreno, Department of Electronics, National Institute of Astrophysics, Optics and Electronics

JUAN CARLOS MORENO-RODRIGUEZ wasborn in Puebla, Mexico in 1971. He received theB.Sc. degree in 1995 and his M.Sc. degree in1998 in Electronic Engineering from Universidadde las Américas-Puebla. He is currently pursuingthe Ph.D. degree in electronics at National Instituteof Astrophysics, Optics and Electronics, Mexico.He has worked as an Assistant Professor in thedepartments of Computer Systems and Electronicsat Tecnologico Nacional de Mexico and Universi-dad Iberoamericana, respectively. His research interest includes biometrics,machine learning and signal processing.

Juan Manuel Ramirez, Department of Electronics, National Institute of Astrophysics, Optics and Electronics

JUAN MANUEL RAMIREZ-CORTESreceivedthe B.Sc. degree from the National PolytechnicInstitute, Mexico, the M.Sc. degree from the Na-tional Institute of Astrophysics, Optics, and Elec-tronics (INAOE), Mexico, and the Ph.D. fromTexas Tech University, all in electrical engineer-ing. He currently holds a researcher position atINAOE. He is member of the Mexican national re-search system (SNI), level 2. His research interestsinclude signal and image processing, biometric,neural networks, fuzzy logic, and digital systems.

Rene Arechiga, New Mexico Tech

RENE ARECHIGA-MARTINEZreceived theB.Sc. degree from the National Polytechnic In-stitute, Mexico, the M.Sc. degree from StanfordUniversity, and the Ph.D. from University of NewMexico, all in electrical engineering. He is cur-rently an Associate Professor at the Electrical En-gineering Department of New Mexico Tech. Hisresearch interests include digital signal processingapplied to speech recognition and thunderstorms.

Pilar Gomez, Department of Computer Science, National Institute of Astrophysics, Optics and Electronics

PILAR GOMEZ-GILreceived the B.Sc. degreefrom the Universidad de las Americas A.C, Mex-ico, the M.Sc. and Ph.D. degrees from Texas TechUniversity, USA, all in computer science. She iscurrently a Titular Researcher in the ComputerScience Department at the National Institute ofAstrophysics, Optics, and Electronics (INAOE),Mexico. She is member of the Mexican nationalresearch system (SNI), level 1. Her research inter-ests include artificial neural networks, time seriesprediction, image processing, and pattern recognition.

Rigoberto Fonseca, School of Mathematical and Computational Sciences

RIGOBERTO FONSECA-DELGADOreceivedthe B.Sc. degree from the Faculty of SystemEngineering of the National Polytechnic School,Ecuador, the M.Sc. and Ph.D. from the NationalInstitute of Astrophysics, Optics, and Electronics,Mexico. He is a professor at the Metropolitan Au-tonomous University, Iztapalapa at Mexico City.His research interest includes classification andprediction of time series, resource allocation, andartificial intelligence applications.

Downloads

Published

2024-10-01

How to Cite

Atenco, J. C., Moreno, J. C., Ramirez, J. M., Arechiga, R., Gomez, P., & Fonseca, R. (2024). Bimodal biometric recognition system using Convolutional Neural Networks and fusion of deep audiovisual feature vectors. International Journal of Combinatorial Optimization Problems and Informatics, 15(3), 4–14. https://doi.org/10.61467/2007.1558.2024.v15i2.289

Issue

Section

Articles