In this paper, a comprehensive approach to biometric user authentication by combining three independent models and their analyzed features was proposed. The biometric data are supposed to be images of the user’s face, thermal images of the user’s face and user’s speech. An own dataset, consisting of recordings of computer work sessions for 20 users, was used for the study and analysis. Independent authentication models are based on pairwise comparison of data using own neural network regressor trained to calculate the distance between two feature instances. To extract features, we used the FaceNet model in the case of processing face images and thermal images, and MFCCs in the case of processing audio recordings. As feature-level fusion methods were used, namely horizontal concatenation, calculation of median vector values, and calculation of average vector angles. Also considered was the aggregation at the evaluation stage, using the geometric mean, harmonic mean, median, and radar diagram.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Complex Approach to Biometric Authentication Based on Human Face, Speech and Thermal Images

  • Pavel Laptev

摘要

In this paper, a comprehensive approach to biometric user authentication by combining three independent models and their analyzed features was proposed. The biometric data are supposed to be images of the user’s face, thermal images of the user’s face and user’s speech. An own dataset, consisting of recordings of computer work sessions for 20 users, was used for the study and analysis. Independent authentication models are based on pairwise comparison of data using own neural network regressor trained to calculate the distance between two feature instances. To extract features, we used the FaceNet model in the case of processing face images and thermal images, and MFCCs in the case of processing audio recordings. As feature-level fusion methods were used, namely horizontal concatenation, calculation of median vector values, and calculation of average vector angles. Also considered was the aggregation at the evaluation stage, using the geometric mean, harmonic mean, median, and radar diagram.