Multimodal Biometric Verification for Business Security

Fraud costs the global economy nearly $5.127 trillion each year. Businesses are heavily investing in cybersecurity technology to combat this problem. Biometric verification, or recognition, is one option that many companies are now considering. But what is biometric verification—and how does it work?

Biometric verification confirms an individual’s identity through unique biological traits like the face, voice, fingerprints, hand geometry, and more. Biometrics cannot be lost like a password and faking them is challenging. Still, unimodal biometrics are not infallible when deployed in a single mode.

That’s why multimodal biometric recognition is quickly becoming one of the current artificial intelligence trends. This system doesn’t rely on only one identifier—it uses several at once, and in turn, makes the system more durable. Before diving too deep into this exciting technology, it’s important that we understand what types of biometric recognition methods exist today.

The Top Recognition Techniques in Biometric Technology

  • Facial recognition
  • Iris scanning
  • Fingerprint identification
  • Voice verification

According to Global Market Insights Inc., the global biometrics market will grow by 15 % compound annual growth rate (CAGR) through 2023. Artificial intelligence biometric systems will propel growth in biometrics.

Biometric technology is evolving to make authentication more convenient and secure. Development trends are currently focusing on physiological measurements (face, iris, and fingerprints) and behavioral measurements (voice).

Facial Recognition for Biometric Verification

Facial recognition technology looks for reference points (around 64) on the face, which are formed in a dimensional descriptor vector of facial features. The most important ones include:

  • The distance between the eyes
  • The width of the nostrils
  • The length of the nose
  • The shape of the cheekbones, forehead, and chin

Whenever a vector is calculated, it’s compared to the existing entries using multiple face images by calculating Euclidean distance to each feature vector of each person in the database, finding a match if it exists.

Related article: Designing Custom Face Detection & Recognition Software


However, the ease of access to photographs is problematic. Anyone can obtain a photo of someone through social media and other open sources.

According to the Keesing Journal of Documents & Identity, two strategies are necessary for facial recognition:

  • Liveness check/detection: Determine that the photo isn’t a mask or altered
  • Compare with formal ID: The submitted selfie should be compared to a formal ID
Liveness Check Against Fraud In Biometric Verification

Sophisticated algorithms and anti-spoofing technologies are used to verify that a live person is in front of the camera. We’ve assessed the different techniques, and to choose the right one, you need to compare different criteria and requirements. These different options which are available to you are presented in our overview of anti-spoofing techniques.

Related article: Anti-spoofing techniques in face recognition


Machine Learning In Biometrics: Optical Character Recognition (OCR) Technology

OCR is not an official biometric verification technology, but it’s useful as a preliminary tool for comparing the provided information with user documents.

Imagine you need to verify an individual using a photo of their face and an official ID. The information found on the ID serves as an important indicator. OCR helps cross-check between the text processed and data received after the barcode reading or MRZ.

OCR is a classic machine learning and computer vision task used to distinguish printed or handwritten text characters inside digital images of physical documents. Our team evaluated the open source and commercial services for OCR to use it as an additional security check (see our findings below).

Related article: How Machine Learning Optical Character Recognition Could Be Implemented


Iris Scanning for Biometric Verification

Iris scanning measures the unique patterns in the colored circle of your eye to verify and authenticate your identity. Unique patterns allow scanners to select up to 260 reference points, which make the technology more accurate.

How does the authentication process work?
  • A monochrome camera obtains a detailed image of the human eye
  • The camera is sensitive to infrared radiation and works in low light/dark conditions
  • A series of photographs are taken since the pupil is sensitive to light and constantly changes size
  • Special filters are applied to each point of the selected area to extract information and convert an eye shell pattern to a digital format
  • The obtained iris pattern is verified by bit with other patterns using machine

Advances in camera technology have made iris scanning more accessible. Iris scanners have made their way into consumer mobile devices, and many experts believe this technology will continue to grow.

Fingerprint Identification for Biometric Authentication

Fingerprint identification is considered the oldest and most developed biometric authentication method. There are different machine learning algorithms that you can use for fingerprint identification.

The most common is based on the selection of details. Prints contain anywhere from 30 to 40 small parts. They are characterized by position, coordinates, type (branching, ending, or delta), and orientation. The fingerprint standard is formed from this set of characteristics data and compared with other samples.


The vast majority of optical scanners are used for fingerprint biometric verification, but they are vulnerable to attacks and less used. There are two other types of scanners: capacitive and ultrasonic.

Capacitive scanners use the difference in electrical capacitance between the hollows of the fingerprint.

Ultrasonic scanners are built into the screen of a smartphone, and the reading takes place using ultrasound. This is an effective method since the ultrasonic wave “covers” the finger from several sides and not just at the point of contact. It creates a semi-3D-fingerprint that allows the ultrasonic sensor to distinguish between a real finger and a fake one.

Voice Verification for Biometric Authentication

Voice verification is rapidly developing and is an affordable solution to implement. This biometric verification method only requires a microphone and a dedicated app that can classify speech.

Good-quality voice samples are obtained by requiring a user to recite a passphrase that may include words or a series of numbers. Voice recognition systems are trained to recognize similarities between the voice characteristic patterns of individuals. The voice sample is then analyzed and compared with the previously stored voice model.

Related article: Case Study: Secure Biometric Authentication Solution Based On Data Science


Voice recognition technology isn’t perfect. An individual’s voice can change depending on their health, age, mood, and other factors.

This is where artificial intelligence in biometrics comes into play. You can introduce common distortions that occur when talking and also background noise. These training noises help the neural network recognize aspects that are not related to an individual’s voice.


Overview of Characteristic Features Of Biometrics

In the table below are presented characteristics, that should be taken into account during the development of biometric recognition solutions.

As you can see, the use of unimodal biometrics cannot be secure enough, and it’s better to use multimodal biometrics to avoid deliberate hacking.

Create Multimodal Biometric Recognition Solutions using Machine Learning and Artificial Intelligence

Multimodal biometric recognition allows you to achieve greater accuracy and security because it involves simultaneously using two or more biometric identifiers. How can you implement a solution like this? Let’s find out.

Imagine a user gains access to the system through multimodal biometric verification. The system contains a face and voice recognition module via a mobile application. Here’s how a multimodal biometric recognition solution would work:

* Creating a neural network on a mobile application is no easy task. A neural network can be integrated into an application using TensorFlow Lite. But there are certain limitations in these conversions. For instance, you should train your models in network with the help of the TensorFlow library. To transfer your model in the right way into the application, the architecture should be thought out beforehand with consideration to such specificities.

** If you don’t have a possibility to use a native application, you can use a remote server to process your data architecture like the Rest API. You’ll have to spend additional time on data transfer and protection during transportation, however.

There is an Nvidia Docker that can simplify the deployment of the system, while a service provider (such as AWS) can provide an uninterrupted communication channel, computing power for neural networks, and a convenient interface for scaling your system.

Test #1: Facial Recognition

The user creates a photo-imprint that’s stored on the device using a camera. This biometric imprint is converted and normalized using the OpenCV library.

The face is identified using a photo, and each of the 64 landmarks are highlighted. A biometric verification landmark includes things like a distance from the bridge of the nose to the eye and other facial features.

These landmarks and a cutout image of the face are transferred to a deep neural network, which is trained using the TensorFlow library.

An eDNA feature vector is formed after neural network processing is completed. The feature vector collects the biometric characteristics of a particular person. The length of the vector is usually 2048 bits.

During verification, eDNA is issued and compared with the anchor record that was formed earlier. Reverse engineering is impossible because there is no access to the vector. The biometric system will periodically update this anchor record to match an individual’s changing looks.

Test #2: Voice Verification

The user provides a voice sample through a microphone that gets processed by the Librosa library. The library reads the audio, transforms and converts it, and then transmits biometrics to the neural network (DNN).

An eDNA feature vector (2048 bits) is formed, which takes into account such biometrics as timbre, intonation, tempo, pitch, and other characteristics that the neural network was trained to respond to.

Verification based on multimodal biometrics is becoming a new trend, and one of the most important reasons for combining different methods is to increase recognition accuracy. This is how selecting the right biometrics technology helps to make your system more secure and reliable. If you have any questions regarding Data Science consulting and Machine Learning, feel free to address us.


Posted with permission from MoviDev