Biometrics identification offers improved safety and security for EVs, writes Peter Donaldson, who examines the technology’s state of play

Electric vehicles are at the nexus of several ongoing technological and socio-economic transformations. The first includes electrification itself, along with the rise of software-defined vehicles and the application of biometrics. The second is centred on demographically correlated shifts in attitudes towards car ownership, brought about by affordability and environmental concerns.

It would be a stretch to claim that technological transformation will solve the socio-economic and environmental problems, but used correctly, they can have a positive impact. Of those technologies, biometrics could prove to be the most significant in socio-economic terms, even though its initial applications focus on security, safety and convenience.

For example, the cost of ownership could push more people into renting vehicles as they need them through peer-to-peer transactions, with trust between the parties being reinforced by reliable and secure biometric identity assurance. Such assurance could, for example, enable all kinds of hands-free e-commerce transactions on the move.

For insights into the technologies, their integration into vehicles, and their wider implications, we have quizzed experts from a global biometrics and digital identity specialist that offers customised identity proofing and biometric authentication, a Tier 1 automotive supplier focused on camera-based facial authentication, and a multi-industry, multi-technology corporation that provides a secure ‘bridge’ between the physical and digital worlds.

The main benefits centre on providing a seamless customer experience, primarily improving safety, security and cybersecurity, but also in-car leisure, says the expert from the multi-industry corporation. Among the main application areas, he says, are vehicle security, advanced driver assistance system (ADAS) functions such as fatigue detection through behavioural biometrics, health monitoring and infotainment.

Measuring people

Biometrics means measuring the characteristics of living things, and there are many that when read by a sensor and interpreted by a computer can be used to identify individual human beings with a very high degree of confidence and/or to gauge their fitness to drive. Those characteristics can be physiological or behavioural in nature.

In the physiological group are, of course, the face and the voice, fingerprints, patterns in the iris and in the retina of the eye, patterns formed by veins just under the skin, and the geometry of the hand and so on. Others, such as heartbeat, respiration rate, blink rate, facial expression and speech patterns provide warnings of physiological problems that could cause an accident, although there is an overlap here and some of these can also be used to identify people.

Behavioural biometrics can also be used to support physical measurements to confirm a person’s identity and detect any issues that affect their fitness to drive, such as tiredness, illness or intoxication.

Paradigm shift

There is nothing inherent in EVs and hybrids that makes biometric technologies more important for them than for IC-engined vehicles. However, as EVs and the infrastructure required to support them represent a paradigm shift for industry and government, they are investing in advanced capabilities upfront, the biometrics and digital identity specialist notes.

“While the benefits apply across the board, the practicalities of implementing them are encouraged by the migration to EVs and the software platforms that support software defined vehicles,” he says.

Because electrification is a such a major change in technology, and is developing alongside software definition of key vehicle functions, it presents an opportunity to integrate these technologies at an early stage, providing a platform for innovation, he says.

“If I am designing a car, I can imagine it being like a large smartphone that also transports people. Its software will be updated on-the-fly, and I can put new capabilities in. I am automatically looking at this platform very differently and allowing the innovation hubs to be embedded into it.”

The expert from the multi-industry corporation points out that electrification is accelerating the rise of the software-defined, digital vehicle. “In this respect, biometric technology will be implemented more in the EV ecosystem, not only on the car itself, but in the charging experience as well,” he says.

Biometric technologies could be particularly relevant for mobile in-car payment at charging stations, the Tier 1 supplier’s expert says. “Those technologies simplify enormously the process for installing and releasing charging apps from OEM app stores. Another use case is checking drivers’ licences for shared or rented vehicles,” he adds.

The primary technologies used in vehicles at the moment are fingerprint readers and facial recognition systems, with iris scanners somewhat behind, the experts say, initially for access-related identification. All these require sensing, processing and a back-end data management infrastructure.

The expert from the multi-industry corporation explains that its bridge between the physical and digital worlds is supported by the ability to capture, encrypt and manage personal biometric data, and verifying an individual’s identity by comparing live data from the person with stored digital records. “The combination of these two types of expertise enables successful management of any kind of ID check, enrolment and access management, both physical and logical,” he says.

The high level of security it provides against spoofing is one of the most important benefits of biometric identification, the expert from the Tier 1 supplier notes, adding that the technology can prevent cyber criminals from copying and exploiting the identities of legitimate car users.

“Another is secure, convenient and fast authentication of users,” he adds. “For example, manual entry of a long and complex pin code is no longer necessary, and no further devices are needed for authentication.”

The biometrics and digital identity expert anticipates further security benefits, such as hindering or even eliminating car-jacking, referring to a recent incident at a petrol station where the jacked car had a child in the back seat. (The police were able to stop the car and the child was unhurt.) “Now imagine if the car was disabled because the unknown intruder was trying to drive it away – it just eliminates that risk,” he says.

He also argues that biometric authentication will also make allowing a different authorised driver to use the car much more convenient, while allowing the owner to retain a degree of control. “I see a world where I can lend my car to my son while setting a 2 hour time window to allowing him access to the vehicle he needs to go to the school prom, say, but not to go more than 25 miles from the school. There are all sorts of things that you can do if the key is biometric-based that a physical key does not enable.”

Coming of age

Referring to general use rather than in automotive applications, iris scanning, fingerprint reading and face identification are the most mature technologies, the latter two having received a major boost from their application to smartphones, the Tier 1 supplier notes.

The biometrics and digital identity specialist argues that they have matured rapidly in recent times thanks to the use of AI and machine learning (ML). “The impact of ML in terms of the accuracy of these algorithms has been enormous; they have come on leaps and bounds, multiple orders of magnitude in the past 5 years alone,” he says.

However, maturity is not uniform across all applications and environments, he notes. “It’s really a matter of what is most suitable and most mature for the purpose. I would argue that for outside the car, fingerprint or face recognition is best, while inside the car it is situational, and face and voice recognition each has certain advantages.”

The Tier 1 supplier has developed a secure face authentication capability in its in-car driver identification display, which was shown at the 2032 Consumer Electronics Show as an addition to driver monitoring.

As a monitoring camera covers the driver at all times, face recognition can be used in conjunction with voice recognition for an extra layer of security to authorise e-commerce transactions. “For example, many vehicle infotainment systems will read out text messages, and with biometric identification it is possible to respond to, for example, an urgent payment reminder,” the biometrics specialist says.

“You can say I would like to pay my latest Amazon order. Your voice can be used to authenticate that, and you don’t have to take your hands off the wheel or your eyes off the road.”

Voice is regarded as a secondary factor for authentication rather than the primary, and its worldwide application is challenging because the technology has to accommodate many languages and accents, as well as mitigating the effects of noise in cars and variations in microphone quality, the multi-industry corporation expert points out.

Whatever the automated identification technology is used though, the biometrics specialist adds, there are two fundamental measurements of their reliability – the false match rate and the false non-match rate.

“The false match rate is a probabilistic indicator of the chance that you can impersonate me,” he explains. “The false non-match rate is a probabilistic indicator of the likelihood of the system rejecting me, even though I am the right person. There is a general correlation between them but they move in opposite directions – as one increases, in general the other decreases and vice versa.”

This, he explains, is because making a system more accurate involves matching more points of comparison, which increases the chance of a rejection. “There might be a slight performance degradation in terms of being asked to look at the camera twice or put your fingerprint down a second time for extremely accurate readings, but with the latest generation AI/ML that is happening less and less.”

Pointing the finger

While automated fingerprint recognition has benefited from decades of development and has reached a high level of maturity, its application to cars is fairly new.

Fingerprints consist of ridges and valleys that form the main pattern, along with smaller details such as ridge endings and bifurcations known as minutiae. These can be imaged by a number of sensors on places such as exterior door handles and start buttons, and include optical, ultrasonic, thermal and capacitive devices.

Optical sensors have long been the most common. These consist of a light source that illuminates the fingertip placed in contact with a prism that refracts through a lens and onto a CCD or CMOS sensor that converts the image to pixels.

In cell phones though, they proved easy to fool using prosthetic fingerprints or high-quality images, so they have been replaced by capacitive sensors or hybrid systems that use capacitive and optical sensors together. The skin that forms ridges on a fingertip has more capacitance than the air in the valleys between the ridges, and sensors determine the value of each pixel from the measured capacitance to form the image.

Hyundai became the first manufacturer to integrate fingerprint scanning, offering it on its 2019 Santa Fe SUV, where the system can unlock the car and start the engine using information from a capacitive fingerprint reader. According to Hyundai, the chance that the system will misidentify another person’s fingerprint as belonging to the legitimate driver is 1 in 50,000, making it five times more effective on average than conventional vehicle keys including smart keys, some of which remain vulnerable to relay attacks.

Facial recognition

Development of automated face recognition began in the 1960s, but it was the 2010s before computers powerful enough to handle the task in real time were inexpensive enough for routine application to consumer electronics such as smartphones.

Face recognition is a software-intensive technology in which an image captured by a camera is fed to an algorithm that reads the geometry of the face. It uses measurements such as the distance between the eyes and from the forehead to the chin, and the 2D positions of facial landmarks such as nose, eye, eyebrow and mouth corners to generate a facial signature in the form of a mathematical formula.

Using convolutional neural networks, this formula is compared to others in a database of known faces, or to the authorised user’s facial signature stored during enrolment with the vehicle’s security system.

Accuracy in face recognition systems has been affected by factors including race, age and sex to varying degrees, so a lot of r&d effort is going into solving these problems. Cooperation from the subject improves accuracy, however, and drivers who want to use their cars are motivated to cooperate with the system.

Benchmarking of facial recognition is carried out by the US National Institute of Standards and Technology (NIST), and progress has been significant. NIST reports that, as of April 2020, the best algorithm it had tested achieved an error rate of just 0.08%, a major improvement over the best algorithm of 2014 that exhibited a 4.1% error rate.

Iris scanning

Iris scanning has some inherent advantages for identification and authentication, despite the fact that at only 11 mm across on average, the iris is small and can be difficult to image, according to a Cambridge University paper. Principal among these advantages is that the variation in patterns between individuals is enormous, the left and right eyes are different, the iris is well-protected from the environment and does not change much through a person’s life.

Distinctive details include what are known arching ligaments, furrows, ridges, crypts, rings, corona, freckles and a zigzag collarette. Colour is determined mainly by the density of the pigment melanin, with blue irises resulting from a lack of it. The predominant pattern in visible light is a ligament mesh, while the near-infrared (NIR) reveals deeper patterns in a network of fibres called the stroma.

Iris recognition systems are relatively insensitive to illumination and viewing angles, because they are planar objects. The iris also makes the job of a recognition system easier, because the eyes are easy to localise on the face and the iris within the eye thanks to its annular shape.

In general, iris recognition systems run through a staged process that begins with the location of the pupil, then the iris and the eyelids. Next, the system excludes the eyelashes and eyelids, clipping them out of the image to leave the iris, and then divides the image of the iris into blocks that it converts into feature values for comparison with a large database of scans or, for vehicle user identification, those obtained from the user’s enrolment scan.

The rear-view mirror is a fairly obvious place to put the camera system, the favoured technology being an imaging chip such as a CMOS device working in the NIR part of the spectrum.

The huge variation between individuals makes iris scanning arguably the most secure authentication technology, with one developer of a mirror-integrated system claiming that the chance of the wrong person being allowed to use the car is 1 in 10 million. That is better than face, voice and fingerprint recognition.

Behavioural biometrics

While physiological biometrics measure aspects of what people are, behavioural biometrics measures things that they do, usually unconsciously. People have characteristic ways in which they move, speak, gesticulate and generally present themselves, and in which they carry out tasks such as typing on a keyboard, for example, or operating the controls of a car.

“These days, in financial services, we use behavioural biometrics to do continuous authentication,” says the expert from the biometrics specialist. “How you fill out a password or how you access a website, how you hold your phone and swipe left or right are behavioural characteristics. You can combine a lot of these signals about a person to get a very accurate picture.”

He believes this will be important for cars in the future, because it contributes not only to authentication but also to gauging a person’s level of attention and state of health. “Take driver fatigue for example, facial biometrics can identify that it’s the right driver,” he says, “but behavioural biometrics might tell you he’s falling asleep or that he is driving slightly erratically.  Maybe his eyes are moving around or he’s moving the wheel too much for the kind of road he’s on, or he’s hitting the accelerator and the brake when there’s no traffic.”

Heart monitoring at the wheel

Another biometric that can be used for identification, authentication and health monitoring is the electrocardiogram (ECG), and the required sensors can be built into steering wheels to provide continuous and unobtrusive monitoring of the driver.

This has been under development since the early 2010s, with a Portuguese team reporting on the creation of the CardioWheel system in 2015. Electrodes on the wheel were linked to signal acquisition and filtering functions, and then to functions that detected the driver’s hands, measured heart rate, removed outlier signals and extracted patterns for classification and driver identification.

The electrodes were made from a conductive fabric that is easily integrated into the wheel. Signal processing was a challenge, according to the team, because drivers move their hands around the wheel and regularly take one hand off, while ECG systems are designed to work in static environments with the electrodes firmly attached to the patient.

The prototype system featured a custom PCB focused on noisy ECG signals, using an ARM processor for signal acquisition and filtering, and an Intel Edison computational unit for signal processing and classification.

Last year, an r&d team in Germany reported its work on the use of printed, flexible polyurethane electrodes with silver conductive elements on the steering wheel, achieving higher signal-to-noise ratios than other materials.

They recorded data from 19 people, comparing readings from the steering wheel sensors with those from control electrodes on the driver’s chest that provided an accurate independent set of measurements – the ‘ground truth’ – for comparison. Measurements were made in four scenarios: at rest and in city, highway and rural driving, recording for 5 minutes at rest and 15 minutes while on the move.

The scenarios yielded different signal-to-noise ratios, with highway the lowest and rural the highest, while the steering wheel provided usable ECG signals for different proportions of the drive time, with city at 42.6%, highway 46.67% and rural 47.72%.

Overall, the team concludes, steering-wheel ECG is feasible and delivers reliable recordings from more than 45% of the drive time. The team took 950 minutes of recordings, and says they are publicly available for further analysis.

Useful combinations

The fact that multiple human physiological parameters can be measured raises the question of what the best combinations to improve security, safety, comfort and convenience might be.

The biometrics specialist develops software to implement combinational biometrics, and the company’s expert argues that the combination of face, voice and behavioural biometrics could make the biggest difference in automotive applications, with the application of behavioural biometrics to the face itself being particularly powerful.

“While your face won’t change if you’re tired, how you present it will,” he says. “I can detect if your eyes are closing or your head is dipping a lot more than it should, if your blink rate deviates from the average of once every 3 to 5 seconds.” 

Combining such a face-behavioural’ biometric with voice analysis brings more capabilities. If the system detects a difference in the driver’s tone it will be able to detect signs of stress or even duress by recognising pre-arranged codewords, for example, but it can also detect if the driver is compromised by alcohol, drugs or a mistake with the dosage in a medication.

“All of a sudden, they are getting very tired, and either they are droopy or they are slurring their speech,” the expert says.

The Tier 1 supplier’s expert says that the use of other biometrics such as body size and weight of both driver and passengers will help increase safety by optimising airbag deployment, enhance comfort by applying seat adjustments to suit the occupants, and boost convenience by adapting the settings of systems such as heating and air conditioning to suit their needs.

Capturing biometric information at a distance while the subject is moving is an important requirement if the system is to operate smoothly and seamlessly and without placing constraints on the user, the multi-industry corporation’s expert says. He singles out face recognition on smartphones as an example of where this has been achieved.

Training biometrics

Making judgements using biometrics information is an area in which AI, particularly ML. is essential. The expert from the multi-industry company emphasises that biometric performance is greatly improved through the use of AI, which he calls the simplest technology for solving very complex problems.

The expert from the Tier 1 supplier notes, “AI is needed to train the software algorithm during the development phase. Once loaded in a final product, the software doesn’t need the ML anymore.”

He adds that, in general, the reliability of AI networks depends on the amount and variety of the training data. “This is the biggest challenge,” he says. “The bigger the training network the more reliable the results in the final product. That means investment in a large AI network during the development phase leads to a more robust system in the final product, with almost no false-positive results.”

The expert from the multi-industry corporation says its models are assessed against big data in order to measure performance and accuracy of the AI algorithm.

The biometrics specialist’s expert says, “Training is a big piece of it – being able to present a deep neural network with good data and bad data, and have it learn. We are a global company and have been in this for years, so we have a lot of data. We have learned from other applications what works and what doesn’t, and bring that into biometrics.

“The adage ‘Garbage in, garbage out’ was never more true than in biometrics,” he adds. “Computers don’t make mistakes, machine models don’t make mistakes; they learn from what you give them, so one of the things we are always very conscious of is ground truth in the data.

“Hacking in the future won’t be about code, it will be about data, inserting bad data into good, leading to bad a result. That’s how hackers will attack ML systems – with adversarial AI.”

That is important for biometrics, because the more advanced face recognition systems will learn continuously as they are presented with new images of individual users under different conditions over time, he says. “The power of ML is critical; without it the algorithm is static, and if it stays static it can never adapt.”

Implementation challenges

Vehicle electronic architecture is evolving from multiple dedicated control modules for individual or groups of functions to a more centralised approach based on smaller numbers of very powerful computing platforms, and this will provide good support for biometrics applications. Implementation challenges remain, however.

“Biometrics vendors will need to adapt their offerings to adapt to the vehicle market and its specifics,” the multi-industry corporation’s expert notes. “Although many of today’s applications are shaped by security needs, including authentication, automotive will also focus on new use cases such as ADAS, fatigue detection, health monitoring and infotainment.”

There are also sensor technology issues to be ironed out. With cameras, the problem might be large and frequent changes in light levels, while external and internal sources of noise such as rain and passengers can be challenging for microphones and voice recognition software, particularly in small cabins. “Vendors will also need to adapt their algorithms, even the if technologies are closed-source, like other businesses do,” he says.

However, biometric technologies have become a lot more resilient in the past 4 or 5 years, the expert from the specialist company notes. “They have got better at filtering out bad signals as they continuously improve through machine learning,” he says, adding that about 10 years ago voice recognition systems had problems because voices in the background would interfere with their ability to recognise the main speaker. Such environmental factors are now much less of a problem.

Likewise, cameras used for face recognition are much more robust to interference, partly because the latest ones have a broad spectral response and are therefore not entirely reliant on the visible spectrum, so they are much less susceptible to sources of interference such as glare.

While the performance of voice and face recognition systems does depend partly on good quality microphones and cameras, devices with more than adequate performance are no longer prohibitively expensive, he expert from the specialist company says. They are also multi-purpose devices, in that their outputs can be used for health and behavioural biometrics as well as recognition and authentication.

What’s more, most new vehicles come from the factory with cameras and/or microphones for a variety of purposes unrelated to biometrics. “If you look at Teslas, there are cameras all over them, both inside and out, so all you are doing is taking a feed from one or more of them and running it through the software,” says the biometrics and digital identity specialist.

“That is very easy. All those cars also have Bluetooth for voice calls and, as they are software-defined vehicles, you could push an update into them tomorrow that does voice and face biometric identification, and no additional hardware would be needed.”

Our expert from the Tier 1 supplier confirms that its secure face authentication system reuses established camera-based driver monitoring hardware, which is supplemented by another optical hardware component, and says it is very compact and simple to integrate.

With the software infrastructure, however, the automotive industry has come to rely on platforms such as Apple Car Play and Android Auto from the big technology companies. “Now they are beginning to realise that they need their own versions of that,” the biometrics specialist says.

Biometrics applied to vehicles in general and EVs in particular are just starting to show what they can do, and embedding its capabilities into computing platforms such as Snapdragon will make them much more accessible for vehicle OEMs, enabling secure access to multiple networked services, he concludes. “If I look at where we are going, the more you can do with the car beyond just driving, knowing the identity of the person in the car is core to that.”

Acknowledgements

The author would like to thank Connor White at Daon, Daniel Naujack at Continental and Florent Abat at Thales Automotive for their help with researching this article.

Some suppliers of automotive biometrics systems

Facial recognition  

Tier 1 supplier 

Iris recognition 

Voice recognition 

Fingerprint recognition