Facial masks are one of the best defense against the spread of COVID-19, but their growing acceptance has a second, unintended effect: breaking through facial recognition algorithms.
Wearing face masks that adequately cover the mouth and nose causes the error rate of some of the most commonly used facial recognition algorithms to range between 5 percent and 50 percent, a study found by the U.S. National Institute of Standards and Technology (NIST). Black masks were more likely to cause mistakes than blue masks, and the more the nose was covered by the mask, the more difficult the algorithms found to identify the face.
“With the advent of the pandemic, we need to understand how facial recognition technology deals with masked faces,” said Mei Ngan, report author and NIST computer scientist. “We started by focusing on how an algorithm developed before the pandemic could be affected by people wearing face masks. Later this summer, we plan to test the accuracy of algorithms deliberately developed with masked faces in mind. ”
Face recognition algorithms, such as those tested by NIST, work by measuring the distances between features in a target’s face. Masks reduce the accuracy of these algorithms by removing most of these functions, although some still exist. This is different from how facial recognition works on iPhones, for example, which use depth sensors for added security, so that the algorithms can’t be fooled by showing the camera an image (a hazard not present in the NIST scenarios has been raised about).
While there is a lot of anecdotal evidence about face masks thwarting facial recognition, NIST’s study is particularly definitive. NIST is the government agency in charge of assessing the accuracy of these algorithms (along with many other systems) for the federal government, and its ranking of different suppliers has a significant impact.
Notably, NIST’s report only tested a type of facial recognition known as one-to-one matching. This is the procedure used at border crossings and passport control scenarios, where the algorithm checks whether the target’s face matches their ID. This is different from the type of facial recognition system used for mass surveillance, where a crowd is scanned to find matches with faces in a database. This is called a one-to-many system.
While the NIST report does not include one-to-many systems, these are generally considered to be more errors than one-to-one algorithms. Choosing faces in a crowd is more difficult because you cannot control the angle or exposure on the face and the resolution is generally lowered. That suggests that if face masks break one-to-one systems, they are likely to break one-to-many algorithms with at least the same, but probably greater, frequency.
This is consistent with reports we have heard from within the government. An internal bulletin by the United States Department of Homeland Security earlier this year, indicated by The interception, said the agency was concerned about “the potential impact that the widespread use of protective masks could have on security operations involving facial recognition systems”.
This is welcome news for privacy lawyers. Many have warned against the rush of governments around the world to embrace facial recognition systems, despite the hair-raising effects that such technology has on civil liberties, and the widely recognized racial and gender biases of these systems, which tend to underperform anyone not a white man.
Meanwhile, the companies building facial recognition technology have quickly adapted to this new world and designed algorithms that only identify faces use the eye area. Some vendors, such as the leading Russian company NtechLab, say that their new algorithms can identify individuals even if they are wearing balaclava. However, such claims are not entirely reliable. They usually come from internal data, which can be picked to produce flattering results. That is why external parties such as NIST offer standardized tests.
NIST says it plans to test specially tuned face recognition algorithms for mask wearers later this year, along with investigating the effectiveness of one-to-many systems. Despite the problems masks cause, the agency expects the technology to continue. “As for facial mask accuracy, we expect the technology to continue to improve,” said Ngan.