The 2018 Gender Shades study by Joy Buolamwini and Timnit Gebru found commercial systems misclassified darker-skinned women far more often than lighter-skinned men. In 2019, the National Institute of Standards and Technology tested nearly 200 algorithms. Many produced false matches for Asian and African American faces 10 to 100 times more often than for white faces.
Those errors have consequences. In 2020, Detroit police arrested Robert Williams, a Black man, after software wrongly matched him to a shoplifting suspect. Most publicly known wrongful arrests tied to facial recognition have involved Black people.
Police can search images from driver's licenses, mugshots and social media, often with no warrant and no notice to the person identified. San Francisco banned city use of the technology in 2019, and other cities followed. There is still no comprehensive federal law limiting its use by police.