Tampa Police have decided to scrap their much-criticized face-recognition system, admitting that during a two-year trial the system did not correctly identify a single suspect. Similar face-recognition systems are still in use in Pinellas County, Florida, and Virginia Beach, Virginia, though neither of these systems have ever resulted in an arrest either.
Face-recognition technology evokes images of automatic cameras scanning bustling crowds, automatically picking out terrorists from the millions of faces that pass by. One day the technology may be able to deliver on this, but currently it is still necessary for a human controller to zoom in on individual faces using a joystick. A 2001 St. Petersburg Times article describes a Tampa police officer scanning the weekend crowd in Ybor City, checking 457 faces out of the some 125,000 tourists and revelers in an evening.
Let’s do some quick math. The police are only scanning 457 out of 125,000 people on a given night, or 0.3%. That means even if ten known bad guys from the watch-list are in the crowd, there’s still only a 4% chance any one of them will be looked at by the system. That number drops to 0.4% if there’s only one bad guy in the crowd that night.
Then there’s the chance that the face recognition system doesn’t sound an alarm. A recently published evaluation of the Identix system used in Tampa gives a base hit rate of 77% (that is, 77% of people on a watch-list were correctly identified). However, that was with a watch-list of only 25 faces. The hit rate goes down as watch-list size goes up, down to 56% with a watch-list of 3000 faces. According to the Associated Press, the Tampa database had over 24,000 mug shots on its watch-list. Then there’s the problem that mug shots were taken indoors and the surveillance cameras were outdoors. According to the evaluation, mixing indoors and outdoors can reduce hit rates by around 40%. (The 40% reduction was seen on identity verification tasks; the watch-list task is actually more difficult.) Finally, these results all assume a 1% false-positive rate, which would result in five false alarms per night. Given all these (well-known) problems, it’s amazing anyone ever thought this was a good idea.
There’re several reasons I hope this failure dissuades similar attempts by other law-enforcement communities. First, as a 2001 ACLU report on the Tampa system points out, our resources could be better spent, and face recognition can give us a false sense of security. Second, a face-recognition systems in a public space gives the impression that everyone is a suspect, regardless of whether the system actually works. And finally, face recognition technology continues to improve. It won’t happen in the next few years, but at some point the technology is going to reach the point where recognition is completely automated, high accuracy, and robust. When that happens, it will be possible to track large numbers of people as they go about their daily lives, and even track people retroactively from recorded video. Hopefully by this time our society will be so inoculated against such privacy violations that such uses will be inconceivable.
References
- Tampa police eliminate facial-recognition system (Associated Press, 20 August 2003)
- Click. BEEP! Face captured (Lane DeGregory, Petersburg Times, 19 July 2001)
- Facial Recognition Vendor Test 2002
- Drawing a Blank: Tampa Police Records Reveal Poor Performance of Face-Recognition Technology (ACLU, 3 January 2002)