TSA still pushing on CAPPS II

/ Big Brother /

It seems the Transportation Security Administration is still determined to go forward with their test of the Computer Assisted Passenger Prescreening System (CAPPS II) with live data, even if it means forcing airlines to cooperate. Airlines are understandably hesitant, since Delta Airlines withdrew support after facing a passenger boycott and JetBlue is now facing potential legal action for handing over passengers’ data to a defense contractor without passenger knowledge or consent.

For those who haven’t heard about CAPPS-II, the idea is to replace the current airline security system where passenger’s names are checked against a no-fly list and people with “suspicious” itineraries like one-way flights are flagged for extra search. The TSA has released a disclosure under the Privacy Act of 1974, and Salon published a nice overview on the whole debate a few weeks ago. The ACLU also has a detailed analysis. Extremely briefly, the new system would work like this:

  1. Airlines ask for your Name, Address, Phone Number and Date of Birth.
  2. That info plus your itinerary goes to the CAPPS-II system, which
  3. sends it to commercial data services (e.g. the people who determine your credit rating) who
  4. send back a rating “indicating a confidence level in that passenger’s identity.”
  5. CAPPS-II sends all the info to the Black Ops Jedi-Mind-Reader computer that was provided by aliens back in 1947.
  6. The Black Ops computer comes back with a rating of whether you are or are not a terrorist, ax murderer, or likely to vote against the President.
  7. Based on both identity and threat ratings, the security guard either gives you a once-over, strip-search, or shoots you on sight (actually, just arrest on sight).

Number 6 is the part that really scares people, because the TSA refuses to say anything about how the (classified) black box computer system will identify terrorists. It could be based on racial profiling, political ideology, or i-ching and no one would ever know.

There’s a lot of speculation that the whole “airline security” story is just an excuse to collect travel information from everyday citizens for use in something akin to the Total Information Awareness project that was just killed (or at least mostly just killed) by Congress last week. I’m of two minds on that theory. On the one hand, I can’t believe the people at the TSA would really be so stupid as to think something like CAPPS-II would work for the stated purpose, so they must have ulterior motives. On the other hand, maybe I’m being too generous and they really are that stupid, or at least have been deceived by people a little too high on their own technology hype. Of course, there might be a bit of both going on here.

Too many details are left out of the TSA’s description of CAPPS-II to do a full evaluation, but even with what they’ve disclosed there are some huge technological issues:

  • The commercial database step (#4) is to verify that you are who you say you are. The classified black-box step (#6) is to verify that the person you say you are is not a terrorist. This means a terrorist only has to thwart one of the two checks: he either steals the identity of a mild-mannered war hero who is above suspicion, or he gives his real identity and makes sure he doesn’t raise any red flags himself. Since no biometric info (photo, fingerprints, or the like) is used, it would be trivial to steal someone else’s name, address, phone number and birth date and forge a driver’s license for the new identity.
  • Like all automatic classifiers, CAPPS-II needs to be tuned to trade off the number of false positives (innocent people arrested) vs. false negatives (terrorists let through with just a cursory search). Make it too sensitive and every third person will trigger a request for a full search (or worse, arrest), slowing down the security lines. Make it too lax and terrorists will get through without giving up their nail files. The trouble is that airports screen over a billion people a year, and yet even with our supposed heightened risk these past two years far fewer than one in a billion is a terrorist who plans to hijack a plane. Given those numbers, even if our CAPPS-II system correctly identified an innocent person 99.99999% of the time, we would still arrest 1000 people per year due to false information. And with a 99.99999% accuracy requirement on false positives, the odds are good that even Jedi-mindreading alien technology won’t have a great false-negative rate. This isn’t to say risk-assessment has no effect — it may still give better odds than the system we use currently — but most of the benefit from our security screening comes from the added random risk of being caught that a terrorist faces. And that brings us to the third technical problem: intelligent opponents.
  • Standard classification is a pattern recognition problem. A computer is given large amounts of data and expert knowledge, and tries to predict what class a sample (in this case, a passenger) falls into. Classification of intelligent adversaries is different though — it leaves the realm of normal pattern recognition and enters into game-theory. Once this happens it’s a constant arms (and intelligence) race: terrorists commit 9/11 with one-way tickets, so we double-search people with one-way tickets. So all but the stupidest of terrorists now buy round-trip tickets, thus giving them even better than random chance to get through with just a once-over. Of course, we know that’s what they would do, so we should switch to letting one-way tickets through and double-search round-trip tickets, at least until the terrorists catch on and change their plans. (Surely I cannot choose the wine in front of me.) There is a solution to all this madness: completely random selection of passengers for extra screening cannot be gamed in this way. Anything else and it become a question of who can figure out the other side’s profile faster, and given an intelligent foe who can probe the system to his heart’s content, I know who I’d bet on in that race.

Given that Congress has just moved to delay CAPPS II until the General Accounting Office makes an assessment, I can only hope they’ll have similar questions and concerns. This system is either lunacy or a boondoggle to keep a database on the travel habits of every single American — neither is a comforting option.