Your 99 percent accurate test will perform with 99.99 percent inaccuracy.
That's the paradox of the false positive.
When you try to find something really rare, your test's accuracy has to match the rarity of the thing you're looking for. If you're trying to point at a single pixel on your screen, a sharp pencil is a good pointer: the pencil-tip is a lot smaller (more accurate) than the pixels.
But a pencil-tip is no good at pointing at a single atom in your screen. For that, you need a pointer — a test — that's one atom wide or less at the tip.
This is the paradox of the false positive, and here's how it applies to terrorism:
Terrorists are really rare. In a city of twenty million like New York, there might be one or two terrorists.
Maybe ten of them at the outside. 10/20,000,000 = 0.00005 percent.
One twenty-thousandth of a percent.
That's pretty rare all right.
Now, say you've got some software that can sift through all the bank-records, or toll-pass records, or public transit records, or phone-call records in the city and catch terrorists 99 percent of the time.
In a pool of twenty million people, a 99 percent accurate test will identify two hundred thousand people as being terrorists. But only ten of them are terrorists.
To catch ten bad guys, you have to haul in and investigate two hundred thousand innocent people.
Guess what? Terrorism tests aren't anywhere close to 99 percent accurate.
More like 60 percent accurate. Even 40 percent accurate, sometimes.
What this all meant was that the Department of Homeland Security had set itself up to fail badly. They were trying to spot incredibly rare events — a person is a terrorist — with inaccurate systems.
Is it any wonder we were able to make such a mess?
I stepped out the front door whistling on a Tuesday morning one week into the Operation False Positive.
I was rockin' out to some new music I'd downloaded from the Xnet the night before — lots of people sent M1k3y little digital gifts to say thank you for giving them hope.
I turned onto 23d Street and carefully took the narrow stone steps cut into the side of the hill. As I descended, I passed Mr Wiener Dog.
I don't know Mr Wiener Dog's real name, but I see him nearly every day, walking his three panting wiener dogs up the staircase to the little parkette. Squeezing past them all on the stairs is pretty much impossible and I always end up tangled in a leash, knocked into someone's front garden, or perched on the bumper of one of the cars parked next to the curb.
Mr Wiener Dog is clearly Someone Important, because he has a fancy watch and always wears a nice suit. I had mentally assumed that he worked down in the financial district.
Today as I brushed up against him, I triggered my arphid cloner, which was already loaded in the pocket of my leather jacket. The cloner sucked down the numbers off his credit-cards and his car-keys, his passport and the hundred-dollar bills in his wallet.
Even as it was doing that, it was flashing some of them with new numbers, taken from other people I'd brushed against. It was like switching the license-plates on a bunch of cars, but invisible and instantaneous.
I smiled apologetically at Mr Wiener Dog and continued down the stairs. I stopped at three of the cars long enough to swap their FasTrak tags with numbers taken off of all the cars I'd gone past the day before.
You might think I was being a little aggro here, but I was cautious and conservative compared to a lot of the Xnetters. A couple girls in the Chemical Engineering program at UC Berkeley had figured out how to make a harmless substance out of kitchen products that would trip an explosive sniffer.
They'd had a merry time sprinkling it on their profs' briefcases and jackets, then hiding out and watching the same profs try to get into the auditoriums and libraries on campus, only to get flying-tackled by the new security squads that had sprung up everywhere.
Other people wanted to figure out how to dust envelopes with substances that would test positive for anthrax, but everyone else thought they were out of their minds.
Luckily, it didn't seem like they'd be able to figure it out.
I passed by San Francisco General Hospital and nodded with satisfaction as I saw the huge lines at the front doors.
They had a police checkpoint too, of course, and there were enough Xnetters working as interns and cafeteria workers and whatnot there that everyone's badges had been snarled up and swapped around.
I'd read the security checks had tacked an hour onto everyone's work day, and the unions were threatening to walk out unless the hospital did something about it.
A few blocks later, I saw an even longer line for the BART. Cops were walking up and down the line pointing people out and calling them aside for questioning, bag-searches and pat-downs.
They kept getting sued for doing this, but it didn't seem to be slowing them down.
I got to school a little ahead of time and decided to walk down to 22nd Street to get a coffee — and I passed a police checkpoint where they were pulling over cars for secondary inspection.
School was no less wild — the security guards on the metal detectors were also wanding our school IDs and pulling out students with odd movements for questioning.
Needless to say, we all had pretty weird movements. Needless to say, classes were starting an hour or more later.
Classes were crazy. I don't think anyone was able to concentrate.
I overheard two teachers talking about how long it had taken them to get home from work the day before, and planning to sneak out early that day.
It was all I could do to keep from laughing.
The paradox of the false positive strikes again!
Sure enough, they let us out of class early and I headed home the long way, circling through the Mission to see the havoc.