Kaiser Fung on Lie Detectors

AntiPolygraph.org has received a complementary copy of statistician Kaiser Fung’s new book, Numbers Rule Your World: The Hidden Influence of Probability and Statistics on Everything You Do (New York: McGraw Hill, 2010), a short primer on statistics written for a general audience.

In Chapter 4, Fung addresses the trade-off between false positives and false negatives in diagnostic testing, using as examples drug testing of athletes, polygraph testing of criminal suspects, job applicants, and employees, and data mining for terrorists. Fung explains how altering decision thresholds to lower the rate of false positives necessarily increases the rate of false negatives, and vice versa, and how a low base rate of the thing being tested for in the population being tested can make attempts to detect it impractical.

With regard to polygraphy, Fung in particular focuses on the Preliminary Credibility Assessment Screening System (PCASS), the hand-held lie detector developed by the U.S. Department of Defense to screen locally hired workers and suspected insurgents in Iraq and Afghanistan. Fung shows that even if we were to assume a 90% accuracy rate for the procedure (an assumption not warranted by scientific evidence), given the low incidence of insurgents seeking employment with the US armed forces, many false positives can be expected for every true positive. (Of course, there is also the problem of countermeasures: any insurgents among the hiring pool can readily fool the PCASS.)

Fung closes his treatment of PCASS with a take-home quote from Dr. Stephen Fienberg, who chaired the National Academy of Sciences panel that in 2002 authored a landmark report on polygraphy: “It may be harmless if television fails to discriminate between science and science fiction, but it is dangerous when government does not know the difference.”

Fung also tells the story of Jeffrey Deskovic, from whom interrogators extracted a false confession to the murder of a high school classmate after he failed a polygraph test. After 16 years in prison, Deskovic was vindicated by DNA evidence and released. Fung concludes, “Statistical analysis confirms that many more Deskovics, perhaps hundreds or thousands a year, are out there, most likely hapless.”

A complete review of Numbers Rule Your World is beyond the scope of this blog, but for further commentary, see reviews by Wayne Hurlbert, Andrew Gelman, and Christian Robert.