Stephen E. Fienberg
Maurice Falk University Professor of Statistics and Social Science
Department of Statistics
Center for Automated Learning and Discovery
Center for Computer and Communications Security
Carnegie Mellon University
Committee to Review the Scientific Evidence on the Polygraph
National Research Council
The National Academies
Subcommittee on Energy
Committee on Energy and Natural Resources
September 4, 2003
Mr. Chairman, and Senators. I am pleased to appear before you this morning. I am Maurice Falk University Professor of Statistics and Social Science, in the Department of Statistics, the Center for Automated Learning and Discovery, and the Center for Computer and Communications Security, all at Carnegie Mellon University. I also served as the Chair of the National Research Council's Committee to Review the Scientific Evidence on the Polygraph. Accompanying me today is Dr. Paul Stern, who served as the Study Director for the committee. The committee's report, The Polygraph and Lie Detection, which was released last October reviewed the scientific evidence underlying the use polygraphs for security screening of employees at the national laboratories. It also considered the potential alternatives to polygraph testing for the detection of deception. My testimony today is based on that report and its implications for the Department of Energyís policy on polygraph screening..
My draft proposed testimony was prepared before I was aware of the testimony of Deputy Secretary McSlarrow,. But he had the courtesy to alert me to the changes he planned to propose this morning and thus I will attempt to react to this shift in policy.
When one devotes the better part of two years to an enterprise, such as my colleagues in the NRC committee did to the preparation of our report on the polygraph, having the import of ones work distorted or ignored, as appeared to be the case when the Department of Energy issued its proposed regulations in April, is disheartening at best. Thus I am especially gratified by the remarks of Deputy Secretary McSlarrow this morning, since they not only represent a shift in thinking at the Department of Energy but one that appears to be strongly influenced by parts of our report.
The NAS-NRC Committee Report
The committeeís report begins by setting the current debate over the efficacy of polygraph testing in the context of the mystique that surrounds itóthis includes a culturally shared belief that the polygraph is nearly infallible. As we note in the report, the scientific evidence strongly contradicts this belief.
Let me now briefly summarize the committeeís principal conclusions:
1. The scientific evidence supporting the accuracy of the polygraph to detect deception is intrinsically susceptible to producing erroneous results.
2. In populations of naïve examinees untrained in countermeasures, specific incident polygraph tests can discriminate lying from truth telling at rates well above chance, though well below perfection. But the accuracy of the polygraph in screening situations is almost certainly lower.
3. Basic science gives reason for concern that polygraph test accuracy can be degraded by countermeasures.
4. The scientific foundations of polygraph screening for national security are weak at best and are insufficient to justify reliance on its use in employee security screening in federal agencies.
5. Some potential alternatives to the polygraph show promise, but none has been shown to outperform the polygraph and none is likely to replace it in the short term.
I have appended the Executive Summary of the report to this testimony as it contains the specific wording of these conclusions and details explaining how the committee reached them.
The DOE Proposed Regulations from April, 2003
In April of this year, the Department of Energy released new draft regulations on its program of polygraph testing of eight classes of federal employees and contractors who have access to classified information. The new regulations would continue a policy that was set in place in 2000 but suspended in 2001, pending the report of the NAS-NRC committee. Thus it might be natural to ask what in the report is of direct relevance to the proposed regulations.
Let me return to the specific wording of the committeeís recommendation on the matter of security screening:
Polygraph testing yields an unacceptable choice for DOE employee security screening between too many loyal employees falsely judged deceptive and too many major security threats left undetected. Its accuracy in distinguishing actual or potential security violators from innocent test takers is insufficient to justify reliance on its use in employee security screening in federal agencies.
How did DOE square these conclusions with its plan to continue the polygraph policy unchanged? It said that the polygraph, though "far from perfect, will help identify some individuals who should not be given access to classified data, materials, or information." This may be true, but two other things about polygraph screening are also true that should have given DOE pause.
First, for every such individual identified, hundreds of loyal employees will be misidentified as possible security threats. Our report make clear that, given DOEís own expected rates of security violations, someone who "fails" the DOE polygraph screening test has over a 95 percent chance of actually being a truthful person. Unfortunately, the DOE doesn't have any other scientific tool to fall back on to distinguish the security violators from the innocent people falsely accused.
Second, any spy or terrorist who takes the DOE's polygraph test is far more likely to "pass" the test than to "fail" itóeven without doing anything to try to "beat" the test. Efforts at so-called countermeasures are likely to increase further the chances that a committed spy or terrorist will "beat" the test. This is the most serious problem with polygraph screening, especially in these times of terrorist threat: the possibility that security officials will take a "passed" polygraph too seriously, and relax their vigilance.
The original DOE regulations give every indication that the agency has just this sort of overconfidence in polygraph tests that give "passing" results. The proposed regulations say, "DOE's priority should be on deterrence and detection of potential security risks with a secondary priority of mitigating the consequences of false positives and false negatives." The committee found little scientific evidence to support the effectiveness of the polygraph in this regard. Moreover, it concluded that the consequences of false negative testsótests that deceivers "pass"óshould have top priority, because it is those test results that leave the nation open to the most serious threat, from people whose continued access to sensitive information is justified because they "passed the polygraph."
By continuing to rely on polygraph screening just as before, the DOE in its original response to our report was doing more for the appearance of security than for the reality.
Fortunately, the new proposals rely less on the polygraph than before and thus have moved towards a position that could be viewed as consonant with the conclusions in our report. Let me highlight some of these changes.
The New DOE Position
While I have had only a limited amount of time to assess the changes signaled by Deputy Secretary McSlarrowís testimony there are at least six key features that are worth highlighting as they relate to issues addressed in our report.
There may still be a place for polygraph testing in the DOE labs, for investigations of specific incidents and for a small number of individuals with access to the most highly sensitive classified information, if the testís limited accuracy is fully acknowledged. DOE, in announcing its new plans, does propose to limit the number of polygraphs, but I found the proposed numbers remarkably high. Until DOE is able to further curtail its reliance on the polygraph, the broad use of this flawed test for screening will probably do more harm than good. National security is too important to be left to such a blunt instrument.
Let me conclude by reminding you that polygraph testing now rests on weak scientific underpinnings despite nearly a century of study. And much of the available evidence for judging its validity lacks scientific rigor. Our committee sifted the existing evidence and our report made clear the polygraph's serious limitations in employee security screening. Searching for security risks using the polygraph is not simply like search for a needle in a haystack. It is true that, of the large groups of people being checked, only a tiny percentage of individuals examined are guilty of the targeted offenses. Unfortunately tests that are sensitive enough to spot most violators will also mistakenly mark large numbers of innocent test takers as guilty. Further, tests that produce few of these types of errors, such as those currently used by the DOE, will not catch most major security violatorsóand still will incorrectly flag truthful people as deceptive. Thus the haystack analogy fails to recognize the unacceptable trade-off posed by these two types of errors.
Our committee concluded that the government agencies could not justify their reliance on the polygraph for security screening. The original proposed DOE regulations issued in April appeared to disregard our findings and conclusions. Todayís testimony by Deputy Secretary McSlarrow signals a change in direction that I applaud. The new proposals seem more consistent with the scientific evidence documented in our report. My hope is that, as these proposals get refined and developed in the form of new regulations, the DOE continues along the path of reducing its reliance on the polygraph and developing a scientifically justifiable security program. As a nation, we should not allow ourselves to continue to be blinded by the aura of the polygraph. We can and should do better.
The National Research Council stands ready to assist the department, both in the short term as the regulations are developed, and in the longer term when the careful evaluation of their impact needs to be carried out.
I would be happy to answer your questions and amplify on these comments.
AntiPolygraph.org Home Page > Reading Room