George,

Matte's site has a downloadable file that is a supplement for his book.

I believe Gordon was directing his answer to your specific question about the known field sensitivity and specificity of CQT polygraph.   

I will reiterate what both Gordon and I have stated, sensitivity and specificity are established in a controlled research setting and the outcome dependent on the variables measured and accounted for.  I am not avoiding your question.  I plan to respond to your more recent post in the separate thread where this discussion belongs.

J.B.,

Where on the website http://www.mattepolygraph.com did you find a theoretical explanation of CQT polygraphy?

You also write:

Quote:


To whom is the sensitivity and specificity of CQT polygraphy known? It appears to be unknown to the U.S. Government. (As Gordon wrote, "...I know of no official government statistic regarding sensitivity and specificity.")

And what precisely is the "known" sensitivity and specificity of CQT polygraphy, J.B.? This is a question you have avoided answering in the message thread The Scientific Validity of Polygraph, which you initiated. In addition, please specify on the basis of what peer-reviewed research the sensitivity and specificity of CQT polygraphy is known.

George,

Specificity and sensitivity can be expressed in a controlled setting where the number of cases of each are known and can be confirmed.(i.e.. 50 truthful and 50 deceptive)  It can also be expressed in a field setting with a given sample of cases where there is a known outcome of the sample. If you set base rates based on a human assumed outcome of a population, the true percentages are bound to be skewed.  The fact is in a forensic field science there is no way of telling how many of one or another will come in a given day, month, year, etc...  

For a simple break down of the Theory of Polygraph go to: http://www.mattepolygraph.com/ ;

Quote:

 
As for first part of your statement, there is a known sensitivity and specificity for polygraph that has been established and proven through peer-reviewed scientific research.  The later part of your statement has little or nothing to do with the prior portion.  As has been said before, this is simply a statistical probability that a given sample is or is not.  As for ones' ability to testify based on statistical probability, DNA experts testify to just that.  They say that based on a known data base and assumed probability the chances of someone having the same DNA is 1 in (enter probable number).  Yet there is no way of being certain what the true percentage is unless the entire world were to submit to DNA testing, the samples are collected, processed, entered into a world database, and all samples are checked one against another.

Note: this post was edited to correct a formatting problem; except for this note, no words were added, deleted, or changed. --AntiPolygraph.org Administrator

Gordon,

Quote:


The diagnostic test for which no sensitivity and specificity can be specified is a very strange one indeed. All the more strange in that it seems to have no compelling theoretical explanation, either.

If no sensitivity or specificity can be specified for CQT polygraphy, then how can any predictive validity be determined for it when applied as a screening test to any particular population group? Or is that somehow not important?

The answers to these questions are crucial to any rational consideration of what is more effective than the polygraph.

Quote:


If the sensitivity and specificity of CQT polygraphy are not known (and, as you admit, not likely to be known in the foreseeable future), then is it not a fraud on the court for any polygrapher to testify, based on analysis of the charts collected during a CQT polygraph examination, as to the probablity of a subject's having been deceptive?

J.B. McCloughan wrote on Apr 13^th, 2002 at 5:09am:


The effects of amplifiers, filters, and sampling procedures
have clearly adulterated the data.... This is not a scientific project... there is no reason to believe that the [algorithm] developed at APL has any power whatsoever to discriminate between truthful and deceptive subjects... In summary, the contractors [APL] have developed, delivered, and sold an algorithm to separate DI from NDI subjects that has no demonstrated validity...

The entire article is here

George,

The "mathematical certainty", as you so call, is a computerized score based on confirmed polygraph data.  The data collected is then compared to the known data to arrive at a 'probability', not 'certainty', of deception, inconclusive, or truthfull.

George,

You're looking for a simple answer.  What I'm trying to convey is that there is none.  We can talk about the accuracy of examiner decisions (which is what the 1969 Bersh study examined), the accuracy of blind chart evaluations (which is what the more recent studies examine), the accuracy of decisions on individual questions within a multi-issue test (as opposed to overall decisions based on the test as a whole), etc.  The sensitivity and specificity vary according to what criterion you are using as a decision and how the CQT is employed. 

At the risk of repeating myself, I must stress that one of the more important variables is the definition of deception.  You  feel more comfortable with confining it to lies of commision, which is what most of the laboratory research has focused on, but which ignores the complexities of many real-life situations.

To answer your question as best I can, I know of no official government statistic regarding sensitivity and specificity.  I doubt that one exists, nor is likely to exist in the foreseeable future.  A lot depends on the situation in which the CQT is used, e.g. a single issue criminal examination versus a multi-issue screening examination.

The probabilities cited in the Nicholson case refer to how Nicholson's reactions compared to the total number of verified truthful and deceptive sets of charts in the data base used by that particular algorithm.  The size and make-up of the data base has varied over the years as more and more verified cases are added to it.

Peace,

Gordon

Quote:


Gordon, CQT polygraphy is represented as a science-based diagnostic test for the detection of deception. If that's true, then it follows that, independent of the base rate of deception in any particular population group being tested, it must have a knowable sensitivity and a specificity for the detection of deception. I'm asking you to state what that sensitivity and specificity are.

For example, in the Affidavit in U.S. v. Harold J. Nicholson which you cited above, the probability of deception is stated with mathematical certainty:

Quote:


It follows that the U.S. federal polygraph community has some sense of what the sensitivity and specificity of CQT polygraphy are. What are they?

George wrote:

Quote:


The commonsense understanding you described is limited to the clear-cut, black-and-white situation in which the person is either being completely truthful or completely deceptive (i.e. lies of comission).  This is often found in laboratory research paradigms, but ignores many real-world situations frequently found in security screening, which can be rife with ambiguities.  This is why the examiner often uses the term DI (deception indicated) or NDI in criminal investigations, yet uses SR (significant response) or NSR to describe the results in screening situations.

The sensitivity and specificity of the polygraph technique depends upon a number of variables, such as how deception is defined, what the test issue is, how many test issues are on a test, the ambiguities inherent in the testing situation, and whether the test is a criminal investigation or security screening.  Can you define your question more precisely?

Peace,

Gordon

Gordon,

Quote:


You left out the important distinction that in particle physics, as in other real scientific disciplines, logical inferences are made on the basis of controlled experiments, unlike in CQT polygraphy, which is completely lacking in any genuine control.

Quote:


You're avoiding the question I posed to you, Gordon. Again, what is the sensitivity and specificity of CQT polygraphy for the detection of deception? The answer is highly germane to any rational discussion of how effective the polygraph is for catching spies (or for any other application).

You also wrote:

Quote:


The person who is being completely truthful regarding the relevant questions may well react more strongly to the relevant questions, just because, for example, he fears the consequences of not being believed with regard to the relevant questions. If a person reacts more strongly to the relevant questions than to the "control or comparison questions, how can the polygrapher determine whether the person was anxious-but-truthful or anxious-and-deceptive? Your response above doesn't answer this question.

George,

You asked:

Quote:


That is the whole purpose of the comparison questions.  If the person knows he is being completely truthful regarding the relevant questions, but has doubts as to whether he can be completely truthful when answering the comparison questions with a simple "No," he tends to react more to the comparison questions than the controls.  They serve to protect against that cause of false positive errors.

Peace,

Gordon

George,

As to whether the polygraph detects deception, there seems to be a problem with semantics.  While the polygraph measures arousal rather than deception per se, this does not mean that the polygraph technique does not detect deception.  The evidence for deception is indirect, not direct.  In a sense, it is analagous to determining the presence of a subatomic particle by the trail it leaves behind in a cloud chamber in a physics lab, since the particle cannot be seen directly.

Pragmatically, in a criminal investigation when the examiner concludes that a person is DI,  absent any verifiable explanation by the subject to the contrary, the most reasonable explanation as to why the person responded as he did is that the person was being deceptive.  Decades of research involving mock crimes supports that these decisions are accurate at levels far above chance.  As with any inference or any diagnostic procedure, there is always the possibility of error.

Peace,

Gordon

Gordon,

Quote:


I agree with you that the polygraph does not detect deception per se, even as it does not detect spies per se. And yet the federal polygraph community represents to the public that polygraph "tests" detect deception. The notion is implicit in the official DoDPI term for polygraphy: "psychophysiological detection of deception."

Would you agree that CQT polygraphy also lacks "control" within the scientific meaning of the word? How can the polygrapher possibly know that he has "so structure[d] the testing environment to control extraneous responses?" That is, how can the polygrapher distinguish between the anxious but truthful subject and the anxious and deceptive subject?

Quote:


One of the standard questions asked in counterintelligence-scope polygraph screening is something similar to, "Did you ever provide classified information to any unauthorized individual?" Clearly, the person who disclosed classified information to his (uncleared) girlfriend is being deceptive when he answers that question, "No."

Quote:


Implicit in the above statement is the unfounded assumption that the polygraph examiner can actually distinguish between a lying and truth-telling in the first place.

Quote:


I don't think it has been established that the fact that a question causes a person to think of something specific every time it is asked will necessarily result in a reaction to the question that is measurable by the polygraph instrument.

With regard to the wisdom of making admissions to relevant questions, our advice in The Lie Behind the Lie Detector is not strictly speaking to "make no admissions whatsoever" with regard to the relevant questions but rather to have an explanation prepared in advance that cannot be turned into a damaging admission, as explained in the subchapter "To Explain or Not to Explain Responses to Relevant Questions" (pp. 134-35 of the 2nd ed.).

You earlier started a message thread titled CM advice on dealing with DI results misguided. I don't think that you or the anonymous polygrapher whose message you forwarded offered a convincing argument that a subject accused of deception in the course of a polygraph interrogation would benefit by making an admission against interest.

Quote:


In all the polygraph studies I've read, "deception" seems to have meant that the subject answered the question "untruthfully," in the common sense of the word. For the sake of argument, let's say that the person who provided classified information to his girlfriend who has no security clearance is asked, "Did you ever provide classified information to an unauthorized individual?" If he answers "No," he's deceptive. Similarly, if a person who has robbed the 1st National Bank is asked "Did you rob that bank?" (after it has been made clear to him which bank "that bank" is) answers "No," he's deceptive.

With this commonsense understanding of "deception" in mind, what is the sensitivity and specificity of CQT polygraphy for the detection (or inference, if you will) of deception (i.e., whether a person has answered a question truthfully)?

Sorry for the delay in responding to all of your replies.  I've been out of town.  You all raise a number of points.  Let me address one by George first.  George said:

Quote:


The polygraph does not detect deception per se, for as you have often stated, there is no such thing as a lie response, a response which occurs only when a person lies, and never under any other circumstance.  The human body just isn't built that way.  The polygraph records any short term physiological arousal to the questions.  A decision of "deception indicated" is an inference, based upon the elimination of other sources of reactions.  To the extent that the examiner can so structure the testing environment to control extraneous responses, the decisions are likely to be accurate.  

One of the primary sources of reactions is when the subject may be answering the question truthfully in the literal or technical sense, i.e., he is not a spy, and yet the question makes him think of something specific he has decided not to tell the examiner (such as "In order to impress her, I told my girl friend about a classified project I worked on").  The polygraph examiner cannot distinguish between a lie of commision ("No") and a lie of omission ("No, but....") because the body itself does not make that distinction.  

As long as the question causes the person to think of something specific every time it is asked, he will react to the question.  The associated thoughts being concealed often do not rise to the level of disqualifying an applicant from employment or an employee from continued employment, yet you advise people to make no admissions whatsoever relating to the relevant questions.  This makes it more difficult for the person in that type of situation to be cleared on the polygraph, for the examiner is unable to reword the question to remove that source of reaction.  I believe your advice ill-serves the many people who face that dilemma.

As for your question about the sensitivity and specificity of the polygraph in detecting deception, a lot depends upon your definition of deception.  When they are intended to make yourself look better than you actually are, as in the example of the girl friend, do you consider deliberate omissions to be deception?  I believe most psychologists who study deception do.  This concept is one of the key issues in defining true versus false positives.

Peace,

Gordon

Quote:


Dr. Barland,

My position is the latter, Mr. Lee was branded a LIAR. He PASSED his tests, but the FBI insisted that he was "deceptive".

If he truly violated National Security, then yes he should be punished. My personal opinion is he didn't sell or pass anything to foreign agents or governments, he simply took sensitive work home, that in itself hardly warrants the treatment he was subjected to.

One last comment Dr., Yes the process missed Ames and Montes, and they continued to spy. So why do these high profile criminals slip through the cracks when you tout the polygraph as highly "accurate"? Could it be possible that they knew countermeasures?

Fred F.

Again with regard to how spies are caught, at the U.S. Senate Committee on the Judiciary's 9 April 2002 hearing on "Reforming the FBI in the 21st Century: The Lessons of the Hanssen Espionage Case," former FBI and CIA Director William Webster stated (even as he advocated expanded reliance on polygraphy):

Quote:

Gordon, 

Some additional comments on other things you wrote...

Quote:


Polygraph examinations are purported to detect deception, not spies per se. What is the sensitivity and specificity of CQT polygraphy for the detection of deception?

You also wrote:

Quote:


That must be a very comforting non-falsifiable ad hoc hypothesis for polygraphers whose consciences might otherwise bother them about the innocent people they harm: "Well, if the subject wasn't lying about the relevant questions, he must have been holding back about something." Ed Curran expressed similar thoughts on the CBS 60 Minutes II segment, "Final Exam":
Quote:


Where's the proof?

You also wrote:

Quote:


Edward J. Curran was a policymaker who doubtless had much more complete information than that available to the general public. He was the FBI's senior counterintelligence agent at the time of CIA spy Aldrich Ames' arrest, at which point he was seconded to the CIA, where he took charge of counterintelligence. While at CIA, Curran presided over a polygraph jihad that left hundreds of CIA employees unable to "pass" their polygraph "tests." Later on, Curran headed the Department of Energy's Office of Counterintelligence, where he advocated a greatly expanded polygraph screening program. And yet Edward J. Curran doesn't seem to be aware of any spy who was ever caught by a polygraph screening exam. The following exchange occurred during the CBS 60 Minutes II segment "Final Exam":

Quote:

Gordon,

Quote:


It would be more correct to say that I am not aware of the existence of any screening measure that is capable of catching (detecting) spies at better than chance levels. This is not to say that no screening measure has any utility in making personnel security determinations. Certainly, such measures as financial record checks, criminal records checks, background investigations, and yes, polygraph interrogations (in the form of admissions obtained from naive subjects who don't understand that polygraphy is a fraud) have some utility.

Your initial characterization of my position regarding polygraphy was more to the point: "If I understand your position correctly, you believe that the polygraph is counterproductive to the national security  because spies can so easily be trained to beat it, it gives a false sense of security to security personnel, and it causes too many false positives."

Quote:


Whether polygraphy depends on a subject's belief in it is germane to the discussion at hand. My understanding of CQT theory is that the expectation that the truthful will respond more strongly to the "control" questions and the deceptive to the relevant questions depends on the subject's belief that deception will be detected, and that the "stim test" is used to reinforce that belief.

In any event, it has been argued that the main utility of polygraph screening is the admissions obtained therefrom. For example, former CIA and DOE counterintelligence chief Edward J. Curran, explaining on the CBS 60 Minutes II segment "Final Exam" what good the polygraph is, after acknowledging that it is not scientific, stated:

Quote:


It seems logical to suppose that the subject who doesn't believe in the lie detector will be less likely to make damaging admissions/confessions than the subject who does.

Perhaps if you ever get around to addressing the questions raised in the message thread Theory of CQT-Polygraphy (Attn: Gordon Barland), which you've shrugged of for nearly a year, you could provide a theoretical explanation for why belief in the polygraph does not affect accuracy.

You also stated:

Quote:


To what extent does our advice damage the national security? I think you confuse the vested interests of the federal polygraph community with the national interest. It seems to me that public information about countermeasures can only "damage" the national security to the extent that national security officials place any reliance on the pseudoscience of polygraphy. We've documented some of the damage done to national security by this misplaced reliance in Chapter 2 of The Lie Behind the Lie Detector. I think that far from damaging national security, our publicly pointing out that the polygraph emperor is naked will ultimately strengthen it.

I have started a new thread under "Polygraph Procedure" entitled:

"Did the Polygraph catch Nicholson?"

Dr. Barland suggested it did.  Check out remarks then Director Deutch made after Nicholson's arrest and judge for yourself.

Quote:


Mr. Mallah has already questioned the validity of your proof. I would have to add a healthy degree of doubt as well. There is no indication when the 'computerized review' took place on Mr. Nicholson's polygraph test. I will contact the Special Agent who's sworn affadavit you quote in order to ascertain when the review took place. In the meantime, at least one investigative journalist also bolsters the assertion that you are wrong on whether or not a polygraph exam caught Mr. Nicholson. Susan McCarthy writes in part:

Spies Aldrich Ames and Harold J. Nicholson passed routine CIA polygraph exams. In fact, it's not clear that any spies have ever been caught by polygraph testing. "It's inconceivable that [a spy] won't know how to beat the test," Lykken says. "So the spy won't be caught, and a few innocent people will have their careers ruined, and the secretary of energy will say that we did everything we could. I think it's a scandal."

The complete article is here.

I look forward to other cases you cite that demonstrate a failed polygraph was the precursor to rooting out a spy.

Quote:


Dr. Barland, don't put words in my mouth! I merely asked for actual case evidence in which a spy had been caught by first failing a polygraph.

Gordon,

I agree that there is some complexity to the analysis, but it is not entirely subjective.  Certainly we know that the polygraph has done grave harm in the form of Ames, Chin, Montes, and CIA intelligence operations in Cuba (even before Montes).  This is certainly a high pile of garbage.  All of it can't be blamed on the polygraph, but some of it can.  It is going to take a lot of construction work to erect a benefits pile that reaches higher than that.  So far I have not seen them, nor have you really detailed any.


Do you have any supporting data on these security violations?  My experience and assessment is that examiners with "deceptive" charts want to validate them in any way possible with a confession at best, but they'll take a damaging admission on security, even if they have to manufacture one.  And I can tell you this: there is nary a soul in the intelligence community who can withstand a full-on scrutiny as to whether he or she has ever violated some security procedure at some point in their career.  For example, people talk to their spouses.  Even the most scrupulous inadvertently reveal things that a foreign intelligence service might find of some valule.  These and other security related peccadillos happen all the time in hundreds of innocuous ways.


While the information of policy makers may be more complete (if they take advantage of their access), their analysis is heavily politicized.  Theirs is not a dispassionate analysis, nor can scientific integrity simply defer to their judgment.

As a general comment, I am troubled by what I see as you hiding behind claims of national security, protecting classified information, et al.  As George has pointed out more eloquently than I'm about to do right now, a scientific technique that shies away from peer reviewed scrutiny and critique can't claim to have much scientific integrity.  "Trust us, it works, and we have the supporting data but can't tell you" does not pass scientific muster, even if its concerns are legitimate.

Also troubling is that it strikes a decidedly anti-democratic tone.  Where's the accountability?  Are we to rely on the word of a small group of people who claim that their machine detects lies and protects national security, but when pressed for further elaboration, claim that they can't reveal it on national security grounds, we just have to believe them.  This is dark.  

Thank you for participating in this discussion.

Mark,

Your view is that the polygraph harms national security because the overall impact of the errors (false positives and negatives) is greater than the overall benefit (true positives and negatives).

As you well know, this is an extremely complex issue that cannot be easily resolved.  Lots of the data necessary to make these calculations is unavailable, and likely to remain so.  Moreover, such calculations are inevitably subjective.  How many false positives does it take to outweigh one true positive?  You might as well ask how many stars are in the sky.

Moreover, merely defining how the polygraph should turn out in order to be deemed a correct versus an incorrect outcome, is often difficult, for there are many ambiguities in real-life screening situations.  For example, if a person is not a spy, but has knowingly and deliberately mishandled classified information in a significant way which he is deliberately concealing from the examiner, should the polygraph show he is telling the complete truth, or should it show that he is concerned about the relevant issue on the test?  I believe this is a source of many claims of a false positive by the poor person caught in that dilemna and who chooses not to discuss the problem with the examiner when the reactions are brought to his attention. 

All I can say is that the polygraph has caught many more spies than the critics are aware of.  The policy makers, looking at the cost/benefit information available to them, obviously conclude that the benefits outweigh the costs.  Their information is obviously incomplete as well, but it is likely more complete than that available to the general public.

It will be interesting to see what the National Science Foundation committee concludes.

Peace,

Gordon

Gordon,

Your information about Nicholson is incorrect.  CIA Director Deutch was asked directly whether suspicion fell on Nicholson because of the polygraph, and he hedged quite a bit.  That suggests to me that it was not the polygraph that caught him (since he did not affirm that it did when given the opportunity), but some other source he did not want to reveal.  Maybe a recruitment or defector  

I don't have his quote available with me now, but I will attempt to dig it up.  Apologies in advance because I lost the newspaper source in which it appeared.  I imagine it can be tracked down.

Also, nothing in the FBI affidavit indicates whether the polygraph confirmed or first uncovered Nicholson's activities.  I think your suggestion that it first uncovered his activities is completely unsupported by the section you included.

Fred F. wrote that the polygraph missed Ames, Montes, and Lee.

Who can disagree with the statement that the polygraph missed some important spies?  Sure it has, but what diagnostic test is perfect?  The more important issues are what proportion of spies does it catch (the sensitivity of the test), and what proportion of nonspies does it correctly exclude from suspicion (the specificity of the test).

Even if the polygraph could detect only ten or twenty percent of the spies -- and I think it does far better than that, -- what other screening technique has a 10 to 20 percent chance of detecting them within a similar half day period?

To calculate the polygraph's sensitivity and specificity, we need data on how many spies were examined on the polygraph while they were spying, and what the outcomes of those tests were; and how many non-spies were examined, and the outcome of those exams.

The number of spies currently operating who were incorrectly cleared on the polygraph is unknowable, just as we also don't know how many spies are currently operating who have never been examined on the polygraph.  In time, we will become aware of some, but always too late.

The intelligence agencies are understandably reluctant to release information about what led to the detection of a spy, for that type of feedback to the foreign intelligence services can only make it harder to catch their spies in the future.  In some cases it comes out.  In many others it remains highly classified.  This is one reason why some polygraph successes remain unsung.

Incidentally, that same line of reasoning is why I refuse to discuss how Federal examiners are able to detect some unknowable proportion of people who use countermeasures on the polygraph; that would enable the Williamses, Maschkes and Scalabrinis of the world to improve their advice, which would damage the national security even more.

Fred, I've got a question for you:  Ames and Montes continued to spy after their polygraph exams; in my book the polygraph process missed them.  You included Lee in with them.  Is it your opinion Lee was a spy and he passed the polygraph, or that he was not a spy and the polygraph branded him a liar?

Peace,

Gordon