Latest Study Indicates "Lie Behind the Lie Detector" Hurts Innocent, Doesn't Help Guilty

[raymond.nelson]

As I stated in a previous post, scientific studies can expose results which are ancilary to the original purpose of the study, but are still significant in their own right. Given the study in question, it appears that this is the case. I notice you do not challenge my assertion that the study indicates the accuracy of the polygraph (in this test anyway) is no better than 73%. I will take that lack of challenge as agreement to this assertion.

Given that fact then, with an error rate of over 25%, how can any honest reviewer conclude that the study proves anything regarding the effectiveness of countermeasures, when the polygraph itself could not detect blatant deception?

I think you fill find near universal agreement that this study was not intended to and is insufficient to reach any conclusions about the overall accuracy of the polygraph.

Your conclusion that the polygraph could not detect deception is in error.

That analysis revealed that CPSp|t values for guilty participants, M=0.40, SD=0.33,
were significantly lower than were the CPSp|t values for innocent participants,
M=0.72, SD=0.33, F(1; 39)= 9.24, p=.004, h2=0.2. Neither the main effect for
information nor the interaction of guilt and information were significant,
F(1; 39)=0.02, ns, h2=0.01 and F(1; 39)=0.19, ns, h2=0.005, respectively.

These data show the polygraph did detect detection at statistically significant levels. They also show that prior information, as provided, produced no statistically significant results.

If you have read and understood the study, then we might conclude that you are providing that inaccurate and misleading information intentionally, and are therefore in violation of the posting policies. Please be more careful.

r

[nopolycop]

Mr. Raymond:

What I see from this study, is that in 25%+ of the cases, the polygraph did not detect deception.  Certainly better than random chance, but not what I would want to bet my freedom, carreer or reputation on.

With a study that itself shows 25% error, one cannot logically conclude then the underlying premise that countermeasures do not work is true.  

Personally, I don't really care about the countermeasure issue, but just brought up the fact that the study itself is fatally flawed.

[nopolycop]

 (This was a single-issue application of the TES, a screening exam.  However, they ran it as a single-issue test.  We don't do that, for reasons I'd argue would lower accuracy, which is what we see here.  This is either an apple or an orange, but not both.  In any event, it's not correct to compare the two.)

Wouldn't it make sense that if the error rate on a single issue test was 25%, that the likelyhood of a higher percentage of error would follow, if more than a single issue was being tested?

[raymond.nelson]

nope,

You seem still to be misunderstanding. This study was not intended to determine accuracy, and the observed error rate in this study is not a fatal flaw - unless one has advance intentions of rejecting the opportunity to learn from it.

This study could not reject the null hypotheses that there would be no difference between results of truthful and deceptive persons when they are provided the opportunity to acquire prior knowledge from tbltd. On the other hand, data do support accepting the null hypothesis that there is no significant difference within the scores of deceptive and truthful persons who were provided the opportunity to acquire prior knowledge from tbltd.

What this actually says is that the data from this study could not support the hypothesis that countermeasures work. That alone doesn't mean they don't work, but it sure doesn't mean they do. Honts and Alloway seem to have known this is not a simple problem, and they would need more information. So, they seem to have planned for that.

Hont's and Alloways covariance ratios, as illustrated earlier in this thread (sometimes referred to as correlation ratios, but the the term "correlation" in this useage tends to prompt erroneous assumptions and expectations around the concept of correlation - its about covariance), suggest two things: 1) the polygraph did detect deception and truthfulness at statistically significant rates in this laboratory study, and 2) it still detected deception and truthfulness at statistically significant rates when the laboratory subjects were provided the opportunity to educate themselves in advance, using tbltd.

You may not like the study or the results, but the data are the data.

r

[Barry_C]

Wouldn't it make sense that if the error rate on a single issue test was 25%, that the likelihood of a higher percentage of error would follow, if more than a single issue was being tested?

You missed my point.  The test is not a test we run in the field (not "filed" - tired fingers?) as a single-issue test.  As Ray has pointed out, ad nauseum, this study was designed to test whether CMs worked.  They didn't.  The study did show a 75% accuracy rate (which too was statistically significant, showing just how robust polygraph is), but, once again, in a format not used in the field.  The TES - a screening exam - generally produces accuracy rates of about 80% with multiple issues, which just adds fuel to the fire that you shouldn't be wrapped up in the accuracy figure.

It's interesting to note that even at 75%, the argument that the CQT - and a DLCQT at that - doesn't work better than chance is a dead issue even when not run optimally.  That's the exact opposite of the prediction made by Lykken et al.

[nopolycop]

[

 The study did show a 75% accuracy rate (which too was statistically significant, showing just how robust polygraph is), but, once again, in a format not used in the field.  The TES - a screening exam - generally produces accuracy rates of about 80% with multiple issues, which just adds fuel to the fire that you shouldn't be wrapped up in the accuracy figure.

It's interesting to note that even at 75%, the argument that the CQT - and a DLCQT at that - doesn't work better than chance is a dead issue even when not run optimally.  .

My point here, which you confirm, is that this test shows a 25% inaccuracy rate, which is also statistically significant, especially when someone is being accused of a crime, or sexual infidility, or whatever.

I will admit that the results are better than chance.  We have agreement here.

What I am saying here, is that you'all can't have it both ways.  You can't use a study to conclude that countermeasures do not work, when one fourth of the subjects either were falsely accused of being either innocent or guilty.  The error rate is simply too great to draw those conclusions.

Additionally, as others have pointe dout, the self-reporting aspect of this study is too variable to draw any conclusions.  

To first decide if countermeasures work, one must first accept the premise that the polygraph actually works.  In this case, it only worked 75% of the time.

[Barry_C]

My point here, which you confirm, is that this test shows a 25% inaccuracy rate, which is also statistically significant, especially when someone is being accused of a crime, or sexual infidility, or whatever.

You have no idea what "statistically significant" means do you?

I don't know how to make it any more clear.  Your logic is wrong, and you're understanding of statistics is lacking.  The study wasn't designed to test accuracy; therefore, you can't draw conclusions from those results.  Moreover, you don't know what would happen to accuracy if the tests were hand-scored (like real-life) or if the probabilities were changed to determine the cut-off for truthful scores.  You could catch more of them by changing cut-offs, but then of course, you'd likely increase other types of errors.

[1904]

 You could catch more of them by changing cut-offs, but then of course, you'd likely increase other types of errors.

Yeah, like this one :

The video of the CBS 60 Minutes report "Lost in Translation," which documents the allegations of FBI whistleblower Sibel Edmonds, which include the FBI's penetration by a polygraph-passing Turkish spy.

What happened??? Cutoffs changed a bit too much...

[nopolycop]

My point here, which you confirm, is that this test shows a 25% inaccuracy rate, which is also statistically significant, especially when someone is being accused of a crime, or sexual infidility, or whatever.

You have no idea what "statistically significant" means do you?

I don't know how to make it any more clear.  Your logic is wrong, and you're understanding of statistics is lacking.  The study wasn't designed to test accuracy; therefore, you can't draw conclusions from those results.  Moreover, you don't know what would happen to accuracy if the tests were hand-scored (like real-life) or if the probabilities were changed to determine the cut-off for truthful scores.  You could catch more of them by changing cut-offs, but then of course, you'd likely increase other types of errors.

Sir:

Perhaps my working knowledge of the term "statistically significant" is wrong and my knowledge of statistics is lacking as you say, but what I do know from reading the study in question is as follows:

Of the 10 people who "stole" the item in question, but lied about it on the test and had access to the information about countermeasures. 20% beat the test.

Of the 10 people who did not "steal" the item in question, and told the truth about not stealing it, and who had access to the information about countermeasures, 30% were found to be deceptive, although they told the truth.

Of the 10 people who "stole" the item in question, then lied about it and did not have the information about countermeasures, 20% beat the test.

And, of the 10 people who did not "steal" the item and told the truth about not stealing it and did not have access to the information about countermeasures, 40% were falsely accused of this faux crime.

[Barry_C]

Think about that long and hard and see if you can figure out where you went wrong.  (Hint: your argument requires you to talk out of both sides of your mouth.)

How many examiner scored the charts that way?  None.  Why? Because the issue wasn't accuracy.

[nopolycop]

Think about that long and hard and see if you can figure out where you went wrong.  (Hint: your argument requires you to talk out of both sides of your mouth.)

How many examiner scored the charts that way?  None.  Why? Because the issue wasn't accuracy.

Sir, did you read the study?  I did.  It states the following:

'This study examined
whether the provision of such information would affect the validity of the Test for
Espionage and Sabotage, a directed lie variant of the CQT.
Method. Forty participants were divided into four equal groups: guilty, guilty
informed, innocent, and innocent informed. During a first appointment, participants
either did or did not commit a mock crime: then some were provided with a book
containing detailed information on the CQT, including possible countermeasures. After
1 week with the book, all participants were administered a CQT during their second
appointment. '

It also stated:

"The CPSpjt values were also used to generate decisions of truth and deception. Those
decisions are illustrated in Table 2"

Table 2 reported what I previously stated.  Now, to me it looks like they were testing truth and deception.  

[Barry_C]

'This study examined
whether the provision of such information would affect the validity of the Test for
Espionage and Sabotage, a directed lie variant of the CQT.

That was the purpose of the study.

Table 2 reported what I previously stated.  Now, to me it looks like they were testing truth and deception.

Of course they were testing truth and deception.  However, the goal wasn't to see how accurate the test was in that regard.  After all, they used the computer to "score" charts.  They simply set the threshold at .7.  They could have set it at another point, or they could have used two cut-off points and included an INC zone as is done in the field.  They didn't do that as the overall accuracy was not the goal.  That has been established in other studies.

[1904]

It seems patently clear that a study to examine the effects of poly knowledge and possible use of CM's on the validity of a polygraph test -- would be to ascertain whether such test was still accurate, regardless of whether the charts were scored manually or by PS.

What other 'validity' would be of significance....................??

Some people open their mouths only to change feet.

[sombody]

George,

Here is the million dollar question.  Even as you have noted several supposed shortcomings to this research study, do you still believe that countermeasures should be practiced by all examinees?  You purport to be most concerned with innocent examinees taking screening examinations.  Do you still believe those same individuals should practice countermeasures, even though it may hurt them in the end?  

I have seen no scientific study, either flawed or perfect, either peer reviewed or not, which indicates that countermeasures help an innocent person pass a polygraph test, yet we have seen two studies which prove (as much as you wish to discount them) countermeasures do not help guilty individuals and actually hurt the innocent.  How do you know for a fact that the countermeasures you advise individuals to undertake are not in fact hurting those same people?  For someone who has made it his life's mission to eradicate polygraph and supposedly help people, you are doing these folks a disservice and you may be ensuring that truthful examinees don't get the jobs that they are seeking.  

Maybe you should step back and take a look at the big picture.  Just suppose............suppose for just one second.............that this research study is correct.  I hope you are not so egotistical to believe you are perfect and beyond mistake.  Do you believe this research study is without ANY basis and should be TOTALLY disregarded and it is ABSOLUTELY false?  What if this research study is accurate, even a little bit???   It would seem that you may have ruined many lives and careers........

[George W. Maschke]

George,

Here is the million dollar question.  Even as you have noted several supposed shortcomings to this research study, do you still believe that countermeasures should be practiced by all examinees?

I have never believed that countermeasures should be practiced by all examinees. Rather, my position has been, and continues to be, that those facing polygraph "testing" should make an informed decision regarding 1) whether to agree to submit to polygraphic interrogation and 2) whether to use polygraph countermeasures to protect against the risk of a false positive outcome.

You purport to be most concerned with innocent examinees taking screening examinations.

Indeed, I very much am.

Do you still believe those same individuals should practice countermeasures, even though it may hurt them in the end?

As before, I believe that those facing polygraph "testing" should make an informed decision regarding whether to use polygraph countermeasures. Personally, if I were facing a polygraph "test," I wouldn't leave the outcome to chance.

I have seen no scientific study, either flawed or perfect, either peer reviewed or not, which indicates that countermeasures help an innocent person pass a polygraph test, yet we have seen two studies which prove (as much as you wish to discount them) countermeasures do not help guilty individuals and actually hurt the innocent.

For reasons I have explained earlier in this message thread, the 2007 study by Honts & Alloway offered by Skip Webb in the opening post of this message thread proves no such thing. And Honts, Amato & Gordon's 2001 study (Honts, C.R., S.L. Amato, and A.K. Gordon, "Effects of spontaneous countermeasures used against the comparison question test." Polygraph Vol. 30 [2001], No. 1, pp. 1-9.) concerned "spontaneous" (untrained) countermeasures -- something completely different from the kind of countermeasures described in The Lie Behind the Lie Detector. It's dishonest to conflate the two.

How do you know for a fact that the countermeasures you advise individuals to undertake are not in fact hurting those same people?  For someone who has made it his life's mission to eradicate polygraph and supposedly help people, you are doing these folks a disservice and you may be ensuring that truthful examinees don't get the jobs that they are seeking.

The Lie Behind the Lie Detector is being downloaded something on the order of a thousand times a week. AntiPolygraph.org is not receiving the kind of feedback we would expect to receive were our suggested countermeasures hurting our readers.

Maybe you should step back and take a look at the big picture.  Just suppose............suppose for just one second.............that this research study is correct.  I hope you are not so egotistical to believe you are perfect and beyond mistake.  Do you believe this research study is without ANY basis and should be TOTALLY disregarded and it is ABSOLUTELY false?  What if this research study is accurate, even a little bit???   It would seem that you may have ruined many lives and careers........

For reasons I've explained earlier in this message thread, I don't think Honts & Alloway's 2007 study can be generalized to field conditions.