Latest Study Indicates "Lie Behind the Lie Detector" Hurts Innocent, Doesn't Help Guilty

[EJohnson]

Wrong!----Give that man a gym membership folks, and thank him for playing! George, of course there is raging debate. The anti-crowd only accept hard science, the real scientific community isn't so preoccupied with the differences and distinctions between hard and soft science---the classic example being behavioral psychology vs. Neuro-psychology. Plain and simple---the polygraph test is a unique hybrid of the two, and despite your venom and Gino's spit, polygraph is scientificaly valid and is effective at detecting deception at far better than chance levels (NAS STUDY.) You folks clearly don't like the method and it's mechanisms-----fine. You can't burn steal though.

The NAS study used the term "better than chance" to describe specific incident testing, and specified that it was only pertinent when the subject population was untrained in countermeasures.  

How can you tell is someone is untrained in countermeasures?  If they show no signs of using them?  Wouldn't a person skilled in countermeasures also show no signs of using them?

From page 214 of the NAS report:

Notwithstanding the quality of the empirical research and the limited ability to generalize to real-world settings, we conclude that in populations of examinees such as those represented in the polygraph research literature, untrained in countermeasures, specific-incident polygraph tests for event-specific investigations can discriminate lying from truth telling at rates well above chance, though well below perfection.

Accuracy may be highly variable across situations. The evidence does not allow any precise quantitative estimate of polygraph accuracy or provide confidence that accuracy is stable across personality types, sociodemographic groups, psychological and medical conditions, examiner and examinee expectancies, or ways of administering the test and selecting questions. In particular, the evidence does not provide confidence that polygraph accuracy is robust against potential countermeasures. There is essentially no evidence on the incremental validity of polygraph testing, that is, its ability to add predictive value to that which can be achieved by other methods.

It's amusing that you referenced the NAS report when we keep hearing from polygraph examiners (including the APA's web site, which devotes an entire page to refuting the NAS report) that is was flawed and should be disregarded.  I can understand why polygraph examiners are desperate to discredit the NAS report - it is a damning condemnation of their pseudoscientific profession.  The conclusions in the report are clear - the polygraph presents a danger to national security objectives.  

I've read the report and it is compelling, even to a layman.  It is logical and and the conclusions drawn from the research cited are quite understandable.  Hearing from polygraph examiners that the report should be ignored is no more credible than hearing from Big Tobacco that the health warnings regarding cigarettes should be ignored.

I am amused that you are amused. We are amused together, laughing and singing. Aside from that, the fact remains that there are shortcomings from the study, and there are accuracies from the NAS study----and it seems that polygraph examiners are the only ones who readily admit both---while the negativists cling to the nasty parts---like teen boys watching those dirty HBO documentaries on sex toys. Fine. Your analogy of Big Tobacco was inteligent, but inadequate. Big Tobacco has not been accused of their product causing MS, and this site is making such types of claims (countermeasure success) without peer reviewed scientific research to back it up. Er---maybe Big Tobacco isn't the right analogy---I have always distrusted Big Breakfast Cereal Lobby. Perhaps I'll use an analogy with those weasals after I take the boys to the dentist today.
Essentially the moto here is to ping pong claims-----to argue that polygraph is "pseudo-science",
but when examiners present clearly that polygraph is based in science (psych, testing, statistics, Phys)---
then you launch into the "well, the test is useless because of our countermeasures"----

then we say "your countermeasures are not scientifically proven effective against countermeasure-trained examiners"---

then you guys say "well, polygraph is pseudoscience and isn't based in science anyway"-

--then we all start the whole process over again. No debates. No discovery, just a contest as to who is the more clever writer of the same circular dialogue. I do like writing though.

[Sergeant1107]

Essentially the moto here is to ping pong claims-----to argue that polygraph is "pseudo-science",
but when examiners present clearly that polygraph is based in science (psych, testing, statistics, Phys)---
then you launch into the "well, the test is useless because of our countermeasures"----

then we say "your countermeasures are not scientifically proven effective against countermeasure-trained examiners"---

then you guys say "well, polygraph is pseudoscience and isn't based in science anyway"-

--then we all start the whole process over again. No debates. No discovery, just a contest as to who is the more clever writer of the same circular dialogue. I do like writing though.

I don't recall ever writing that the polygraph is useless because of "our" countermeasures.  I have repeatedly written and I continue to believe that the polygraph is incapable of detecting truth or deception, and it is for that reason that it is useless.

I have always been concerned solely with polygraph screening, such as the type I was subjected to in my pre-employment tests.  The NAS study concluded that in such circumstances the accuracy of the polygraph was essentially the same as random chance.  It seems reasonable to me that if the polygraph is incapable of detecting truth or deception in screening situations, it also has serious shortcomings in other situations as well.

I think it is far more reasonable to take the word of the National Academy of Sciences (whose members have over 170 Nobel Prizes to their credit) over the word of a profession that bases its opinion on the fact that without the polygraph they would be out of a job.

[1904]

Why did the US Govt enact the EPPA ?

Wasn't that also motivated by incorrect p/g screenings?

Just a question... not an invitation to a gunfight.

[raymond.nelson]

[
<snip>

Mr. Sergeant1107,

You are posting inaccurate and misleading information here. Please be careful, as that is a violation of the posting policies.

The NAS did not reach any conclusions about screening tests being the same as random chance. Please either check your facts, or refrain from such misleading inaccuracies.

Its quite difficult, if not impossible, to have a in intelligent discussion about these matters when conclusions are drawn from personal experience, simple appeal to authority, and your well-manner form of ad hominem attack (based on your assumptions about the job of your opponent) against your opponent, instead of arguing the substance of matter at hand.

r

[nopolycop]

Its quite difficult, if not impossible, to have a in intelligent discussion about these matters when conclusions are drawn from personal experience, simple appeal to authority, and your well-manner form of ad hominem attack (based on your assumptions about the job of your opponent) against your opponent, instead of arguing the substance of matter at hand.

r

Mr. Nelson:

I quite agree with you here, which is one of the reasons I have asked the questions I have in such a non-emotional way.  Please feel free to join my discussion regarding blood pressure v. pulse rate, (but only if you can explain in simple terms!  

[Sergeant1107]

Its quite difficult, if not impossible, to have a in intelligent discussion about these matters when conclusions are drawn from personal experience, simple appeal to authority, and your well-manner form of ad hominem attack (based on your assumptions about the job of your opponent) against your opponent, instead of arguing the substance of matter at hand.

Are you not drawing conclusions from your own personal experience?  That is the reasonable thing to do, and I expect you are aware of that.  If I had never taken and failed a polygraph while being truthful I doubt I would have any strong opinion about the polygraph at all.  If you had not administered polygraph exams for years I doubt you would have a strong opinion either.

The argumentum ad verecundiam (appeal to authority) is only properly termed as such when the authority to which the debater refers has no expertise in the subject area.  The National Academy of Sciences certainly has expertise in all "hard" and "soft" sciences pertaining, no matter how remotely, to the pseudoscience of polygraphy.  Citing their research study to back up the opinions I formed based on my own experiences with the polygraph is logical and perfectly justifiable.

[raymond.nelson]

I have not offered, in this discussion, information from my personal or professional experience.

It is fine, in discussion, to cite the conclusions and and inforamtion from the NAS. Please do so accurately so that you are not misleading others with false information. It is fine to understand the concerns they expressed, but it is dishonest (and a violation of the posting policies) to twist those conclusions into dramatized, false, and inaccurate information.  Also, when citing information, you have a choice whether you terminate your involvement in the discussion with your conclusion or alignment with your chosen authority. That ends the discussion. It would be more satisfying for you to remain engaged in the substantive discussion instead, of retreating behind straw-man labels like "pseudo-science."

If you want to learn, then you have to stay in the discussion. That's fun. If all you want is to hurl labels and insults, beat the drum for the anti-fodder, and help inoculate a few more bad guys against the good work of hard working law enforcement investigators, that's not so fun.


r

[George W. Maschke]

...It would be more satisfying for you to remain engaged in the substantive discussion instead, of retreating behind straw-man labels like "pseudo-science."

It is no straw-man argument (a term you've evidently misunderstood) to point out that polygraphy is pseudoscience. In this regard, see the message thread, Polygraphy as Pseudoscience.

[raymond.nelson]

George,

I realize this is your turf, and you get to have your way, by creating any discussion thread you want, and emphasizing the editorial angle that serves your mission. But you are, in fact, engaging in a straw man argument when you make silly claims about mind-reading, and begin to extinguish all thought by communicating in labels instead of discussion.

r

[George W. Maschke]

Raymond,

Early in this message thread, I pointed out serious shortcomings in the design of the 2007 Honts & Alloway study that is this thread's original topic -- shortcomings that make generalization to field conditions all but impossible. I note that ad hominem arguments aside, no one has seriously disputed my critique.

[Sergeant1107]

If you want to learn, then you have to stay in the discussion. That's fun. If all you want is to hurl labels and insults, beat the drum for the anti-fodder, and help inoculate a few more bad guys against the good work of hard working law enforcement investigators, that's not so fun.

How, exactly, could anything being written on this forum be helping to "inoculate a few more bad guys against the good work" of law enforcement?

You have cited the Honts study as proof that the countermeasure information on this web site hurts the innocent and doesn't help the guilty.

Surely you can't be referring to any reference to the 2003 National Academy of Sciences research study?  That information is freely available to anyone who chooses to take the time to look for it.

I don't know what else it could be.  I do not understand how you have come to the belief that I am somehow helping bad guys.

[1904]

Its interesting to have pro-poly people debating poly issues on this site, mainly because the level
of debate / argument that takes place here would not be tolerated on the somewhat sterile PP
site.

If one posts anything even remotely anti-poly on PP , the post is deleted rapidly

Which is obviously why the pro-poly* folk come here.

What I dont understand is, their* apparent wish that we should not argue against them, but simply
accept their arguments and then keep quiet.

And when one does argue a point, then the discussion starts going south and ends in futile aggression
and ad-hom attacks.
Its like: "Agree with us or shut up"

How depressing.

[raymond.nelson]

Raymond,

Early in this message thread, I pointed out serious shortcomings in the design of the 2007 Honts & Alloway study that is this thread's original topic -- shortcomings that make generalization to field conditions all but impossible. I note that ad hominem arguments aside, no one has seriously disputed my critique.

Mr. Maschke,

You are once again providing false and inaccurate information to your readers, in violation of your posting policies.

Mr. Webb has provided a thoughtful response to your critique.

Please try to be more careful.

Just in case you missed it, I have copied it here.

Re: Latest Study Indicates "Lie Behind the Lie Detector" Hurts Innocent, Doesn't Help Guilty
Reply #7 - Oct 16th, 2007, 8:46am Quote
George,  Thank you for commenting on the study I posted.  It appears to have struck a nerve!  Your comments about the value of the study require some explanation to those who might not routinely avail themselves of research and prefer to accept what they read on the internet as the Gospel.

You stated the following:

Weaknesses of this methodology include:  

Motivational conditions were low. Participants were students who volunteered their time in exchange for class credit. There were no adverse consequences for failing to pass the polygraph, and the only "reward" for passing was paltry (movie passes);

RESPONSE: George to be fair, you should point out that virtually all mock crime studies, indeed all psychological studies usually involve students as participants who volunteer their time in exchange for class credit or a minor monetary reward or token.  This certainly isn't a weakness in this or any other study.


Although participants provided with copies of The Lie Behind the Lie Detector were "encouraged to study the book before taking their examination," there is no way of knowing to what extent they actually did so. Students received class credit whether or not they read the book. While participants reported spending an average 1.58 hours reading it (with a standard deviation of .96), these self-reported study times may well have been overstated by participants wishing to be perceived as having heeded the encouragement to study the book;  

RESPONSE:  The conditions you describe are exactly the same as the "real" people who come onto your site and read your book.  There is no way to determine the amount of time they spend or the degree to which they "study" the book prior to their examination.


The study relied on examinee self-reporting of whether examinees used countermeasures. Perhaps this was unavoidable, as there is no reliable technique for detecting countermeasures. Nonetheless, such self-reporting may or may not have been truthful;
RESPONSE:  Now George, let's don't even go there.  Everything on your site is self reporting and there is certainly no way to determine the truth about what the people on your site report when they deny crimes or allege successful countermeasures used.


There is no indication that examiners were blind with regard to examinee status (e.g. guilty/innocent, informed/uninformed). And examiners were almost certainly not blind with regard to base rates for each category.

RESPONSE:  Now George, that was beneath you.  You are much smarter than that and so are most of the people who read the information on this site.  You certainly know that Dr. Honts would not bother to conduct a study in which the examiners were not blind to the condition of the participants.  That's simply reaching on your part.  I'm embarrassed for you on that one.

Finally, studies are just that...studies.  They allow thoughtful people to form conclusions and opinions based upon what they see happen in the scenario presented.  This one was particularly insightful as it replicated a previously conducted study that indicated the same result.  I placed the study here so that thoughtful people might read it and make decisions for themselves about the use of countermeasures.  Many, like you, won't change their mind but some smart, truthful people might just read this and decide that they don't want to put their career and their fate into the hands of someone who has become so entangled in his own web that he can no longer step back and look objectively at anything that challenges his belief system.  Smart people might just decide not to drink the kool aid George

[George W. Maschke]

Raymond,

I concede that Skip Webb's rejoinder to my critique was earnest, but as I explained earlier in this thread, I don't think he offered anything approaching a credible counterargument to my criticism of Honts & Alloway's methodology.

[Barry_C]

As I stated in a previous post, scientific studies can expose results which are ancilary to the original purpose of the study, but are still significant in their own right.  Given the study in question, it appears that this is the case.  I notice you do not challenge my assertion that the study indicates the accuracy of the polygraph (in this test anyway) is no better than 73%.   I will take that lack of challenge as agreement to this assertion.

Given that fact then, with an error rate of over 25%, how can any honest reviewer conclude that the study proves anything regarding the effectiveness of countermeasures, when the polygraph itself could not detect blatant deception?

The study does show that.  The question is whether that would generalize to filed applications.  Since we don't do those types of tests in the field, then we don't know for sure.  (This was a single-issue application of the TES, a screening exam.  However, they ran it as a single-issue test.  We don't do that, for reasons I'd argue would lower accuracy, which is what we see here.  This is either an apple or an orange, but not both.  In any event, it's not correct to compare the two.)