Latest Study Indicates "Lie Behind the Lie Detector" Hurts Innocent, Doesn't Help Guilty

[skip.webb]

Information does not affect the validity of a comparison question test
Authors: Honts, Charles R.1; Alloway, Wendy R.1
Source: Legal and Criminological Psychology, Volume 12, Number 2, September 2007, pp. 311-320(10)
Publisher: British Psychological Society


Abstract:
Purpose: Detailed information about the comparison question test (CQT) and possible countermeasures are now available on the Internet. This study examined whether the provision of such information would affect the validity of the Test for Espionage and Sabotage, a directed lie variant of the CQT.

Method: Forty participants were divided into four equal groups: guilty, guilty informed, innocent, and innocent informed. During a first appointment, participants either did or did not commit a mock crime: then some were provided with a book containing detailed information on the CQT, including possible countermeasures. After 1 week with the book, all participants were administered a CQT during their second appointment. Following the polygraph, participants responded to a questionnaire that asked them about their behavior and perceptions during their examination.

Results: There were no significant effects of providing information on the validity of the CQT. However, the reported use of countermeasures was associated with a lower probability of truthfulness. Results of the debriefing questionnaire were found to support predictions made by the theory of the CQT.

Conclusions: Concerns that readily available information will enable guilty individuals to produce false-negative errors seem unfounded. Moreover, the results actually indicate that the use of countermeasures was associated with a lower probability of truthfulness, which was exactly the opposite outcome predicted by the CQT critics.
Document Type: Research article
DOI: 10.1348/135532506X123770


Submitter's note:

The participants were given the downloadable manual "The lie Behind the Lie Detector" from this site and told to study the book as it could help them to pass the polygraph test.  After having the book for a week, the guilty subjects with the book were no better at passing the test than were the guilty group without the book.  However, the innocent group with the book failed the test at a higher rate  (false positives) than the innocent group without the book.  Makes one wonder if this site is doing more harm than good.  If it doesn't help the guilty to pass and it causes the innocent to fail the test at a higher rate then why would one use the book?  This study indicates the exact opposite from what this web site predicts will happen!

[Paradiddle]

Information does not affect the validity of a comparison question test
Authors: Honts, Charles R.1; Alloway, Wendy R.1
Source: Legal and Criminological Psychology, Volume 12, Number 2, September 2007, pp. 311-320(10)
Publisher: British Psychological Society


Abstract:
Purpose: Detailed information about the comparison question test (CQT) and possible countermeasures are now available on the Internet. This study examined whether the provision of such information would affect the validity of the Test for Espionage and Sabotage, a directed lie variant of the CQT.

Method: Forty participants were divided into four equal groups: guilty, guilty informed, innocent, and innocent informed. During a first appointment, participants either did or did not commit a mock crime: then some were provided with a book containing detailed information on the CQT, including possible countermeasures. After 1 week with the book, all participants were administered a CQT during their second appointment. Following the polygraph, participants responded to a questionnaire that asked them about their behavior and perceptions during their examination.

Results: There were no significant effects of providing information on the validity of the CQT. However, the reported use of countermeasures was associated with a lower probability of truthfulness. Results of the debriefing questionnaire were found to support predictions made by the theory of the CQT.

Conclusions: Concerns that readily available information will enable guilty individuals to produce false-negative errors seem unfounded. Moreover, the results actually indicate that the use of countermeasures was associated with a lower probability of truthfulness, which was exactly the opposite outcome predicted by the CQT critics.
Document Type: Research article
DOI: 10.1348/135532506X123770


Submitter's note:

The participants were given the downloadable manual "The lie Behind the Lie Detector" from this site and told to study the book as it could help them to pass the polygraph test.  After having the book for a week, the guilty subjects with the book were no better at passing the test than were the guilty group without the book.  However, the innocent group with the book failed the test at a higher rate  (false positives) than the innocent group without the book.  Makes one wonder if this site is doing more harm than good.  If it doesn't help the guilty to pass and it causes the innocent to fail the test at a higher rate then why would one use the book?  This study indicates the exact opposite from what this web site predicts will happen!

Thank you Mr. Webb.

tap tap tap.............George?
Mrs. Maschke, can George come out and play?
He can bring his inhaler.

[Wonder_Woman]

Information does not affect the validity of a comparison question test
Authors: Honts, Charles R.1; Alloway, Wendy R.1
Source: Legal and Criminological Psychology, Volume 12, Number 2, September 2007, pp. 311-320(10)
Publisher: British Psychological Society


Abstract:
Purpose: Detailed information about the comparison question test (CQT) and possible countermeasures are now available on the Internet. This study examined whether the provision of such information would affect the validity of the Test for Espionage and Sabotage, a directed lie variant of the CQT.

Method: Forty participants were divided into four equal groups: guilty, guilty informed, innocent, and innocent informed. During a first appointment, participants either did or did not commit a mock crime: then some were provided with a book containing detailed information on the CQT, including possible countermeasures. After 1 week with the book, all participants were administered a CQT during their second appointment. Following the polygraph, participants responded to a questionnaire that asked them about their behavior and perceptions during their examination.

Results: There were no significant effects of providing information on the validity of the CQT. However, the reported use of countermeasures was associated with a lower probability of truthfulness. Results of the debriefing questionnaire were found to support predictions made by the theory of the CQT.

Conclusions: Concerns that readily available information will enable guilty individuals to produce false-negative errors seem unfounded. Moreover, the results actually indicate that the use of countermeasures was associated with a lower probability of truthfulness, which was exactly the opposite outcome predicted by the CQT critics.
Document Type: Research article
DOI: 10.1348/135532506X123770


Submitter's note:

The participants were given the downloadable manual "The lie Behind the Lie Detector" from this site and told to study the book as it could help them to pass the polygraph test.  After having the book for a week, the guilty subjects with the book were no better at passing the test than were the guilty group without the book.  However, the innocent group with the book failed the test at a higher rate  (false positives) than the innocent group without the book.  Makes one wonder if this site is doing more harm than good.  If it doesn't help the guilty to pass and it causes the innocent to fail the test at a higher rate then why would one use the book?  This study indicates the exact opposite from what this web site predicts will happen!

Now boys, how are you going to spin this one?  Anyone thinking about using CM's you have been warned.  We have been saying this site does a dis-service to applicants for months.  I also believe Dr. Lou Rovner (a real Ph.d.) did a similar study before this one and Honts/Alloway's study confirmed it again. Thanks Skip for citing the study.  

[Sergeant1107]

If that study is accurate, and the information on this site doesn't help the guilty pass, then I assume everyone who has accused George of "helping child molesters" will apologize?  Apparently the information he collected doesn't do anything to "help child molesters" after all.

[Bill Crider]

I will come out and play. roughly 4 years ago, on 3 different occasions, with 3 different tests, 3 different FBI polygraphers accused me of lying about selling and using drugs. Ok, 1 was "inconclusive" , so he wasnt sure. Just turned 39 years old and still haven't sold or used an illegal drug in my entire life. I still think about this injustice every day. 4 years later and the smug self-rigtheousness of this thread has reawakened the pissed-off in me.

What you fail to realize is that this site is not the product of disgruntled liars. This site exists because of the falsely accused. when we cant imagine why we fail, we look for answers and we find them here. we find that the liars were on the other side of the machine.

I have nothing to hide. my real name is on this posts. My whole story is on this site. SOmeone investigate me and find my drug selling. WHy don't you do it, Mr webb. or Wonder Woman. Do you realize how stupid you look to equate a comic book magical artifact to a law enforecement interrogation technique?

So, F$@K you and your study Mr. Webb. You bastards and your piece of S&%T machine stole a career for which I had spent 14 years pursuing. I really don't give a damn about your study or your proof. These idiots dared to look me in the eye and call me a drug dealer when they have no idea except a few scribbles on a page they mistake for mind reading.

My deepest wish is that one day one of you or a loved one is falsely accused of a crime based on polygraph evidence and sent away for a very long time at a prison infested with gay gangs. Then we will see what you think about your studies when it actually affects you. For now though, remain in your ivory tower with your studies and papers and tell people like me how wrong we are about people who were so wring about us.

[J.B. McCloughan]

Bill,

I believe that Skip's post was clear that this is simply information that directly answers one of George's long burning questions as to whether or not knowledge of the polygraph and a given format used with it (its procedure) might affected the outcome of an examination.  There was no purporting about false positives or your examination outcome by Skip or anyone else for that matter.

I in fact indicated the same information as that regarding the aforementioned research did in a post some time ago.  The fact of the matter is that there are at least two studies that specifically address this very question and both purport the same findings.

I am sorry for the loss of your dream.  I doubt that anything or anyone but you could change your thoughts and beliefs about polygraph.

To the last part of your post.  I am unaware of any case where an individual was convicted based solely on the results of a polygraph examination or any other single source of forensic evidence.

[George W. Maschke]

Dear Mr. Webb,

First, I'd like to welcome you to AntiPolygraph.org message board! Your views and perspective are most welcome. (I'd like to point out to readers who may not recognize Mr. Webb's name that we have the rare honor of speaking with a past president and executive director of the American Polygraph Association.)

The Honts & Alloway study whose abstract you've cited has previously been the subject of brief discussion in the message thread Critique of Louis I. Rovner's Polygraph Examination and Testimony in Ohio v. Sharma, where I also posted the article abstract. As I noted then, the 2007 Honts & Alloway study has serious methodological shortcomings. A question I raised then, and raise again here, is: if Lou Rovner truly believed that knowledge of polygraph procedure and countermeasures has no effect on polygraph accuracy, why did he tell the court that the information in The Lie Behind the Lie Detector is breathtakingly "good and accurate" but tell his examinee (Sahil Sharma) that it's "bogus?" It would seem that Dr. Rovner is skeptical of Honts & Alloway's titular conclusion that "Information does not affect the validity of a comparison question test." And with good reason. Shortcomings in Honts & Alloway's experimental design make generalization to real world (field) conditions problematic. Study participants were 40 students in an introductory psychology course who participated for class credit. The authors describe their experimental design thus (at pp. 313-15):

Procedure

Participants responded to a call for participants posted in a common area for students in the lower division classes. Participants were screened on this initial contact. Potential participants who reported being under the care of a physician were excluded from participation. Acceptable participants were then given individual appointments to report to a psychology laboratory on campus. Each participant was greeted by a research assistant and was asked to watch a brief video on the computer located in the room. The video described the basic parameters of the study and described the possible conditions of assignment. After watching the video, the participant reviewed a consent form with the assistant and signed the form if he or she agreed to participate in the study. The participant was then asked to randomly choose a manila envelope from a box. There were ten envelopes for each of the four conditions shuffled in the box. Random assignment was thus constrained by the number of envelopes in the box. Each envelope contained instructions for the participant to watch a particular video on the computer, as well as a password for that video. The second video informed the participant about the condition to which he or she was assigned and then provided instructions regarding what to do next.

Participants in the guilty conditions were instructed to go to the psychology department, located in another building, and steal an envelope containing movie pass vouchers and labelled with the name Sam Stone. Participants in the innocent conditions were instructed to leave an envelope in a box on a door in the psychology department. The destination door was the same for all participants and thus they had the same experience except for the act of stealing the Sam Stone envelope. All participants were informed to return to the psychology laboratory after 20 minutes. Those assigned to one of the information conditions were given a copy of The lie behind the lie detector (Maschke & Scalabrini, 2000) upon their return to the laboratory. The lie behind the lie detector is an extensive treatment of the CQT and includes specific information about the directed lie variation, the TES and possible countermeasures. The instructions for our information participants indicated that the book contained information that might help them pass their polygraph. Participants were encouraged to study the book before taking their examination. All participants were told that in addition to receiving course credit for their participation, if they could pass their polygraph, they would receive movie passes as a bonus. All participants were then given a second appointment to take their polygraph test at a later date.

Participants returned individually at least 1 week later for their second appointment. After a brief discussion of the procedure with the examiner, the participants were asked to sign a consent form. They were then given a CQT about the theft of the movie passes. The CQT in this study took the form of the TES and was modelled as closely as possible on the protocol used by the US Government (Dollins, 1997). During the pre-test interview of the TES, participants were asked questions regarding their general health, demographic information and past experience with polygraph examinations. The polygraph instrument and the sensors were then described and placed on the participant. An acquaintance (stimulation) test was then administered to the participant (Dollins, 1997; Raskin & Honts, 2002) as a demonstration of the validity of polygraph. All of the questions to be used in the examination were then reviewed with the participant. The question list from this experiment is reproduced in Table 1.

Table 1. The questions used in the TES polygraph examination

Question Type	Question Wording
Relevant	Did you take the movie pass vouchers? Do you know where the missing movie pass vouchers are right now? Do you know for sure who took the missing movie pass vouchers? Did you take the missing movie pass vouchers from that door in the Education Building?
Comparison	Before the year 2000, did you ever say anything that you later regretted? Before the year 2000, did you ever make a mistake? Before the year 2000, did you ever tell even one lie in your entire life? Before the year 2000, did you ever violate even one law, regulation, or rule?
Neutral	Does 2 + 2 = 7? Are we in Idaho?

Participants were then administered a polygraph examination that consisted of two
periods of data collection. Each period of data collection contained two relevant
questions, each of which was repeated three times. Following the data collection, the
examiner ran the CPS discriminant analysis classification software (Kircher & Raskin,
1988). Participants who produced probabilities of truthfulness of .7 or greater were told
they passed and were given the movie pass bonus. All participants were then debriefed....

Weaknesses of this methodology include:

• Motivational conditions were low. Participants were students who volunteered their time in exchange for class credit. There were no adverse consequences for failing to pass the polygraph, and the only "reward" for passing was paltry (movie passes);
  
  
• Although participants provided with copies of The Lie Behind the Lie Detector were "encouraged to study the book before taking their examination," there is no way of knowing to what extent they actually did so. Students received class credit whether or not they read the book. While participants reported spending an average 1.58 hours reading it (with a standard deviation of .96), these self-reported study times may well have been overstated by participants wishing to be perceived as having heeded the encouragement to study the book;
  
  
• The study relied on examinee self-reporting of whether examinees used countermeasures. Perhaps this was unavoidable, as there is no reliable technique for detecting countermeasures. Nonetheless, such self-reporting may or may not have been truthful;
  
  
• There is no indication that examiners were blind with regard to examinee status (e.g. guilty/innocent, informed/uninformed). And examiners were almost certainly not blind with regard to base rates for each category.

It should be noted that although the study purports to use the Test for Espionage and Sabotage format (which is a screening test for conduct not known to have occurred), the polygraph examinations administered were not screening examinations, but rather concerned a specific incident known to have occurred. 35% of innocent examinees failed the polygraph, and overall correctness of classification obtained in this study was only 72%.

Finally, the number of study subjects was low, and the study likely lacks statistical power. Regarding the claim that The Lie Behind the Lie Detector "hurts the innocent," it should be noted that only 2 of the 10 "innocent" study subjects who were provided with a copy of The Lie Behind the Lie Detector reported using countermeasures. The study does not indicate whether they passed or failed. The study data is insufficient to support any conclusion that The Lie Behind the Lie Detector hurts the innocent

[skip.webb]

George,  Thank you for commenting on the study I posted.  It appears to have struck a nerve!  Your comments about the value of the study require some explanation to those who might not routinely avail themselves of research and prefer to accept what they read on the internet as the Gospel.

You stated the following:

Weaknesses of this methodology include:

Motivational conditions were low. Participants were students who volunteered their time in exchange for class credit. There were no adverse consequences for failing to pass the polygraph, and the only "reward" for passing was paltry (movie passes);

RESPONSE: George to be fair, you should point out that virtually all mock crime studies, indeed all psychological studies usually involve students as participants who volunteer their time in exchange for class credit or a minor monetary reward or token.  This certainly isn't a weakness in this or any other study.


Although participants provided with copies of The Lie Behind the Lie Detector were "encouraged to study the book before taking their examination," there is no way of knowing to what extent they actually did so. Students received class credit whether or not they read the book. While participants reported spending an average 1.58 hours reading it (with a standard deviation of .96), these self-reported study times may well have been overstated by participants wishing to be perceived as having heeded the encouragement to study the book;

RESPONSE:  The conditions you describe are exactly the same as the "real" people who come onto your site and read your book.  There is no way to determine the amount of time they spend or the degree to which they "study" the book prior to their examination.


The study relied on examinee self-reporting of whether examinees used countermeasures. Perhaps this was unavoidable, as there is no reliable technique for detecting countermeasures. Nonetheless, such self-reporting may or may not have been truthful;
RESPONSE:  Now George, let's don't even go there.  Everything on your site is self reporting and there is certainly no way to determine the truth about what the people on your site report when they deny crimes or allege successful countermeasures used.


There is no indication that examiners were blind with regard to examinee status (e.g. guilty/innocent, informed/uninformed). And examiners were almost certainly not blind with regard to base rates for each category.

RESPONSE:  Now George, that was beneath you.  You are much smarter than that and so are most of the people who read the information on this site.  You certainly know that Dr. Honts would not bother to conduct a study in which the examiners were not blind to the condition of the participants.  That's simply reaching on your part.  I'm embarrassed for you on that one.

Finally, studies are just that...studies.  They allow thoughtful people to form conclusions and opinions based upon what they see happen in the scenario presented.  This one was particularly insightful as it replicated a previously conducted study that indicated the same result.  I placed the study here so that thoughtful people might read it and make decisions for themselves about the use of countermeasures.  Many, like you, won't change their mind but some smart, truthful people might just read this and decide that they don't want to put their career and their fate into the hands of someone who has become so entangled in his own web that he can no longer step back and look objectively at anything that challenges his belief system.  Smart people might just decide not to drink the kool aid George.

[G Scalabr]

Mr. Webb,

I also thank you for choosing to participate here in this forum. I feel that it is important for all points of view to be properly represented.

Many, like you, won't change their mind but some smart, truthful people might just read this and decide that they don't want to put their career and their fate into the hands of someone who has become so entangled in his own web that he can no longer step back and look objectively at anything that challenges his belief system.

Everyone who submits an application for employment where there is a requirement to "pass" a polygraph screening "examination" does just this. It's simply playing Russian roulette with your reputation.

[Ludovico]

Everyone who submits an application for employment where there is a requirement to "pass" a polygraph screening "examination" does just this. It's simply playing Russian roulette with your reputation.

That is simply an excess in drama.

Russian roulette is a potentially deadly game. Even the people who whine regularly on this site about their polygraph experiences seem to have no such extreme impact.

The point is that this website is not in fact providing a helpful service to its readers. In fact there is evidence that the information on this website is hurtful to decent people.

[skip.webb]

Mr. Scalabrini, I don't argue with your contention that polygraph is an imperfect tool, nor do I think that any intelligent examiner would argue that it is a perfect one.  Most of us understand that there is no perfect test or tool and that polygraph, when used as a method of screening, should be one part of a process not the deciding factor.  I think we can also agree that there is no perfect psychological tool or test one can use to make a decision concerning someone's suitability to hold a particular position of trust, yet it is also used by most of the same agencies to make hiring decisions as are written tests and interviews which are surely subjective and subject to abuses as well.  I argue only with your methods.  Were you and George to direct your efforts towards insuring that polygraph tests not be used as the sole deciding factor in hiring decisions then we would have little with which to disagree.  I think both of you are intelligent people.  I find it difficult, therefore to believe that you can read study after study that clearly shows that polygraph works and the underlying theories, contrary to George's constant assertions, are well founded in both psychology and physiology and then deny those facts or attack the methods or procedures used.

There are countless medical tests in use that don't rise to the level of 100% perfection with no chance of false positive or false negative.  We depend on the results of those "imperfect" tests daily and make life and death decisions based, in part, upon those results.  In that they are imperfect would you have them thrown out or discontinue their use? The reality is that the most accurate medical test for determining a given condition is the autopsy.  Most people don't want to undergo the test, however as it is somewhat intrusive.  It is, however, extremely accurate.

The reality is that in a world where 1000 people compete for 100 positions, 900 won't be selected.  Many of those 900 will be disqualified for obvious, undeniable reasons and others will not be selected based upon some very subjective decision rules concerning their college grades, past credit history, psychological testing scores, their responses during interviews, physical attributes and abilities and, yes, their results on a polygraph test.  Some or those not selected will walk away believing they were better qualified than some of the people selected and they may be correct in their belief.  

In short, Mr. Scalabrini, life is not fair.  We all know that but we also know that cheating as a method of trying to level the playing field isn't right either.  Neither you nor George would ever turn in the academic work of another because a professor was overly subjective or unfair nor would you submit someone else's urine sample for a drug test because you were disagreed with the test.   You wouldn't falsify your credentials in an effort to obtain a position of trust.  Those tests aren't perfect either and they are subject to errors of decision and judgment because they are performed by humans.  No one can take your integrity from you.  It must be given away.  Why then would you urge others to give away the one thing that no one can take away from them by attempting deceit on the polygraph?

 

[Squeezecheeze]

Mr Webb,

A posting such as yours is long over due. Thanks!

SC

[StudebakerHawk]

Who is Gino Scalabrini anyway?  Another "expert" like "Dr." Richardson?  

[Sergeant1107]

In short, Mr. Scalabrini, life is not fair.   

Is this sort of rationalization indicative of the mindset a polygraph examiner has when they fail someone?  That a false positive is acceptable because life is not fair?  Is it common for an examiner to think, "I have no idea how many of the DI's I scored today were false positives, but I'm not concerned about it because life isn't fair."?

I have no experience with the use of polygraphs in anything other than pre-employment screening.  However, with regards to pre-employment screening, I know they do not work, because I failed three out of four while telling the truth and not withholding any information.  Your comments seem to indicate that you are aware of the polygraph's shortcomings in this regard, yet you still have no problem with them being used for such purposes.

Your analogy regarding 900 of 1000 applicants being disqualified due to credit scores and physical testing is flawed, I believe.  In my experience the polygraph is random in its results on pre-employment screening.  A more accurate analogy would be if all 1000 applicants were subjected to a coin toss and everyone who came up "heads" were automatically disqualified.  You could pass that off by saying "life isn't fair" as well, but that wouldn't change the fact that using such a process is wrong.

Regarding your comments about integrity and the polygraph, I believe that the only requirement for ethical behavior on the part of the test subject is to tell the truth.  If the subject is truthful and does not withhold any information, he or she is fulfilling their responsibility to behave morally and ethically.  If they choose to do long division in their head, or bite their tongue, or recite poetry or song lyrics after answering each question, how can you consider that to be "giving away their integrity?"

[G Scalabr]

Were you and George to direct your efforts towards insuring that polygraph tests not be used as the sole deciding factor in hiring decisions then we would have little with which to disagree.

Mr. Webb, we definitely have some common ground here.

Still, had the polygraph industry kept its own house in order by auto imposition of a common sense policy like the one you describe, we would not even have any efforts to direct. This organization simply would not exist.

The regulatory bodies that govern polygraphy have failed due to inaction on so many levels, I don't know where to begin.

First, and foremost, the APA could have completely distanced itself from polygraph screening. While we can agree to disagree about the validity of specific issue testing, there is unanimous contempt among scientists and researchers for polygraph screening.

For this to have been more than lip service, it would have required serious sanctions against any members conducting such "exams" (e.g. one warning and then the boot).

Short of that, the APA could have taken the advice you just gave us and insisted that polygraph screening never be used as a sole determining factor in making any kind of adverse employment decision.

Again, for this to have been substantive, it would have involved speaking out against the FBI and Secret Service (who have done this for years), coming to the support of applicants who were wrongly dismissed following a single failed polygraph, and most importantly, the issuance of severe sanctions (e.g. expulsion) against any APA member who performed screening "exams" for an agency using them as a sole determining factor in employment.

Lastly, with so many members polygraphing applicants and asking questions about deception on employment applications, the APA can have no public credibility whatsoever while looking the other way against verified allegations of "luminaries" in the field who have presented demonstrably false educational credentials (read: FAKE PhDs) to examinees, clients and even the courts. Again serious sanctions against violators would have been the only way to communicate sincerity here.

This could have been done tactfully... For example phone calls could have been made to people like "Dr." Ed Gelb giving a 48 hour period to make a public statement retracting the false credentials (e.g. "I was unaware my school was unaccredited...", "I didn't know that you can't use the prefix if your degree is from an unaccredited school...", etc) before an APA public statement echoing this information.

Yet, the APA did not implement any of these common sense ideas. In addition to hurting job applicants and depriving the government of highly capable employees, the damage done to the former by the inaction of polygraph organizations pales in comparison to the damage that has been done to CQT polygraphy as a whole.

All of this brings me back to the following statement...

someone who has become so entangled in his own web that he can no longer step back and look objectively at anything that challenges his belief system

If some in your industry had not been so greedy and taken an objective look at what was happening (namely, the rampant abuse of polygraph screening) and actually done something to curtail the problem, this huge threat to polygraphy would not exist (and your members could make better use of their evenings than an organized effort participating here in an attempt to do damage control).