The Comparison Question Test: Does It Work and If So How?
Dr.s Heinz Offe, Susanne Offe; Law and Human Behavior, Volume 31, Number 3, June 2007 , pp. 291-303(13)

In a mock crime study of the comparison question test (CQT), 35 subjects decided to participate as guilty and 30 as innocent. Two conditions were varied: Explaining the comparison questions in the pretest interview and re-discussing comparison questions between charts. Higher identification rates (∼90% for guilty and innocent participants) were achieved in groups with explanation of comparison questions than in groups without explanation. Re-discussing comparison questions had no effect on identification rates. Ratings of subjective stress due to relevant and comparison questions were also obtained and can be seen as indicators of the significance of the questions. The significance of comparison questions was hardly affected by the different testing conditions. When effects are detectable at all, they contradict theoretical expectations in their direction. Results are discussed in terms of the significance of comparison questions used in polygraph testing.

Here, we have yet another peer reviewed scientific study, this time published in 2007 from researchers in Germany, that demonstrates the value of the comparison question polygraph test.  The authors title question says it all.  Does it work and if so how?  Their methods were some of the best I have seen in over 20 years as a polygrapher  The usual arguments about mock crime studies center around  the argument that mock crime studies don’t replicate “real life” and that people who are “programmed” as guilty or innocent don’t display the same emotions or reactions as “real life” people.  Well these researchers did a number of things to quell that argument.

First they let the participants chose whether they wanted to be the criminals or the innocent suspects.  After all that’s what people in “real life” do.  They then gave the guilty subjects two weeks to go about committing the crime, thereby allowing them to choose the most appropriate time to do the deed while unobserved.  The guilty subjects had to steal a money voucher from a box on a desk in an office area accessible to all the participants.  This method again replicates the manner in which a “real crime” would occur.

Next, they conducted the polygraph examinations between one week and five months of the “crime” again replicating “real life” in much the same way one might become a suspect in a criminal case at some point during the investigation.

Their questions to be answered were:

1. Is it possible to achieve significantly higher than chance identification rates in a mock crime study under more realistic conditions of subjects deciding for themselves, whether they participate as guilty or innocent?  

2. Does explanation of the comparison questions (CQ) in the pre-test interview affect the identification rates as expected?

3. Does discussion of the comparison questions between tests – based on the concept of “delicate balance” affect identification rates?

4. Is the subjectively felt stress imposed by questions related to test results based on physiological measures?

5.  Does the subjective uncertainty about the truthfulness of negating the CQ contribute to the significance of these questions.

Rewards were offered to motivate the participants.  The guilty were offered double the reward should they be later identified as non-deceptive and half for those innocent participants who successfully pass the polygraph.

The results were very telling. The subjects were divided into 4 groups with Group 1 receiving an explanation of the CQ both during the pre-test and between test charts.  Group 2 received explanation of the CQ only during the pre-test interview.  Group 3 received no CQ explanation during the pre-test but did receive discussion of the questions between test charts.  Group 4 received neither explanation of the CQ during the pre-test nor discussion of those questions between test charts.

To further explain the term “explanation” and “discussion”, explanation is the totality of what we as polygraph examiners do when we explain the comparison questions and their relative importance to the overall outcome of the test during the pre-instrument phase.  “Discussion” as used here, means that the QC were discussed between charts during the data collection phase.  An example of discussion might be “Now, Georgina, are any of those questions about your past honesty causing you any problems?”.  This type of discussion would occur between each chart. 

When explanation of the CQ was accomplished during the pre-test and between test charts, the results were 88.9% for both conditions being correctly identified.  

When explanation of the CQ was accomplished during the pre-test but with no discussion between test charts, the identification rates were 93.3%.  

Group 3 participants who received only between chart discussion of the CQ were identified correctly at a rate of 80.0%.

Group 4 participants who received no explanation or discussion were correctly identified by condition at a rate of 58.8%

Questionnaires completed by all participants after their polygraph examinations, revealed that both guilty and innocent participants subjectively “felt” that the relevant questions were the most important of the questions answered on the test.  The guilty, however “felt” that they reacted stronger to the relevant questions and the innocent “felt” they responded stronger to the comparison questions.

This test has confirmed, yet again that when performing the CQT under normal conditions in mock crime studies rates of 90% or better can be obtained even when the participants chose their condition. Innocent subjects subjectively felt the relevant questions were the most important yet they felt they responded stronger to the CQ.  Guilty subjects actually were identified at lower rates when the explanation of the CQ was not given and innocent subjects were correctly classified at a higher rate when explanation of the CQ was provided.  In other words, it helped them pass the test at higher rates.

To answer Mr. Maschke’s usual & obvious questions:

1.  Yes the examiners were blind to conditions of the participants.

2.  Even with inconclusive results included in the study, the resulting identification rates differed only slightly and were not significant.

3.  The “delicate balance” argument made by many polygraph opponents (lykke, Iacono) did not play out.  Discussion of the CQ between the test charts did not increase or decrease identification rates.

This test shows, in the words of the authors “that the differential significance of questions is essentially achieved through the different significance of the relevant questions for guilty and innocent participants, but not through a difference in significance of CQ.”  In other words, the CQT works just as it is designed to do and it works well. 

Yes, from Springer for $40.00.

I am looking forward to George's analysis of this study, as I am not convinced that it is worth MY money to review at this time.

Dr. Maschke,, 

We both know that it is an absolute impossibility for David Lykken to ever publish a review of the Offe study.  I do not think it is reasonable to state that the quoted passage is germane to the Offe study until you have the time to review the study and its findings to illustrate a FULL comparison of the differences and similarities that may or may not exist between their study and the studies analyzed years before by David Lykken. Especially since he died before the Offe study was released. 

Whether or not one agrees with any or all of anything David Lykken ever wrote or said, it is fair to give him credit as a driving force behind continued polygraph research. He managed to inspire those who agreed with him regarding polygraph and challenge those who disagreed, which in my humble opinion is a greater legacy than any conclusion he ever reached. To invoke his name in such a haphazard fashion is a disservice to what he stood for. 

Although I cannot conceive that it was your intent; a premature statement like the one you made might lead someone to conclude that the Offe study somehow possessed whatever flaws David Lykken believed existed in the other studies that he analyzed for his book.   

It is just as likely that upon review of the Offe study, David Lykken would have changed his mind completely and retracted the very comments you quoted.  If I had made the presumptuous statement that the Offe findings would have changed his mind absent a careful analysis and supporting argument, I believe that you would have been among the first to criticize my conclusion, and justifiably so.   

I look forward to your informed analysis of this study after the first of the year. 

I know you are busy and I am still,
Sancho Panza

Every now and then, when I hear some of the musing that comes forth about David Lykken and what he would think, I feel a bit like Lloyd Bentsen listening to Dan Quayle speak about JFK.  David and others have spoken about the weaknesses of probable lie control question tests (PLCQTs) in largely two contexts.   

One is “What is the proper way (environment) to analyze witchcraft?" consideration.  Within this context, issues of a lack of external validity with laboratory studies and the discarding of data points (the discarding of false positive polygraph results (obviously) not followed by confessions) with confession-based (i.e., ground truth determined through confessions) real case studies come into play.  Both of these  phenomena make questionable the results of any studies so impacted.  These considerations are however not at the core of the more serious shortcoming of this practice.   

As has been said time and time again quite eloquently by John Furedy, the probable-lie control question test neither contains any form of scientific control nor resembles any sort of recognizable test.  It is centered on a roughly 30 minute to hour and a half unstandardized interview.  The interview has several purposes, some largely unrelated to the administration of the test (e.g., collection of demographic data) and at least one serious and universal confound (developing theme material to be used in an interrogation of a subsequently found to be deceptive examinee following the in-test phase) but largely focuses on the “setting” of control questions in relationship to relevant questions.  This is a vague notion and has no objective measures for ascertaining whether examiner lies regarding question types are believed by the examinee nor more generally whether this “setting” has been accomplished.   

As previously stated, this witchcraft is further characterized by being unstandardized.  Every test is different.  One hundred exams (more specifically the pre-test interviews) given to 100 examinees about the same specific incident are all different.  Any two exams about the same issue given to the same person are different.  As opposed to the ability to give tape-recorded instructions and an administered test with a guilty knowledge test, the currently practiced PLCQT is nothing more than an unstanderdized interview administered differently by every examiner on every occasion. This may be poor art but it is most assuredly not good science.

I believe it is in regard to this latter set of considerations that David was referring when he said that which is quoted on the home page, "...the theory and methods of polygraphic lie detection are not rocket science, indeed, they are not science at all."







     

Thank you for the correction, nopoly4me.  I will make the change to reflect that which you have noted.

Yes, nopoly4me...the comparison questions of the present control question test are the same as were previously (prior to about 15 years ago) referred to as control questions.  Although the polygraph community is to be commended for more correctly referring to what these questions are (and by omission what they are not—a form of scientific control), they are in no way excused from the necessity of providing scientific control for what they purport to be a "test."

Raymond,

Lykken's endnote for the relevant study (which I omitted from my quotation above) is: "C. Patrick and W.G. Iacono, Psychopathy, threat, and polygraph test accuracy, Journal of Applied Psychology, 1989, 74, 347-355 (specifically pp. 348-349)."