Hot Topic (More than 15 Replies) Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?   (Read 9745 times)
Paste Member Name in Quick Reply Box skip.webb
New User
*
Offline



Posts: 18
Joined: Oct 15th, 2007
Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Oct 26th, 2007 at 1:28pm
Mark & QuoteQuote Print Post  
In an article published in the Journal of Forensic Science 23 (3), 596-601, published in 1978 (almost 30 years ago) by Jan Widacki and Frank Forvath, entitled “An experimental investigation of the relative validity and utility of the polygraph technique and three other common methods of criminal identification.”, researchers compared the polygraph, handwriting analysis, eye witness testimony and fingerprints in a mock crime scenario.  80 subjects (college students) were divided into 20 groups of 4.  In each sub-group there was one guilty participant and three innocent ones.  The task of the guilty participant was to go to a particular building; present themselves to a person whom they did not know before hand;  give an envelope to the specified person; receive a package using a fictitious name; sign a receipt for the package using the fictitious signature and then steal the contents of the package.

All participants were then subjected to a polygraph.  The document receipts and envelopes were subjected to handwriting analysis compared against handwriting standards supplied by all participants.  Fingerprints collected from the envelope and receipts were compared to record prints of all participants.  The eye witnesses to the crimes were provided full front photographs from which to select the perpetrator who signed for and stole the packages.  All forensic technicians were blind to the identity of the perpetrators as was the eye witness but all technicians were aware that one in each group of four was the perpetrator and three were innocent as were the eyewitnesses.

The following results were recorded with inconclusive results removed for all:

Polygraph -18 correct identifications, 1 error and 1 inconclusive result for an over all accuracy of 95%.

Handwriting – 17 correct identifications, 1 error and 2 inconclusive results for an overall accuracy of 94%.

Eyewitness – 7 correct identifications, 4 errors and 9 inconclusive results for an over all accuracy of 64%.

Fingerprints – 4 correct identifications, 0 errors and 16 inconclusive results for an overall accuracy of 100%.

Submitter’s comments:  I have provided this study for thoughtful consideration of those on this site who require sound evidence rather than unsupported assertions or ad hominid attacks on polygraph examiners. I realize that students were used so Mr. Maschke may object but as I previously pointed out, students are used in many studies as such studies are an academic endeavor usually conducted in a university setting.  I also realize that this is a mock crime paradigm and as such Mr. Maschke will assert that as there was no real fear of detection or consequences for the perpetrators, the results may not generalize to the public.  Actually, the reverse should be true.  One would assume if polygraph is able to correctly identify the perpetrator in a mock crime scenario then the intensity of reaction in a real life scenario would be greater and therefore easier to discern. I also realize that in the instant study, the polygraph examiners, although blind to the identity of the perpetrator in each group were aware that one of the four was the guilty party.  This does create an advantage but each of the forensic technicians as well as the eye witnesses shared that same advantage.   In summary, polygraph works.  Its utility in criminal investigations is very high.  Its validity compares very favorably with other forensic tools.  Its error rate compares very favorably with other forensic tools. One can argue that any error rate is unacceptable if it does harm to one innocent person.  If only we lived in such a perfect world.  Look back at the eye witness testimony results and ask your self the following questions: “If I am ever on a jury and I am faced with the testimony of an eye witness, how much weight will I give it?”  If I am ever a defendant in a case how much credence do I want the jury to give eye witness testimony?  Well, my friends, such testimony forms a major part of many trials.
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box George W. Maschke
Global Moderator
*****
Online


Make-believe science yields
make-believe security.

Posts: 6060
Location: The Hague, The Netherlands
Joined: Sep 29th, 2000
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #1 - Oct 26th, 2007 at 2:05pm
Mark & QuoteQuote Print Post  
Mr. Webb,

I don't deny that the polygraph may have utility as an aid to interrogation, to the extent that it may help to obtain confessions/admissions that an examinee might not have otherwise been willing to make. But validity is an altogether different question.

You write:

Quote:
...I also realize that this is a mock crime paradigm and as such Mr. Maschke will assert that as there was no real fear of detection or consequences for the perpetrators, the results may not generalize to the public.  Actually, the reverse should be true.  One would assume if polygraph is able to correctly identify the perpetrator in a mock crime scenario then the intensity of reaction in a real life scenario would be greater and therefore easier to discern....


But the National Academy of Sciences, in its review  of the scientific evidence on the polygraph, disagrees (at pp. 3-4):

Quote:
Laboratory studies tend to overestimate accuracy because laboratory conditions involve much less variation in test implementation, in the characteristics of examinees, and in the nature and context of investigations than arises in typical field applications.


Dr. Alan Zelicoff, M.D., conducted a statistical analysis (255 kb PDF) of data from polygraph field studies selected from an impeccably pro-polygraph source and found the data indicate that "if a subject fails a polygraph, the probability that she is, in fact, being deceptive is little more than chance alone; that is, one could flip a coin and get virtually the same result for a positive test based on the published data."
  

George W. Maschke
Tel/SMS: 1-202-810-2105 (Please use Signal Private Messenger or WhatsApp to text or call.)
E-mail/iMessage/FaceTime: antipolygraph.org@protonmail.com
Wire: @ap_org
Threema: A4PYDD5S
Personal Statement: "Too Hot of a Potato"
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box nopolycop
Especially Senior User
*****
Offline



Posts: 383
Joined: Oct 20th, 2007
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #2 - Oct 26th, 2007 at 2:31pm
Mark & QuoteQuote Print Post  
But Mr. Webb, in the study by Honts attempting to ascertain if countermeasures worked, the resultant rate for polygraphy in general was only 73% accurate.  Now, I realize that accuracy wasn't being specifically tested in that study, but it certainly was a "finding" of the study, and reported as such.

There also in not nearly enough information given regarding how the polygraph was administered to form any conclusions.  Did the person confess?  Did the polygraph give them away.  How were they interrogated? 

Lastly, what exactly were the "guilty" persons guilty of?  Following instructions.  No Mr. Webb, I am not impressed.  With all due respect,  if this is the type of "science" that needs to be resorted to to prove the utility of the polygraph, the polygraphy is in big trouble.

I wonder if this is one of the studies the NAS threw out as not having enough scientific protocols to even be considered.  Anyone know?
  

"Although the degree of reliability of polygraph evidence may depend upon a variety of identifiable factors, there is simply no way to know in a particular case whether a polygraph examiner's Conclusion is accurate, because certain doubts and uncertainties plague even the best polygraph exams."  (Justice Clarence Thomas writing in United States v. Scheffer, 523 U.S. 303, 118 S.Ct. 1261, 140 L.Ed.2d 413, 1998.)
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box nopolycop
Especially Senior User
*****
Offline



Posts: 383
Joined: Oct 20th, 2007
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #3 - Oct 26th, 2007 at 2:51pm
Mark & QuoteQuote Print Post  
A few additional observations.  In the first paragraph quoted below, I believe you are WAY OFF BASE.  From what I have read about polygraphy, the more emotion that is attached to a particular event, the more likelyhood of incorrect findings, not the reverse.  Remember, the students were not really guilty of anything, they just followed directions.

#2. Huh?  How can you conclude this, Mr. Webb, based on this study?

#3.  I actually agree with you here, as long as the investigators do not rule out the person as a suspect if they "pass".  Can you say "Gary Ridgway?"

#4.  That is why a good criminal investigator does not rely solely on any investigative tool.

#5.  I am not sure I can agree with this claim, since the protocols of the study were not given.  I also find it interesting that in this analysis, an inconclusive is removed, whereas in the real world, investigators of crime have to deal with an inconclusive finding, both in the street and in court.  And, in the context of an employment screening, an inconclusive is oft times viewed as a failure.


skip.webb wrote on Oct 26th, 2007 at 1:28pm:
I also realize that this is a mock crime paradigm and as such Mr. Maschke will assert that as there was no real fear of detection or consequences for the perpetrators, the results may not generalize to the public.  Actually, the reverse should be true.  One would assume if polygraph is able to correctly identify the perpetrator in a mock crime scenario then the intensity of reaction in a real life scenario would be greater and therefore easier to discern.

In summary, polygraph works.  

Its utility in criminal investigations is very high.  

Its validity compares very favorably with other forensic tools.  

Its error rate compares very favorably with other forensic tools.



  

"Although the degree of reliability of polygraph evidence may depend upon a variety of identifiable factors, there is simply no way to know in a particular case whether a polygraph examiner's Conclusion is accurate, because certain doubts and uncertainties plague even the best polygraph exams."  (Justice Clarence Thomas writing in United States v. Scheffer, 523 U.S. 303, 118 S.Ct. 1261, 140 L.Ed.2d 413, 1998.)
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Sergeant1107
God Member
*****
Offline



Posts: 730
Location: Connecticut, USA
Joined: May 21st, 2005
Gender: Male
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #4 - Oct 26th, 2007 at 6:44pm
Mark & QuoteQuote Print Post  
I think the fact that all the technicians were told that in each group of four people there was one guilty person hopelessly skews the resulting data.  I think this study is worth very little in determining the effectiveness of anything included in it.

Knowing that one and only one person in each four-person group is guilty, the efficacy of randomly picking one person from each group would be 25%.  Would that make random choice effective enough to appear on the list of tools for criminal investigators?

If the technicians knew that one and only one person from each group was "guilty" all they had to do was look for any indications of guilt in each person.  They could have done that with a simple interview, or via a polygraph, CVSA, handwriting analysis, Tarot card reading, or whatever.  At the end of all four interviews they could just point to the person who exhibited the most "tells" that they were being deceptive.  I think that any police cadet with at least a couple of weeks at the academy could have correctly identified the majority of the guilty people.

It would have been somewhat more impressive if there had been one guilty party in the entire group of eighty people.  If there were, say, ten examiners and each one tested eight people, it would be interesting if seven of those examiners concluded that none of the subjects they tested were being deceptive.  It would be even more interesting if the one examiner who was testing the guilty party was able to correctly identify him or her.
  

Lorsque vous utilisez un argumentum ad hominem, tout le monde sait que vous êtes intellectuellement faillite.
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box George W. Maschke
Global Moderator
*****
Online


Make-believe science yields
make-believe security.

Posts: 6060
Location: The Hague, The Netherlands
Joined: Sep 29th, 2000
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #5 - Oct 26th, 2007 at 7:35pm
Mark & QuoteQuote Print Post  
nopolycop wrote on Oct 26th, 2007 at 2:31pm:
I wonder if this is one of the studies the NAS threw out as not having enough scientific protocols to even be considered.  Anyone know?


The 1978 Widacki & Horvath study (note the mis-spelling of the latter's name in the initial post of this thread) didn't make the cut for serious consideration by the National Academy of Sciences and is not mentioned in its report, The Polygraph and Lie Detection.
  

George W. Maschke
Tel/SMS: 1-202-810-2105 (Please use Signal Private Messenger or WhatsApp to text or call.)
E-mail/iMessage/FaceTime: antipolygraph.org@protonmail.com
Wire: @ap_org
Threema: A4PYDD5S
Personal Statement: "Too Hot of a Potato"
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box skip.webb
New User
*
Offline



Posts: 18
Joined: Oct 15th, 2007
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #6 - Oct 26th, 2007 at 11:10pm
Mark & QuoteQuote Print Post  
George, I'm disappointed!  Are we down to critiquing typos on someone’s name now?  The study cited here was not designed or intended to discover or investigate polygraph accuracy, but rather to compare polygraph utility and accuracy when compared to several other common forms of investigative tools routinely used in investigations.  I didn't present it as anything other than that.  As the NAS was interested in polygraph screening and that was their mandate, I doubt that their failure to include this study in their review was an indictment of the study, (as I’m sure you know) but your insulation might be misleading to others less knowledgeable.
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box George W. Maschke
Global Moderator
*****
Online


Make-believe science yields
make-believe security.

Posts: 6060
Location: The Hague, The Netherlands
Joined: Sep 29th, 2000
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #7 - Oct 27th, 2007 at 5:32am
Mark & QuoteQuote Print Post  
Skip,

I pointed out the misspelling of Horvath's name not to embarrass you (I make make such mistakes too), because it could be significant for anyone searching for the report.

I'm perplexed by your argument that "(t)he study cited here was not designed or intended to discover or investigate polygraph accuracy, but rather to compare polygraph utility and accuracy when compared to several other common forms of investigative tools..." How can you seriously contend that on the one hand this study attempted a comparison of the utility and accuracy of investigative tools (or techniques) including polygraphy, and yet on the other hand maintain that it was not intended to investigate (among other things) polygraph accuracy? Especially when the study provides a table of accuracy rates obtained for each of the techniques (including polygraphy)?

The NAS report's omission of the Widacki & Horvath study from consideration didn't stem from the fact that their focus was on polygraph screening. Other polygraph studies not specifically addressing screening were indeed considered. The Widacki & Horvath study simply didn't make the cut.
« Last Edit: Oct 27th, 2007 at 5:50am by George W. Maschke »  

George W. Maschke
Tel/SMS: 1-202-810-2105 (Please use Signal Private Messenger or WhatsApp to text or call.)
E-mail/iMessage/FaceTime: antipolygraph.org@protonmail.com
Wire: @ap_org
Threema: A4PYDD5S
Personal Statement: "Too Hot of a Potato"
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Sergeant1107
God Member
*****
Offline



Posts: 730
Location: Connecticut, USA
Joined: May 21st, 2005
Gender: Male
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #8 - Oct 27th, 2007 at 9:47am
Mark & QuoteQuote Print Post  
skip.webb wrote on Oct 26th, 2007 at 1:28pm:
The following results were recorded with inconclusive results removed for all:

Polygraph -18 correct identifications, 1 error and 1 inconclusive result for an over all accuracy of 95%.

Handwriting – 17 correct identifications, 1 error and 2 inconclusive results for an overall accuracy of 94%.

Eyewitness – 7 correct identifications, 4 errors and 9 inconclusive results for an over all accuracy of 64%.

Fingerprints – 4 correct identifications, 0 errors and 16 inconclusive results for an overall accuracy of 100%.

Why would the inconclusive results be removed, other than to favorably skew the results for the polygraph supporters?

If you believe it is appropriate to remove inconclusive results, you could easily be left with the following scenario:
One hundred subjects tested, one person correctly scored as "No Deception Indicated", the other ninety-nine scored as "inconclusive" = overall accuracy of 100%???

That doesn't seem to make a lot of sense.
  

Lorsque vous utilisez un argumentum ad hominem, tout le monde sait que vous êtes intellectuellement faillite.
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Barry_C
Very Senior User
****
Offline



Posts: 114
Joined: Oct 17th, 2007
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #9 - Oct 27th, 2007 at 9:37pm
Mark & QuoteQuote Print Post  
If we counted "inconclusives" as errors, then coin-flipping would be more accurate than fingerprinting - and I think those cited numbers are high for what I've seen in the field.

If your doctor said he didn't know what was wrong with you after one test, would he be wrong in his diagnosis?  Wait, there is no diagnosis.
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Hunter
User
**
Offline



Posts: 39
Joined: Oct 15th, 2007
Gender: Male
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #10 - Oct 27th, 2007 at 9:47pm
Mark & QuoteQuote Print Post  
So lets include the inconclusives and you have the following figures, and polygraph really wins big time Sgt. 


Polygraph -18 correct identifications, 1 error and 1 inconclusive result for an over all accuracy of 95%

18=90%  Correct
1= 5%    Inconclusive
1=5%.    Incorrect

Handwriting – 17 correct identifications, 1 error and 2 inconclusive results for an overall accuracy of 94%.

17=85%
   1= 5%
   2+10%
Eyewitness – 7 correct identifications, 4 errors and 9 inconclusive results for an over all accuracy of 64%.

7=35%
4=20%
9=45%


Fingerprints – 4 correct identifications, 0 errors and 16 inconclusive results for an overall accuracy of 100%.
4=20%
0=0%
16=80%
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box raymond.nelson
User
**
Offline


If you win the rat race,
you're still a rat.

Posts: 46
Joined: Oct 22nd, 2007
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #11 - Oct 28th, 2007 at 1:48am
Mark & QuoteQuote Print Post  
This is a good example of some of the complexities surrounding the generic term "accuracy."

digithead has alluded to this before, though the discussion is grossly incomplete.

Test designers know that all tests are imperfect, and have identifiable limitations in terms of types of accuracy they can achieve.

Accuracy with inconclusives included is commonly referred to as sensitivity and specificity. Accuracy without inconclusives might be thoughts of as NPV or PPV. Of course, calculation methods for these statistics can differ, and includes within-set and cross-set variants, depending on your bayesian or inferential emphasis or orientation, and the exact question which one seeks to answer.

One of the dangers here is the gross error of substituting classification accuracy (sometimes called PPV or NPV), for the likelihood or probability of a correct classification of any single case. They are not at all the same according to some of the more common systems of mathematical thinking.

Fingerprinting is a good example of a test with very low sensitivity, but very high accuracy. With some tests, like fingerprinting, specificity is of little interest. Positive matches are the only result of any usefulness.

more later.

r
  

Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Brettski
New User
*
Offline



Posts: 22
Location: Ottawa, Canada
Joined: Jul 13th, 2007
Gender: Male
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #12 - Oct 28th, 2007 at 7:10am
Mark & QuoteQuote Print Post  
Barry_C wrote on Oct 27th, 2007 at 9:37pm:
If we counted "inconclusives" as errors, then coin-flipping would be more accurate than fingerprinting - and I think those cited numbers are high for what I've seen in the field.

If your doctor said he didn't know what was wrong with you after one test, would he be wrong in his diagnosis?  Wait, there is no diagnosis.


I agree, inconclusive results should not be classified as errors. There are plenty of stronger arguments against polygraphs without approaching an argument that is an easy win for pro-polygraphs. However, if inconclusive results weren't held against the examinee as the APA claims, then hypothetically it should be possible for an applicant to get nothing but inconclusive results on a polygraph screening tests, and still get hired by XYZ governement agency. To my understanding, this is not the case.
  
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Sergeant1107
God Member
*****
Offline



Posts: 730
Location: Connecticut, USA
Joined: May 21st, 2005
Gender: Male
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #13 - Oct 28th, 2007 at 7:18am
Mark & QuoteQuote Print Post  
Looking at this from a layman’s point of view, it seems that in every polygraph exam the subject is either being truthful or deceptive.  The results of the polygraph should be a binary solution set; either truthful or deceptive.  If you polygraph one hundred subjects, correctly identify one as DI or NDI, and score the other ninety-nine as inconclusive, I don’t see how that can indicate the polygraph is 100% accurate.  I think that would indicate the polygraph is 1% accurate, according to those numbers.

If you want to compare it to a medical diagnosis, you would have to compare it to a test that looks to identify the presence of one specific thing.  You would want to compare it to a test that also had results in a binary solution set.  If you tested one hundred people for the presence of influenza, and you correctly diagnosed one person as having the virus, but said you simply didn’t know about the other ninety-nine, I don’t think you’d be saying the influenza test was 100% accurate.  An accurate test for the presence of influenza would either detect the presence of the virus or confirm that the virus was not present.

It seems like a given that in every polygraph the subject is either being truthful or deceptive.  If the subjects in the study were asked a question such as, “Did you steal the contents of the package,” it is likely they would all answer that they did not.  Their denial would either be truthful or a lie.  A test purported to detect deception should be able to determine which it is.
  

Lorsque vous utilisez un argumentum ad hominem, tout le monde sait que vous êtes intellectuellement faillite.
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Hunter
User
**
Offline



Posts: 39
Joined: Oct 15th, 2007
Gender: Male
Re: Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  
Reply #14 - Oct 28th, 2007 at 8:33pm
Mark & QuoteQuote Print Post  
According to your logic, fingerprints should never be used, they are not 100% accurate when you include the inconclusive rate.Somewhere there is a fallacy in your logic.  Maybe you should re evaluate what you have posted and see if you can correct it.
  
Back to top
 
IP Logged
 
Polygraph Utility – How Does Polygraph Compare to Other Forensic Tools?  

Please type the characters that appear in the image. The characters must be typed in the same order, and they are case-sensitive.
Open Preview Preview

You can resize the textbox by dragging the right or bottom border.
Insert Hyperlink Insert FTP Link Insert Image Insert E-mail Insert Media Insert Table Insert Table Row Insert Table Column Insert Horizontal Rule Insert Teletype Insert Code Insert Quote Edited Superscript Subscript Insert List /me - my name Insert Marquee Insert Timestamp No Parse
Bold Italicized Underline Insert Strikethrough Highlight
                       
Insert Preformatted Text Left Align Centered Right Align
resize_wb
resize_hb







Max 200000 characters. Remaining characters:
Text size: pt
More Smilies
View All Smilies
Collapse additional features Collapse/Expand additional features Smiley Wink Cheesy Grin Angry Sad Shocked Cool Huh Roll Eyes Tongue Embarrassed Lips Sealed Undecided Kiss Cry
Attachments More Attachments Allowed file types: txt doc docx ics psd pdf bmp jpe jpg jpeg gif png swf zip rar tar gz 7z odt ods mp3 mp4 wav avi mov 3gp html maff pgp gpg
Maximum Attachment size: 500000 KB
Attachment 1:
X