Hunter wrote on Oct 28^th, 2007 at 8:33pm:


Hunter:
Re-evaluate what you said.
In the fingerprinting exercise there were 0 errors.
The correct calls are verifiable by direct comparison, there can be no error in the lab nor the field.

In a lab setting with students, errors and correct calls can be verified.
In the field however, correct calls and errors can seldom be verified.
That was the crux of Sarge's post. Unless your subject verified for you - you dont actually know.
You only THINK that you know.

And history tells us that many examiners thought wrong.

Sergeant1107 wrote on Oct 30^th, 2007 at 1:07am:


Now there's some ground truth for the statisticians and pseudo scientists that daily swamp this board with pie-in-the-sky psychobabble and even though some declare that the AP folk bore them, continually
return with their daily tsunami dose of BS. 

EJohnson wrote on Oct 29^th, 2007 at 4:06pm:

My post was meant to show that a person who uses the polygraph to screen potential employees need not have a belief in the ability of the polygraph to detect deception.   

I have personally spoken with two chiefs of police in Connecticut (where polygraph screening for police officers is required by law) who said they are aware the polygraph cannot "detect lies", but that it can be effective at getting applicants to admit to various things.

I wasn't endorsing E Prince and my point was that despite Prince's obvious controversial history, power, and alliances---and having failed his CIA polygraph which according to many people here----cost him a job by virtue of the stated draconian practices of "DI means DQ", he would seem to be the cat who would not use polygraph, rather than hiring 100's of us.

FYI, I'm not a huge Blackwater fan these days---but besides the point.

EJohnson wrote on Oct 29^th, 2007 at 1:21pm:


I would hardly think that Erik Prince should be polygraph's poster boy.

Bad choice mate.









[/quote]

Sergeant1107 wrote on Oct 29^th, 2007 at 3:53pm:


So does your question imply that a person who was falsy accused of lying could then go on to use the instrument in the way that you stated? Unlikely. You have no faith in poly screening per your historic statements, so I gather that you wouldn't use the poly if it were your multi-million dollar operation. Call me crazy.

EJohnson wrote on Oct 29^th, 2007 at 1:21pm:


Don't you believe there is the distinct possibility that he, like many others, believe the polygraph is incapable of detecting truth or deception, but that it does serve to elicit damaging confessions from people who believe it will "detect lies?"

Using the polygraph in such a manner is hardly an endorsement of its scientific legitimacy.  If Blackwater was screening employees who believed that a crystal ball could detect lies the crystal ball would be exactly as effective as the polygraph in eliciting confessions, and it would be exactly as capable of detecting truth or decption.

Quote:



Quote:

Error rates are too high for what C? Despite the disinformation, decisions regarding incarceration and/or applicant rejection as a sole result of polygraph are simply not tolerated, period. Perhaps your concern for "human collateral" stems from a modality from a seperate country---not in the US at this time. So many people are rejected by fed intel and cop shops because the officials seek individuals who are cleaner (can you say boringingly squeeky) than the vast majority of US citizens---in the scope of drugs, sexual history, and theft especially. Let a dog lick your balls as a teen?...well, the chief might not like that ----regardless if it was a typical teenage fancy. Steal cash from grandma while she was in the hospitol?--the cheif might have a soft spot for grandmothers-----but typically many things are overlooked unless they present as unashamed or perpetuating. Individuals, say, in an applicant screening situation, are rejected over many factors---many of which are not told to the applicant, as this is certainly the case with any job whether poly is used or not. Most times the applicant just doesn't have all the right qualities, and the polygraph is scapegoated. Take Erik Prince, the controversial owner and (former Navy Seal) of Blackwater USA, he failed his CIA poly, but it is pretty much accepted that he snagged other concerns--having garnered the reputation of being critical of the Naval Institue as a young attendee (he quit bitterly), and being a right-wing extremist (long story) among other slightly "off" personal attributes such as alleged recklessness (the CIA's greatest phobia)-------but was only told at that time that he simply "failed" his polygraph (according to the latest Newsweek magazine article on Blackwater's Prince.) I find it quite telling that now that he has such high accesses and clearences, that he continues to use polygraph pervasively within the Blackwater organization. Bare in mind that Prince calls the shots and he has a love affair with applicant screening modalities, and if he felt that polygraph was a pseudoscience he would certainly ignore the usage potentials.

ps. 1904, your opinion has never been "humble" and neither has mine. 

Hunter wrote on Oct 27^th, 2007 at 9:47pm:


If in field testing the 5% Incon are called NDI, the error rate is 10%
That is too high in terms of human collateral.

In reality, the errors in field testing are likely to be considerably in excess of 10%.
The NAS study said as much, I'm just too lazy to find the link right now.

Whether error rates are attributable to the polygraph itself, or to inept examiners is disputable.

My humble opinion is that far too many examiners are gung ho, aggressive and inept. 
Aspects that have been highlighted many times by reversed incarcerations and by some who post on this board.

You're looking for influenza or an absence of influenza, just as you are looking for deception or an absence of deception.  In both tests, you will have inconclusives.  Just because you call the absence of deception "truth" (which it is), doesn't make them (or rather, the analogy) all that different.  After all, you could call the absence of influenza "healthy" (and influenza, "sick").

According to your logic, fingerprints should never be used, they are not 100% accurate when you include the inconclusive rate.Somewhere there is a fallacy in your logic.  Maybe you should re evaluate what you have posted and see if you can correct it.

Looking at this from a layman’s point of view, it seems that in every polygraph exam the subject is either being truthful or deceptive.  The results of the polygraph should be a binary solution set; either truthful or deceptive.  If you polygraph one hundred subjects, correctly identify one as DI or NDI, and score the other ninety-nine as inconclusive, I don’t see how that can indicate the polygraph is 100% accurate.  I think that would indicate the polygraph is 1% accurate, according to those numbers.

If you want to compare it to a medical diagnosis, you would have to compare it to a test that looks to identify the presence of one specific thing.  You would want to compare it to a test that also had results in a binary solution set.  If you tested one hundred people for the presence of influenza, and you correctly diagnosed one person as having the virus, but said you simply didn’t know about the other ninety-nine, I don’t think you’d be saying the influenza test was 100% accurate.  An accurate test for the presence of influenza would either detect the presence of the virus or confirm that the virus was not present.

It seems like a given that in every polygraph the subject is either being truthful or deceptive.  If the subjects in the study were asked a question such as, “Did you steal the contents of the package,” it is likely they would all answer that they did not.  Their denial would either be truthful or a lie.  A test purported to detect deception should be able to determine which it is.

Barry_C wrote on Oct 27^th, 2007 at 9:37pm:


I agree, inconclusive results should not be classified as errors. There are plenty of stronger arguments against polygraphs without approaching an argument that is an easy win for pro-polygraphs. However, if inconclusive results weren't held against the examinee as the APA claims, then hypothetically it should be possible for an applicant to get nothing but inconclusive results on a polygraph screening tests, and still get hired by XYZ governement agency. To my understanding, this is not the case.

This is a good example of some of the complexities surrounding the generic term "accuracy."

digithead has alluded to this before, though the discussion is grossly incomplete.

Test designers know that all tests are imperfect, and have identifiable limitations in terms of types of accuracy they can achieve.

Accuracy with inconclusives included is commonly referred to as sensitivity and specificity. Accuracy without inconclusives might be thoughts of as NPV or PPV. Of course, calculation methods for these statistics can differ, and includes within-set and cross-set variants, depending on your bayesian or inferential emphasis or orientation, and the exact question which one seeks to answer.

One of the dangers here is the gross error of substituting classification accuracy (sometimes called PPV or NPV), for the likelihood or probability of a correct classification of any single case. They are not at all the same according to some of the more common systems of mathematical thinking.

Fingerprinting is a good example of a test with very low sensitivity, but very high accuracy. With some tests, like fingerprinting, specificity is of little interest. Positive matches are the only result of any usefulness.

more later.

r

So lets include the inconclusives and you have the following figures, and polygraph really wins big time Sgt.   


Polygraph -18 correct identifications, 1 error and 1 inconclusive result for an over all accuracy of 95%

18=90%  Correct
1= 5%    Inconclusive
1=5%.    Incorrect

Handwriting – 17 correct identifications, 1 error and 2 inconclusive results for an overall accuracy of 94%.

17=85%
   1= 5%
   2+10%
Eyewitness – 7 correct identifications, 4 errors and 9 inconclusive results for an over all accuracy of 64%.

7=35%
4=20%
9=45%


Fingerprints – 4 correct identifications, 0 errors and 16 inconclusive results for an overall accuracy of 100%.
4=20%
0=0%
16=80%

If we counted "inconclusives" as errors, then coin-flipping would be more accurate than fingerprinting - and I think those cited numbers are high for what I've seen in the field.

If your doctor said he didn't know what was wrong with you after one test, would he be wrong in his diagnosis?  Wait, there is no diagnosis.

skip.webb wrote on Oct 26^th, 2007 at 1:28pm:

Why would the inconclusive results be removed, other than to favorably skew the results for the polygraph supporters?

If you believe it is appropriate to remove inconclusive results, you could easily be left with the following scenario:
One hundred subjects tested, one person correctly scored as "No Deception Indicated", the other ninety-nine scored as "inconclusive" = overall accuracy of 100%???

That doesn't seem to make a lot of sense.

Skip,

I pointed out the misspelling of Horvath's name not to embarrass you (I make make such mistakes too), because it could be significant for anyone searching for the report.

I'm perplexed by your argument that "(t)he study cited here was not designed or intended to discover or investigate polygraph accuracy, but rather to compare polygraph utility and accuracy when compared to several other common forms of investigative tools..." How can you seriously contend that on the one hand this study attempted a comparison of the utility and accuracy of investigative tools (or techniques) including polygraphy, and yet on the other hand maintain that it was not intended to investigate (among other things) polygraph accuracy? Especially when the study provides a table of accuracy rates obtained for each of the techniques (including polygraphy)?

The NAS report's omission of the Widacki & Horvath study from consideration didn't stem from the fact that their focus was on polygraph screening. Other polygraph studies not specifically addressing screening were indeed considered. The Widacki & Horvath study simply didn't make the cut.

George, I'm disappointed!  Are we down to critiquing typos on someone’s name now?  The study cited here was not designed or intended to discover or investigate polygraph accuracy, but rather to compare polygraph utility and accuracy when compared to several other common forms of investigative tools routinely used in investigations.  I didn't present it as anything other than that.  As the NAS was interested in polygraph screening and that was their mandate, I doubt that their failure to include this study in their review was an indictment of the study, (as I’m sure you know) but your insulation might be misleading to others less knowledgeable.

nopolycop wrote on Oct 26^th, 2007 at 2:31pm:


The 1978 Widacki & Horvath study (note the mis-spelling of the latter's name in the initial post of this thread) didn't make the cut for serious consideration by the National Academy of Sciences and is not mentioned in its report, The Polygraph and Lie Detection.

I think the fact that all the technicians were told that in each group of four people there was one guilty person hopelessly skews the resulting data.  I think this study is worth very little in determining the effectiveness of anything included in it.

Knowing that one and only one person in each four-person group is guilty, the efficacy of randomly picking one person from each group would be 25%.  Would that make random choice effective enough to appear on the list of tools for criminal investigators?

If the technicians knew that one and only one person from each group was "guilty" all they had to do was look for any indications of guilt in each person.  They could have done that with a simple interview, or via a polygraph, CVSA, handwriting analysis, Tarot card reading, or whatever.  At the end of all four interviews they could just point to the person who exhibited the most "tells" that they were being deceptive.  I think that any police cadet with at least a couple of weeks at the academy could have correctly identified the majority of the guilty people.

It would have been somewhat more impressive if there had been one guilty party in the entire group of eighty people.  If there were, say, ten examiners and each one tested eight people, it would be interesting if seven of those examiners concluded that none of the subjects they tested were being deceptive.  It would be even more interesting if the one examiner who was testing the guilty party was able to correctly identify him or her.

A few additional observations.  In the first paragraph quoted below, I believe you are WAY OFF BASE.  From what I have read about polygraphy, the more emotion that is attached to a particular event, the more likelyhood of incorrect findings, not the reverse.  Remember, the students were not really guilty of anything, they just followed directions.

#2. Huh?  How can you conclude this, Mr. Webb, based on this study?

#3.  I actually agree with you here, as long as the investigators do not rule out the person as a suspect if they "pass".  Can you say "Gary Ridgway?"

#4.  That is why a good criminal investigator does not rely solely on any investigative tool. 

#5.  I am not sure I can agree with this claim, since the protocols of the study were not given.  I also find it interesting that in this analysis, an inconclusive is removed, whereas in the real world, investigators of crime have to deal with an inconclusive finding, both in the street and in court.  And, in the context of an employment screening, an inconclusive is oft times viewed as a failure.


skip.webb wrote on Oct 26^th, 2007 at 1:28pm:

But Mr. Webb, in the study by Honts attempting to ascertain if countermeasures worked, the resultant rate for polygraphy in general was only 73% accurate.  Now, I realize that accuracy wasn't being specifically tested in that study, but it certainly was a "finding" of the study, and reported as such.

There also in not nearly enough information given regarding how the polygraph was administered to form any conclusions.  Did the person confess?  Did the polygraph give them away.  How were they interrogated?   

Lastly, what exactly were the "guilty" persons guilty of?  Following instructions.  No Mr. Webb, I am not impressed.  With all due respect,  if this is the type of "science" that needs to be resorted to to prove the utility of the polygraph, the polygraphy is in big trouble.

I wonder if this is one of the studies the NAS threw out as not having enough scientific protocols to even be considered.  Anyone know?

Mr. Webb,

I don't deny that the polygraph may have utility as an aid to interrogation, to the extent that it may help to obtain confessions/admissions that an examinee might not have otherwise been willing to make. But validity is an altogether different question.

You write:

Quote:


But the National Academy of Sciences, in its review  of the scientific evidence on the polygraph, disagrees (at pp. 3-4):

Quote:


Dr. Alan Zelicoff, M.D., conducted a statistical analysis (255 kb PDF) of data from polygraph field studies selected from an impeccably pro-polygraph source and found the data indicate that "if a subject fails a polygraph, the probability that she is, in fact, being deceptive is little more than chance alone; that is, one could flip a coin and get virtually the same result for a positive test based on the published data."

In an article published in the Journal of Forensic Science 23 (3), 596-601, published in 1978 (almost 30 years ago) by Jan Widacki and Frank Forvath, entitled “An experimental investigation of the relative validity and utility of the polygraph technique and three other common methods of criminal identification.”, researchers compared the polygraph, handwriting analysis, eye witness testimony and fingerprints in a mock crime scenario.  80 subjects (college students) were divided into 20 groups of 4.  In each sub-group there was one guilty participant and three innocent ones.  The task of the guilty participant was to go to a particular building; present themselves to a person whom they did not know before hand;  give an envelope to the specified person; receive a package using a fictitious name; sign a receipt for the package using the fictitious signature and then steal the contents of the package.

All participants were then subjected to a polygraph.  The document receipts and envelopes were subjected to handwriting analysis compared against handwriting standards supplied by all participants.  Fingerprints collected from the envelope and receipts were compared to record prints of all participants.  The eye witnesses to the crimes were provided full front photographs from which to select the perpetrator who signed for and stole the packages.  All forensic technicians were blind to the identity of the perpetrators as was the eye witness but all technicians were aware that one in each group of four was the perpetrator and three were innocent as were the eyewitnesses.

The following results were recorded with inconclusive results removed for all:

Polygraph -18 correct identifications, 1 error and 1 inconclusive result for an over all accuracy of 95%.

Handwriting – 17 correct identifications, 1 error and 2 inconclusive results for an overall accuracy of 94%.

Eyewitness – 7 correct identifications, 4 errors and 9 inconclusive results for an over all accuracy of 64%.

Fingerprints – 4 correct identifications, 0 errors and 16 inconclusive results for an overall accuracy of 100%. 

Submitter’s comments:  I have provided this study for thoughtful consideration of those on this site who require sound evidence rather than unsupported assertions or ad hominid attacks on polygraph examiners. I realize that students were used so Mr. Maschke may object but as I previously pointed out, students are used in many studies as such studies are an academic endeavor usually conducted in a university setting.  I also realize that this is a mock crime paradigm and as such Mr. Maschke will assert that as there was no real fear of detection or consequences for the perpetrators, the results may not generalize to the public.  Actually, the reverse should be true.  One would assume if polygraph is able to correctly identify the perpetrator in a mock crime scenario then the intensity of reaction in a real life scenario would be greater and therefore easier to discern. I also realize that in the instant study, the polygraph examiners, although blind to the identity of the perpetrator in each group were aware that one of the four was the guilty party.  This does create an advantage but each of the forensic technicians as well as the eye witnesses shared that same advantage.   In summary, polygraph works.  Its utility in criminal investigations is very high.  Its validity compares very favorably with other forensic tools.  Its error rate compares very favorably with other forensic tools. One can argue that any error rate is unacceptable if it does harm to one innocent person.  If only we lived in such a perfect world.  Look back at the eye witness testimony results and ask your self the following questions: “If I am ever on a jury and I am faced with the testimony of an eye witness, how much weight will I give it?”  If I am ever a defendant in a case how much credence do I want the jury to give eye witness testimony?  Well, my friends, such testimony forms a major part of many trials.