Lane99,

You say in part:

Quote:


Precisely.

Another (of many) issue of concern would be the selection process for the 100 charts presented.  Are they randomly selected from a sufficiently large pool?  Or are they cherry picked to be "clear," i.e., what one former examiner use to call "Stevie Wonder" charts (even a blind man could read).  If the latter were the case, obviously these would not represent "real world" charts and the accuracy involved in scoring same.

Who selects these charts and what professional relationship does that person or persons have to those evaluating the charts.  Is there a conflict of interest, e.g., having someone from the polygraph community (that tends to benefit from higher/inflated accuracy rates) select these charts, etc?

Because ASTM (as far as I know) does not conduct polygraph exams, it might be rather easy to have such considerations glossed over.

Bill_Brown wrote on Aug 23^rd, 2011 at 5:19am:


Fair enough.  But it might have been better if you had confined your comments to that in the first place. Rather than making a very misleading sweeping generalization about what other examiners did or didn't know.  

Bill_Brown wrote on Aug 23^rd, 2011 at 5:19am:

 
Perhaps you misunderstand the significance of this. It's not that that information alone would ensure testees of an 86% score.

The point is that that information alone does ensure that the test is not truly blind.  But, rather, rigged in favour of producing a higher score than a legitimately blind test would.

Anyone know if organizations certifying examiners under the Marin Protocol these days are using Evidentiary or Investigative Decision Rules?   

ASTM Standard E2324-04 doesn't say, so either the organizations themselves decide this little detail - or the examiners are using whatever decision rules they wish.

lane99

You may assume what you will.  I did not know how many were deceptive or truthful when I became certified under the Marin Protocol.  I scored in the "blind" totally.  

Even with the that information available to an examiner how would the examiner be able to obtain a minimum of 86% to qualify. The "Center" is not a center it is ASTM.  It has nothing to do specifically with polygraph other than establishing scientific standards for all industries.  

However, you may believe as you desire.  I will not argue the point further. 

Bill_Brown wrote on Aug 15^th, 2011 at 3:11pm:


As I understand it, there is some "Center" which is granting some "certification" to some "examiners".

And the examiners applying for certification have access to the information that is quoted above.

If so, then surely the examiners know, in advance of the certification test, that, of the 100 charts they're shown, 50 need to be judged deceptive, and 50 non-deceptive.

Examiners were not aware that 50% were deceptive and 50% were truthful.  They did score Blind.

Thanks for the interesting comments, guys.

I might wonder if, in the certification process, the testees know, in advance, that 1/2 the charts are from deceptive subjects, and 1/2 are from truthful subjects?  And, further, know that they won't be confronted with any charts that are considered inconclusive.

If so, the certification process doesn't seem legitimately "blind". Rather it's bit rigged, it seems to me.

We were discussing the topic of inter rater reliability.  I used the protocol known as the Marin protocol to demonstrate there is inter rater reliability demonstrated by this protocol.    


Certifies:
The Center certifies qualified examiners, in accordance with 
ASTM Standard E2324-04 (The Marin Protocol)
Examiners applying for certification must submit an end-to-
end videotape of an examination showing their proficient 
adherence to a validated methodology, and the interpretable 
chart produced by that examination. They must then blind-
score 100 sets of charts generated in real-world examinations 
in cases where the truth is now known, fifty of which are 
from examinations where the subject was deceptive and the 
rest from examinations where the subject was non-deceptive. 
The test sets are selected from a collection of more than 400 
sets. The applicant is certified if and only if he declares a 
conclusive result for at least 80 per cent of each group of 
fifty, and at least 86 of his conclusive results for each 
group are correct.

Bill_Brown wrote on Aug 14^th, 2011 at 4:14pm:



May be you should tell the polygraphers who practice teh Marin protocol that they dont know what it means. 

http://www.gsrsystems.com/examtypes/marin_protocol.html

http://www.veritascenter.org/

That, or it means one thing in practice and some thing els in you're post.

figs, 

Found this on the APA website, another study to look at: 

[quote] A Replication and Validation Study on an Empirically Based Manual Scoring System1
Ben Blalock, Barry Cushman & Raymond Nelson
Abstract
This is a replication of a study validating the hand scoring system for comparison question polygraph examinations proposed by Nelson, Krapohl and Handler (2008). Nine polygraph examiner trainees at an American Polygraph Association accredited polygraph school used an empirically based three-position manual scoring system involving three evaluative criteria and a reduced set of basic rules to evaluate 100 confirmed event-specific single-issue criminal investigation polygraph examinations from the Department of Defense Polygraph Institute confirmed case archive. Average decision accuracy for the inexperienced examiners was 88% with 13.1% inconclusives. Sensitivity and specificity levels achieved by the trainees did not differ significantly, suggesting they achieved balanced accuracy characteristics using the empirically based scoring system. All nine of the inexperienced examiners scored the sample cases with sufficient accuracy to meet the accuracy requirements specified by the Marin protocol (Krapohl, 2005; Marin, 2000).      Results from this study parallel the results reported in the previous experiment and support the validity of an empirically based three-position manual scoring method. [quote]

Stefano, 

We are trying to get a handle on those examiners you mention.  It is difficult to dictate procedures to individuals that have no respect for proper procedures.  There is no legal authority vested in any examiner that allows censorship.  New rules and guidelines are being enacted by the APA, but again they are not enforceable.  

I am not blind to this problem and do not condone examiners that are not totally professional, I have no power to stop them.  I have lobbied for State and Federal regulation of polygraph.  Even these State Polygraph Examiner Boards do not dictate or enforce "best practices" because we as examiners have not pushed hard enough against our opposition to have them enacted.  Hopefully we will get a handle on this problem in the future, it does not look good right now.

figs, 

Quote:


The Marin protocol requires the examiner score 100 known solution cases with a  minimum of 86% accuracy.  That is inter rater reliability.  This is to qualify under that system as an expert witness in court using paired testing.  And other studies I have seen and not quoted are in the same area of inter rater reliability. 

I will look later and find other studies, sorry I just don't have the time right now.  

Bill_Brown wrote on Aug 13^th, 2011 at 5:39pm:

Of course it was a setup; that was the whole idea. Bill, you know that you are one of the few polygraphists that I have come to respect, but you seem to be blind to the plethora of arrogant polygraphists out there who have taken it upon themselves to do whatever they please apparently without peer scrutiny. It seems that simply going through the 320 hour course somehow makes them untouchable and immune to peer criticism--much like the cops that support each other regardless of whatever travesty they choose to precipitate. There is an examiner in my area who takes the $500, accuses the examinee of attempting countermeasures and gives an Inconclusive. There is no oversight, no scrutiny, they consider themselves demigods and will do as they please. You and others in your profession refuse to take the shitbirds to the woodshed.

Bill_Brown wrote on Aug 13^th, 2011 at 9:49pm:


The Marin protocol is irelevent to interrater reliability. So are all the studies you cite. 

Teh Horowitz study you cite even concludes "The R-I test produced an unacceptable rate of false positive decisions." 

You dont offer a cit by Donald Krapohl. Maybe you can dig up something responsive? I can't, but your the expert. 

figs wrote on Aug 13^th, 2011 at 4:13am:


Podlesny & Raskin (1978) Rovner et al. (1979) Kircher & Raskin (1988) Honts et al. (1994) Horowitz et al. (1997)

There are newer studies also, check the Marin protocol also.  Inter rater reliability is about 86% in that particular study by Krapol.  

That CBS Expose was in 1986, over 20 years ago.  It was also a setup.

Quote:


Haven't you heard of the CBS 60 Minutes Exposé where all the (randomly chosen) polygraph examiners from different companies in New York each repeatedly fingered the person identified as the suspect by the boss, even though the suspect was changed with each round of tests?

The fact that they even had those conversations and allowed themselves to hear who was suspected casts doubt on their honesty. A polygraph examiner who truly stuck to reading the charts would not need to find out which person was suspected before identifying that person as the culprit.

https://antipolygraph.org/blog/?p=110

No matter what anyone says, it can't hurt to follow the advice in "The Lie Behind The Lie Detector" about dressing right, having a good attitude and not acting suspicious. When I read that I realized that I had broken every rule the times that I had failed the polygraph. I was sullen, made many excuses for why I might not have good test results and jabbered on and on nervously, instead of just answering the questions in a confident, businesslike manner.

Bill_Brown wrote on Aug 12^th, 2011 at 4:46pm:


Publication citation, please.

Chuckles, 

We only take data from the chart in making a determination regarding responses.  Totally chart interpretation, nothing to do with how you carry yourself, attitude or suspicion.

lane99, 

I have been involved in studies of examiners scoring charts, and we found examiners do have high agreement rates when scoring charts.  Properly trained examiners are in agreement on results of polygraph charts. 

A polygraph examiner takes many things into consideration when deciding if a person seems deceptive or not, including how the subject carries himself, the subject's attitude and if the subject acts suspiciously. They also take into consideration the opinion of the person calling for the test. If the head guy says he thinks a certain subject is guilty of something, the polygraph examiner is way more likely to say that the "test is picking something up."

I don't believe polygraphs can distinguish between lies and truth.

However, out of curiousity, are they even objective enough so that the charts produced will be read the same by all polygraphers?

If a test is run, and the chart is given to 10 polygraphers, will all 10 (or at least the vast majority of them) come to the same conclusion as to "truthful" or "deceptive"?

Been wondering about this for a while.  I'm rather assuming most blind polygraphers WOULD come to the same conclusion on any given chart.  But not sure about it.