Last week a research paper was published exactly on this topic (polygraph QA). Their major point is building an AI 2nd opinion for screenings. 
They also make several additional points as they go, like:
- human QA is not perfect since examiner errors happen in part because the case is complicated, so human QA is likely to make the same mistake
- experimental lower bound for examiner error rate...   

https://www.nature.com/articles/s41598-023-31775-6

Thanks nonombre.

alterego1 wrote on Aug 2^nd, 2006 at 5:17pm:


Alterego,

Although we are a fairly small team, we quality control every polygraph exam we run (crim specific and screening).  We believe this to be an important part of a good polygraph operation.  And yes, there are times Q.C. will non-concur with the examiner.  When that happens we run the test again..

Regards,

Nonombre

Nonombre,

From what I have gathered from your posts, you work for a local PD.  Do you guys do any form of QC, or are you the last word on all poly charts??

Sergeant1107 wrote on Aug 1^st, 2006 at 6:42am:


Seargeant,

You and are are still coming at this from two different places.  I am looking at the applicants who pass the screening polygraph exams as well as the DI cases that are resolved (the 90% to 95%).

You keep pointing out the few who are not ultimately resolved one way or the other (although I have no doubt most of those applicants know exactly why they failed the test).

So with all due respect, I look at it this way:  From a personnel processing standpoint, I am consuming a large bowl of fried rice with a soup spoon.  In short order, I have taken in all the successfull candidates I need to fill the next acadamy class while you are on your hands and knees trying to pick up a few lost sticky kernals out of a shag carpet with a pair of greasy chop sticks...

I'm sorry Sergeant, but in the real world of applicant screening that is just the way it is...

Regards,

Nonombre

nonombre wrote on Aug 1^st, 2006 at 1:29am:

Nonombre,

I do not assume that every last one of the 5-10% are telling the truth.  I never said I did and I don’t recall ever trying to make people believe that.  I also never said that such an extreme was necessary for the polygraph to be deemed worthless.

My point is that you came up with the 5-10%, and yet you don’t really know if that number is accurate.  It could be one person out of hundred who fails and yet is telling the truth, or it could be fifty out of a hundred failures.  

By your own admission you don’t know how many people tell the truth and yet for some reason “fail” their polygraph.  But you are perfectly willing to show them the door because there are others waiting in line behind them.  

By your own words you are comfortable with a process which rewards an undetermined percentage of truthful people with a “failed” test score and a DQ on their police application.  You seem to be okay with that for no other reason than there are more applicants where those came from…  I think that an open-minded, reasonable person would find that disconcerting, at least…

I think it’s great that you acknowledge the possibility of a false-positive.  Now what are you going to do about it?  

Once you acknowledge that a person can tell the truth and “fail” what is the purpose of the polygraph?  A “DI” means that the person was lying and was caught, or tried countermeasures and was caught, or told the truth and was a false-positive.  An “NDI” means that the person told the truth, or lied and wasn’t caught, or used CM’s and wasn’t caught.  

Without a disqualifying admission, what do you really know for certain after a polygraph, regardless of how it was scored?

I don't want to seem like a kiss-ass to nonombre, nor am I betraying the principle of this site, but I do agree with Nonombre...if, in fact, they do retest the people that have problems to ensure "that's all there is."

I have not taken the FBI poly, and it seems a large number on these message boards have.  It does sound as though the polygraph exams these people have gone through (1st poly, 2nd poly, 3rd poly) with the FBI don't get to "make sure that's all there is" in a fair manner.  It truly sounds as though they fail before the even begin the test.

Nonomre, would you agree that there is a good possibility that the polygraph is used by some agencies in order to weed out those applicants that they can't fail for some other portion of the testing, and use the polygraph as a "safety net" in order to get rid of them?

Sergeant1107 wrote on Aug 1^st, 2006 at 12:55am:


Sergeant,


You and I are BOTH making assumptions here.

I am assuming that healthy percentage of the 5%-10% are deliberately withholding information they do not wish to share.

You are assuming that every last one of the 5% to 10% are innocent/truth telling people that the polygraph identified in error.

At least I admit that false positives occur.  You try to make people believe that every last member of the 5% to 10% group have been "wronged" by the polygraph process.  You then use that as justification to call me "unethical."

Frankly, I am dissapointed in you.  I thought you were more open minded and reasonable.

Regards,

Nonombre

nonombre wrote on Aug 1^st, 2006 at 12:23am:

Nonombre,

So, 5-10% of people who fail the polygraph are simply written off because they have “problems” with the polygraph portion of the application process?  Even though they may, for all you know, be telling the complete truth and not withholding any information at all?

Imagine if all police polygraph examiners, including yourself, were given a CVSA test to root out possible corruption.  You tell the truth during your test but your examiner tells you that you have failed and he has no idea why, since he’s not a mind reader.  However, since you failed, you will be terminated anyway because 90-95% of the other people taking the CVSA didn’t seem to have a problem with it.  Would you consider your career to be an acceptable loss for the greater good?  Or would feel the process itself was unfair and should be scrapped?

Continuing to use the polygraph despite your own admission that applicants may very well be telling the truth and still failing is simply unethical.  A flawed process does not become acceptable simply because you believe it to be less flawed than other similar processes.  If it doesn’t work it shouldn’t be used.

I am curious to discover how you pegged the percentage of applicants who fail for an unknown reason at 5-10%?  Since you admitted you are incapable of reading minds and you don’t why this percentage of people are having troubles with the relevant questions, how do you know it isn’t a greater percentage?  How do you know that the applicants who “pass” were not lying and happen to fall into a group of 5-10% which, for unknown reasons, show no signs of deception?  Unless you obtain a disqualifying admission during your interrogation, how do you ever know for certain that the applicant who "fails" isn’t telling the truth and not withholding any information?

Based on your own admissions, I think it is clear the answer is that you don’t know.

I truly don’t understand your argument that it is acceptable to disqualify those who “fail” for unknown reasons (even though they may be telling the truth) simply because there are more applicants in line behind them who may not fail.   

Sergeant1107 wrote on Jul 31^st, 2006 at 10:48pm:


Sergeant,

And now we come full circle to the arguement I have posted in months past.  On average, 90-95% of the people who "fail" my polygraph examination, provide information that explains the reason for their failure.  They are then tested to make sure that is "all there is."  Generally it works out in the end.  In those cases where the information is not disqualifying, they proceed with the process.  In those cases where it is, then they were not meant for this job in the first place.  Quite simple actually.

Now, for the 5% to 10% that don't/can't explain their responses?  The question one has to ask themselves is what is truly the percentage of that group who truly don't know or can't explain why they are having issues with the relevant questions?  What part of the group is simply refusing to admit to something they know/fear will disqualify them (or worse?).  I truly do not know because I am not a mind reader.  All I know is that there are lots of people standing in line behind them who don't have problems with this part of the process.  I know you don't buy into the "acceptible losses" argument, but in some cases I am afraid that is what it turns out to be.

You know, if you were to look back on my earliest postings, you would see that I suggested a way through this potential problem.  Of course that is simply theory at this point, but if it were up to me, I would apply it.

Regards,

Nonombre
    

nonombre wrote on Jul 30^th, 2006 at 10:07pm:

Nonombre,

Everyone has their own opinions.

In my experience the possibility of “failing” the test while telling the complete truth and not withholding any information is about 75%.

I failed my first three pre-employment polygraphs while telling the truth and disclosing everything.

I think it is quite honorable of you to admit there is a possibility, however remote you believe it to be, that a subject may tell the truth and still “fail” their polygraph exam.  Since you admit that possibility I am curious what you do when someone “fails” a polygraph with you?   

EosJupiter wrote on Jul 31^st, 2006 at 4:45am:


Eosjupiter,

Ah, you are indeed a worthy adversary (I have always liked your spirit).  Yes, due to advances in procedures and technology the battle has just shifted to a new level...

Improvision and adaptation on both sides.  How exciting.  I look forward to the contest...

This website is getting to be "fun" again.  I may stay around for awhile....

Regards,

Nonombre

Nonombre,

Well as the case may be, he did not get caught using CM's by the polygrapher. Now as for the accuracy of a QC piece of software by the feds ... well lets see how much credibilty that gets here. He got through the first hoop but was killed by a piece of software that may or may not exist. But the bottom line is to learn from this and adjust. Don't do the test perfectly everytime. Make a couple of known mistakes and adapt. If this QC software is so dam good, lets see how well it handles adaptation. And we know about this, time to adjust the advice here.  I will be willing to bet that the algorithms are beatable.  But its good to get your side. ANd from past experience ... I always feel lucky !!! 

Regards ....

Quote:


Triple X, please don't mince words.  It does not matter who "caught" you.  You were accurately and properly identified by the process as attempting to employ coutermeasures and it cost you.  In your post, you keep repeating you were only "suspected."  Therefore, am I to believe the FBI said "Well, since headquarters only "suspects" you of using countermeasures, that's okay.  Congrats!  You start your new job on Monday?"

Now Triple X, to my next point.  Put yourself in the place of the truthful applicant reading this website for the first time.  As the result of what he/she has read, here are what seem to be his choices:

1.  Fully cooperate and put his trust in the polygraph process.  Now by what the examinee has read on this website, he has learned it is possible to fail the test anyway.  As an examiner, I do admit that is a possibility, but my experience tells me that possibility is an extremely REMOTE one.

2.  Follow the advice of this website and employ behaviors he has read will "help" him pass the polygraph examination.  Now maybe he will be successfull in this endevor, maybe not.  I have personnally caught a lot of folks trying.  Now I am a pretty nice guy and I always give examinees another chance to cooperate; most do.  Some examiners are not as accomidating (sp), and immediately throw the applicant out of the room.

3.  I am not claiming to have caught them all, but I have caught many.  Therefore, as I stated, the choice is truly an interesting one...Cooperate and put your faith in the process, or DELIBERATELY do something (that you read on some website) that will stand a pretty good chance of causing you to throw away your life's dream...

With all do respect, that brings to mind the old Clint Eastwood line, "Do you feel lucky?"

Regards,

Nonombre

nonombre,

Not exactly an accurate depiction of what actually took place in my pre-employment polygraph proceedings. I would like to clarify the facts, so other potential readers of this message board don't confuse what actually happened, versus what clearly did not happen.

First, I was not "caught" nor "suspected" of employing polygraph countermeasures during the process of my polygraph exam by the examiner. At all!

Secondly, the FBI DC polygraph lab did not catch any polygraph countermeasures per se. What they did catch, was a "text book" example [trend] of a set of perfect polygraph charts... i.e., strong responses to the "test has now began," strong responses to all control questions on all three sets of charts, strong response to being told, "the test is now over," etc.

I was told by the polygraph examiner that the DC lab personnel told him verbatim, that "I did too good to be true," and also that "No one does that good." The polygraph examiner already knew that he was going to be questioned at great length by my SSA, ASAC, SAC and Unit Chief as to what exactly took place.   

That said, I clearly did not get "caught" employing cm's by the examiner or by the DC lab. I was "suspected" of employing polygraph cm's by the DC lab based strictly on the results of my charts. That meaning, that I properly identified all control questions, and produced a strong response to each control question consistently throughout all three sets of charts. Hindsight, I should have intentionally missed a couple of control questions, and not responded to them at all.

I should have only responded to the first two sets of charts with respect to the control questions, and not produced any responses at all to any of the control questions on the third set of charts. 
In addition, my breathing rate/pattern matched exactly what was expected throughout the polygraph exam. 

That said, there are specific details that I cannot share on this message board, as much as I would like to do so. Although, I can say that it was later explained to my SSA/ASAC and SAC why the DC lab had trouble with my charts. It was explained that even currently employed SA's and support personnel miss a couple to a few control questions during a polygraph exam. Meaning, I should have at least missed properly identifying some of the control questions, to include responding to the "test has now began" and "the test is now finished". I responded to ALL control questions from beginning to end.

The DC lab personnel did not "catch" me doing anything wrong. What their computer program did "catch," was a perfect set of polygraph charts that raised questions and suspicions.

I would be less than honest if I said I was not personally disappointed and devastated by the results. I thoroughly enjoyed working with the FBI. However, I’m still doing what I enjoy most, and also what lead to my initial contact and association with the bureau as well as other Federal intelligence agencies. 

Simply telling the truth during a polygraph exam is no guarantee of a favorable result, or of passing the exam. There is no doubt that many truthful people with absolutely nothing to hide have failed polygraph exams for no apparent reason.

I personally believe that anyone facing a potential polygraph exam would be wise to educate themselves on the process and theory of polygraph testing. That way, they could make an informed decision on whether or not to follow through with the polygraph exam; or, take the appropriate actions to reduce the risk of a false positive [DI] result.

v/r
triple x 


 

Quote:


Triple x,

You bring out a very significant point in your last posting.  Whether or not the original examiner suspected CM's (perhaps he did not), this is one reason why all federal agencies quality control every field polygraph examination.  From what you state, the original examiner aparently did not "catch" the CM's you employed.  Quality control quite obviously did.

The system worked.  You were caught.

I see this as food for thought for the next guy who is contemplating rolling the countermeasures dice...

Regards,

Nonombre 8)

alterego1,

I don’t know if all State law enforcement agencies send their polygraph charts off for higher-level QA checking or not… but I do know that the FBI surely does. I would assume [strictly my opinion] that most of the Federal agencies probably do send their charts to the FBI DC polygraph lab, or possibly to the CIA’s polygraph lab. 

The polygraph examiner told me that I passed my pre-employment poly exam, only to learn a week and a half later that the FBI DC polygraph lab suspected cm’s. Thus, I was retested two weeks following the first “successful” polygraph exam, only to come back on the retest as deception indicated for either taking drugs to help pass the polygraph; or, employing cm’s. Keep in mind that the bureau could not nor would not tell me what if anything they suspected me of doing. The only thing the polygraph examiner said to me for sure was… according to the DC lab QA review, “You did too good to be true” and also said, “No one does that good.” 

Although, I do want to point out that the examiner never suspected a thing when he administered my first exam. Some on this board has argued in previous messages posted elsewhere on this message board that “perhaps the polygraph examiner did suspect cm’s during my first polygraph exam…” 

My response back to that was something along the lines of: if the examiner had in fact suspected that I was employing countermeasures, I don’t think he would have been so nice throughout the entire testing process to include following the exam. I was never asked any questions following the polygraph, there was certainly no post test interview or interrogation where the examiner asked me if I was having trouble with any of the questions, etc.

Plain and simple, the polygraph examiner never suspected a thing, and was very deliberate in telling me how well I had done, to include telling the application administrator and his/her assistant[s] that I did perfectly fine on the polygraph. Last but certainly not least, the polygraph examiner gave the application coordinator to include his/her staff, and me the “thumb’s up” as he delivered me back to the lobby. 

There is no question or doubt in my mind whatsoever, that the polygraph examiner was very proud of the fact that I had done well on the polygraph exam. Especially after my SSA and Unit Chief had called him to discuss my pending polygraph.

The DC lab unit saw something in the charts that made them suspect that I had employed cm’s. I’m certainly not implying that the DC lab has a system that can detect countermeasures… however, I do believe that they do have a computer program that looks for certain “trends” in responses throughout the charts.

There is a section in “TLBTLD” that addresses this very issue. In my case, I think I produced a set of “textbook” charts to the questions that I was asked. In hindsight, I should not have produced perhaps as strong of a response to the exact same questions that followed in the 2nd and 3rd sets of charts. In “TLBTLD” it discusses not responding too strong to the same question in each subsequent set of charts. Actually, in the 3rd set of charts, I think “TLBTLD” advises the examinee not to even produce a response to the same control question in the 3rd set of charts. You should simply answer the question asked, and do not produce any cm response to the question.


Triple x

I have often read where federal agencies such as the FBI will send off the polygraph charts for quality assurance purposes after the examination is complete.  The individual administering the polygraph will give his opinion of pass or fail, but ultimately the quality assurance guys have to look at the charts to give it the final approval.

My question is, are all polygraphers having the DODPI "seal of approval" (whether working for Federal or local agencies) required to submit their charts to a "higher authority" for QA purposes, or is this simply an extra step that is at the agency's discretion?