posted 07-26-2009 06:56 PM            

Many accounts of the two most notorius U.S. traitors of modern times; (FBI's Robert Hanssen & CIA's Aldrich Ames), make light of the accuracy of the polygraph examinations used as a deterrent to espionage.

I wanted to examine this issue from various perspectives in order to glean the closest version of the truth.

I selected one source "The Main Enemy" by Milt Bearden a thirty year vet with the CIA, and one involved with the
Ames affair. "Presidio Press" Toronto Canada (2003).

The other source on the opposite end of the spectrum is "Spy Handler" by Victor Cherkashin, a forty year vet with the KGB, and the handler of both Ames and Hanssen.
"Basic Books" Cambridge MA., (2003)

Bearden of the CIA, gives an account that really did not denegrate the accuracy of the polygraph, but exposed the weaknesses in policy making.

On the one hand, one agency traitor named Ed Howard was hired even though during a polygraph examination he admitted to continued drug use. Years later before being posted to Moscow, during a polygraph examination he again admitted to drug use and being a thief, and cheating during a highly sensitive training operation. He was fired, he then sold his secrets to the KGB.

Bearden speaks of a high ranking KGB officer that then recounted a hard and fast rule in that if an American traitor had been polygraphed by the FBI or CIA that he was never to be trusted again since they were probably broken and turned into a double agent. But in 1986 the KGB heard of a change in the American's polygraph policy that had been relaxed.

In 1986, the U.S. was losing informants, and a mole hunt continued for years, but the person leading it "did not want polygraph examinations" of the people connected to the lost sources of information. (go figure)

Perhaps the most disappointing part was Bearden's account of Ames' polygraph in 1991. Ames felt he could pass the test if he could establish a relaxed rapport with the examiner???
Ames showed DI when questioned about money, yet clever misdirection got him passed the examiner (The charts tell the tale)!!!!

Cherkashin-KGB tells a little different story.

Ames was a college drop-out, with three drunk driving offenses. Even then he was hired and allowed to handle Russian traitors.
Ames was described as drinking too much, and his colleagues complained about his shabby clothes and lax personal hygiene. Ames had financial problems, with debts of $40,000, and his wife was suing him for divorce.

Ames' fear of polygraph examinations was diminished by KGB training since the U.S. technology was similar, but the methods were different.
In the Soviet Union, only the examiner was present in the room, while in the U.S., several people were present possibly making it more difficult by increasing a subject's psychological pressure.
The KGB lab taught yoga, and drugs to control physical reactions. Also answering indirectly and dancing around questions.

One interesting method was when Ames was scheduled for a polygraph examination, the KGB arranged for legitimate meetings and actions so that when questioned if he had met with KGB agents or other actions he could answer truthfully. In 1986 prior to being posted to Rome that was the method used and he was given a pass NDI.

I know that we are all devising question formats that would prevent these dodges if we encountered them today, we've come a long way baby. "Nail the Pretest".
So Much for Ames.

According to Cherkashin-KGB, Robert Hanssen provided Moscow with information valued at tens of billions of dollars. He supplied the KGB with thousands of documents, many on twenty-seven computer disks containing information downloaded from FBI servers.

Cherkashin calls Ames America's "deadliest spy" because he unmasked the CIA's human intelligence network, leading to many deaths.

But Hanssen was even more important to the KGB because his disclosures went to the heart of Washington's intelligence infrastructure. He gave the KGB some of the most sensitive guarded secrets of the CIA, NSC, NSA, and the FBI.

Hanssen was described as a "funnel". As the manager of the Soviet program, every bit of information came into him, and went out from him. He was senior enough to have wide access, and because he was a deputy, junior enough to read it all.

Now here is the kicker. According to Cherkashin-KGB, in all the years that Hanssen spied, HE NEVER HAD TO TAKE A POLYGRAPH TEST.

GM's method of holding up American traitors as trophies, and proof that polygraph examinations are worthless simply causes people to closely examine the institutions these people worked for.
I see sloppy policies, minimal standards of conduct, and a lack of integrity compounded by bureaucracy.

I see GM as a loser, that needs a reality check.

[This message has been edited by BrunswickT (edited 07-26-2009).]

[This message has been edited by BrunswickT (edited 07-26-2009).]

posted 07-26-2009 10:37 PM            

Great summary.

Thanks.

quote:

---

One interesting method was when Ames was scheduled for a polygraph examination, the KGB arranged for legitimate meetings and actions so that when questioned if he had met with KGB agents or other actions he could answer truthfully. In 1986 prior to being posted to Rome that was the method used and he was given a pass NDI.

I know that we are all devising question formats that would prevent these dodges if we encountered them today, we've come a long way baby. "Nail the Pretest".
So Much for Ames.

---

Bravado is fun, but if we really want to challenge ourselves to make improvements it will be necessary to not just pat ourselves on the back but instead try to kick apart our own fond ideas.

For example: by what procedural improvements, made since 1986, have we mitigated all likelihood that planned engagement in acts similar to the examination targets might change either the conversation, test questions, or test results?

Pretest... (reviewing the test questions)

Examiner: Did you do it? Because if you did it, then you'll react to this stimulus, and if you didn't then you shouldn't react to this question. So, are you going to react to this stimulus question during the test. In other words, did you do it?

Examinee: Yes.

Examiner: OK. Tell me about that?

Examinee: Well there was this time and that other time?

Examiner: OK. Thanks for letting me know. I'll change the question. Here's the question: besides the fact that you did it those times, are you going to react to this stimulus question. In other words: Besides the fact that you did it, did you do it?

Now sure, we use slightly different exact language. For example, we would probably say: "Besides what you told me, did you do it?" But logically and semantically its the same.

Then, (posttest)...

Examiner: Tell me why you are reacting to this question about doing it.

Examinee: Well, I told you that I did it that time and the other time...

We know a little bit about polygraph accuracy in event-specific investigation circumstances in which the examinee completely denies any involvement in the issue. We know a little less about accuracy in screening situations. And we know still less about accuracy when attempting to test the limits of an examinee's admitted behavior.

As far as I can tell we need more research on this.

In the mean time, it could be that the solution to these complications does exist in both procedures and in administrative policies.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 07-30-2009 02:26 PM            

Think of what we do as using a torque wrench to tighten a lug nut. If we set the torque too high, we tighten the lug until we wring the head off. Not a good result, especially if you’re the nut being tightened or you are the one responsible for keeping the wheels turning safely. If we set the torque too low, we never break a bolt, we do get the warm and comforting sound of the click when we tighten the nut but in reality, one day the wheel is going to run off. Again, not a good result.

Our dilemma appears to be easily solved by just finding the right amount of torque to get the job done without breaking the bolt. In reality, though, every bolt has a different tolerance or breaking point and every job requires a different amount of torque. We don’t have the luxury of knowing either of the unknowns going in.

Added to our problem is when the policy wonks come into play. Human Resources wonks say we’re breaking too many lug nuts so we lower the torque a bit. Security wonks say we’re letting too many wheels fall off so tighten the torque. It’s kind of hard to blame our problem on either the torque wrench or the lug nut. More policy types then add another problem by bringing in weaker and weaker lug nuts and demanding they be used to hold the wheel on. We’re told to use them anyway and don’t break any off in the process. Then we start manipulating the torque settings to accomplish that, knowing when a wheel runs off, we’re the ones who will get blamed

posted 07-30-2009 10:56 PM         

I am awestruck. That is a truly brilliant metaphor!

posted 08-05-2009 09:33 PM         

I enjoyed the paper but it takes a hell of a shot at two out of three of the LEPET RQs. At the last AAPP seminar, Harwell and company were selling the LEPET as the best thing going. Thoughts?

posted 08-06-2009 09:42 PM         

I always appreciate new ways of seeing truths.

posted 08-11-2009 09:55 AM         

Any response to my question above?

posted 08-11-2009 01:41 PM            

It would be good to have a discussion about this.

First, LEPET RQs (from the DoDPI LEPET handbook)

Phase I - Counterintelligence

#4 Do you know anyone who is engaged in espionage or sabotage against the United States.

#6 Have you intentionally mishandled any classified information?

#8 Have you had any unauthorized foreign contacts?

#10 Have you been involved in terrorist or subversive activity?

This last question is defined, in part, as anything that would interfere with, undermine, or deny an individual of their rights guaranteed under the U.S. Constitution, and includes anything that could lead to the violent or illegal overthrow of the U.S. Government.

Phase II - Suitability

#24 Have you ever committed a serious crime?

Including arrests and undetected crimes.

#26 Are you intentionally withholding any information about your involvement with illegal drugs?

There could be discussion about the word "intentionally." We don't use this type of word at all in PCSOT tests in Colorado. Behavior is a CHOICE - unless someone is incompetent. To engage a conversation around behavior as unintentional is to endorse the notion that someone simply couldn't help themselves except to engage in some impulse or compulsion - which would mean that learning to make better choices (which is what therapy is all about) won't make someone safer. The joke in therapy is that if someone can't help it, then we can't help them much anyway. "Unintentionally" is therefore an excuse, but may help with posttest themes.

To be really useful a question should be capable of providing meaning (empirical meaning and epistemological meaning – not existential meaning) whether someone passes it or fails it. So, we have to ask – what does it mean when someone passes this question, and what does it mean when someone fails it?

#28 Are you intentionally falsifying or omitting any information on your application forms?

Keep in mind that basic human nature is to underreport problems and to attempt to make a favorable impression.

There is, of course, always more information – unless some really has minimal or nothing to report.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 08-11-2009 06:36 PM         

One very popular variant of polygraph police screening,
the Law Enforcement Pre-Employment Test (Research
Division Staff 2006), uses questions such as �gAre you
now intentionally withholding any information about your
involvement with illegal drugs?�h On the surface this may
seem like an ideal question but a closer look can point to
potential problems. This question requires examinees to
engage in the cognitive processing of their entire life�Œs
exposure and experience surrounding the concept of illegal
drugs. The phrase �gany information about your involvement�h
is very broad. Moreover, unless carefully defined the
term illegal drugs could be ambiguous (e.g. Does it include
a single case of taking someone else�Œs prescribed pain
medication for a migraine and/or does it include underage
use of cigarettes?) The expectation for truthful persons, that
every detail regarding an aspect of one's behavioral history
has been disclosed, may not be realistic. Because it may not
be realistic to know absolutely everything about someone's
behavioral history, this type of questioning may not provide
realistic and meaningfully interpretable results when an
examinee admits to some historic pattern of recreational
drug use.

AND:

One final mention of concern are questions surrounding
errors and/or omissions in application forms. We should
consider the wisdom and hazards of very broad questions
which are intended to test the veracity of the often
comprehensive forms and booklets that applicants are
required to complete. One obvious complication is the
known tendency for people to under-report on self-report
inventories. We suggest the polygraph is a less than ideal
way to verify the information provided in those booklets.
One broad sweeping question attempting to verify or refute
all of the information provided by an applicant would surely
seem to test the limits of both the subject�fs memory and the
capability of the polygraph test. The obvious exception is
those circumstances in which inaccurate or incompleteness
of information on the required disclosure forms itself
represents an enforceable behavioral transgression.

posted 08-13-2009 10:56 AM            

A really good polygraph target/question would provide interpretable and usable meaning and information whether someone passes or fails the question.

quote:

---

#26 Are you intentionally withholding any information about your involvement with illegal drugs?

#28 Are you intentionally falsifying or omitting any information on your application forms?

---

So, lets start with the merits of these two questions. What are there strengths, in terms of adding incremental validity to police (civilian law enforcement) hiring decisions. What does it mean to a personnel administrator or police psychologist/evaluator if someone passes the question? What does it mean if someone fails it? What does it mean of someone passes or fails with no admissions? What does it mean when someone passes or fails with minimal admissions? And, what does it mean if someone passes or fails with substantial admissions?

There are implications surrounding the work "intentionally," just as there are implications with the words "deliberately" or "purposefully," along with "recklessly" and "negligently," and "willfully." The examinee has advance knowledge of the polygraph test, and the requirement for honesty. Dishonesty is therefore ALWAYS intentional. To suggest otherwise is to endorse and excuse - as if the examinee couldn't help it.

In the end, we examiners will render a decision as to the presence or absence of any "significant responses." Keep in mind that the word "significant" in science means "statistically significant," meaning that it should be based in the statistical comparison of measured reactions to normative or ipsative decision models and decision thresholds(what we examiners simplistically call "cutscores," while most scientists call it "alpha.")

One nifty feature of these questions (#26 and #28) is the use of "Are you now intentionally withholding/falsifying any information..." Which is a piece of carefully constructed linguistic logic (CCLL) that permits the examiner to use the same question regardless of whether the examinee makes any admissions, no admissions, minimal admissions, or subtantial admissions.

On the obvious simple side of things, the CCLL of #26 and #28 seem to suggest we know "everything" there is to know about involvement these issues when someone passes these questions. This is not hard to conceptualize when someone has made no admissions - they were simply not involved. What, however, do the results mean when someone makes minimal admissions and passes? Does that still mean we know "everything." What if someone makes substantial admissions, and passes? Does that mean we know everything? That there is nothing more to know about? Is that possible?

Keeping in mind that a really good polygraph target/question would provide interpretable and useable meaning and information whether someone passes or fails the question, it might be possible that some questions are usable when the examinee makes no admissions or minimal admissions, but may not be as usable when someone makes substantial admissions? The CCLL of these questions (#26 and #28) seems to encourage the notion of attempting to continue to use and interpret these targets/questions regardless of the degree of admissions. To be sure, this is most likely going to occur with the drug question.

What usable information is provided to a police psychologist or personnel administrator by these questions? This is not just a rhetorical question. It is the issue at the core of this discussion, and it needs to be both articulated and scrutinized. The ensuing conversation should address both the merits and limitations of what we might achieve with these questions. Of course, if we're not capable of pondering both of these (merits and limitations), then we're at risk for assuming a very unscientific posture among our sister sciences of testing and measurement.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 08-13-2009 01:46 PM            

We use the drug question as described here. (No matter what you pick in a screening exam, the questions are tough.) We ask the question because we do (so we're told) want to know about the person's whole life (as if that's possible). I pre-test "now intentionally withholding" to mean, "Right now you're thinking of some event you know you should have disclosed but failed to disclose...." I tell them the question doesn't mean, "Could I have forgotten anything...?"

All our RQs are phrased in a similar fashion, and it doesn't seem to be problematic. That is, we're not seeing an unusual number of folks reacting to the question without disclosing more information (that they intentionally failed to disclose).

I take a long time in my pre-employment pre-test. The reason supports Ray's comments. We don't really care about the answers so much as we want to know if a person will lie when there is a chance of negative consequences - something we worry about with men on the street. Does such conduct predict what they hope? I don't know, and in fact I doubt it.

posted 08-15-2009 06:54 PM            

quote:

---

First, I agree with Ray; however, we're not yet where we need to be.

---

Agreed.

quote:

---

We use the drug question as described here. (No matter what you pick in a screening exam, the questions are tough.)

---

Questions are only “tough” for a couple of reasons:

• the target lacks an adequate operational definition (did she want you to do XYZ)
  

or

• the examinee is involved in the behavior (besides the fact that you did it, are you going to react to this stimulus about XYZ)
  

When we select a meaningful target (ie., provides usable/interpretable signal information whether the examinee passes or fails), with an adequate operational definition, it should not be all that difficult.

quote:

---

We ask the question because we do (so we're told) want to know about the person's whole life (as if that's possible).

---

That is my point: it is not possible to know everything, unless there is nothing or near nothing to disclose. When there is 0 to disclose, the stimulus/response equation is simple: “are you going to react to this stimulus” - a questions which describes the examinee's involvement in a behavioral issue that is supported by an operational definition that tells us what it would look like if he did it. The correct answer is “No,” indicating the examinee's complete denial of involvement and assertion that he expects not to react to the stimulus question which describes that target behavior.

The people who most likely won't have trouble with these questions are the squeaky clean people with nothing to disclose.

quote:

---

I pre-test "now intentionally withholding" to mean, "Right now you're thinking of some event you know you should have disclosed but failed to disclose...." I tell them the question doesn't mean, "Could I have forgotten anything...?"

---

By “pretest” you mean the verb “pretest,” if there is such a verb, not the noun “pretest.”

What you are really doing is attempting to provide an operational definition for your concern: “thinking of some event that you failed to disclose.” Hmm.

Question: If we are concerned about some event that is not disclosed, then why not simply ask “besides those X times, did you do it any other times?” or “besides what you reported, did you do it any other times?” Of course, “it” should describe the examinee's involvement in a behavioral concern that is supported by an adequate operational definition.

Answer: Because its not that simple. “Besides what you told me, did you do it” translates to the stimulus/response notion of “besides the fact that you did it those times, are you going to react to the stimulus.”

“Intentionally,” like “deliberately” is a whole nuther rabbit-hole. It implies that there are “unintentional” reasons for withholding – that maybe someone couldn't help it – that maybe lying isn't a choice – that maybe behavior isn't a choice.

Now, the 2002 DoDPI LEPET document at the anti site, states, on page 3, that it is on the Secret Service protocol. The Secret Service, you recall, is in the bidness of protecting the President, and they want only the best of the best and the squeakiest of the squeaky to stand next to the President with a loaded machine gun under one's jacket.

If I were the person in charge of that I would want to force the false-negative rate as low as humanly possible, even if it means accepting a high false-positive rate – which is what I think you are saying when you say these questions are “tough.” I imagine there are far more highly qualified over-achieving applicants than actual jobs, so it may be good service to the Secret Service's goals to screen out all but the very squeakiest and very best, and send the rest home to seek other employment elsewhere.

For the average Podunk USA civilian law enforcement agency, a polygraph test with a high FP rate may not be ideal, and may make personnel selection decisions more difficult than necessary when there is a lower ration of suitable-applicants to jobs.

So, to further beat up on the notion of “intentionally,” we're suggesting that someone who wants to work for the Secret Service, and stand next to the Man with a loaded machine gun under his jacket, could apply for work, complete all the required documentation, be informed of a polygraph appointment date, be advised of the requirement for complete truthfulness about ones' squeakiness or un-squeakiness, and then “uninintentionally” withhold or neglect to report some piece of information that was explicitly requested. Bullshit. “Intentionally” is an EXCUSE, than legitimizes the silly notion that there are unintentional lies.

Lying, by definition, is an “intent to deceive.”

Behavior is a choice, else nobody has to be responsible for nothin'.

Only children (typically age 5-7 and under), psychopaths, and mentally ill persons, believe in some legitimacy of the “I didn't mean it,” as a viable excuse for misbehavior.

To excuse people of responsibility for their behavior it to set the stage for all kinds of mental incompetency attributions for all kinds of problems.

quote:

---

All our RQs are phrased in a similar fashion, and it doesn't seem to be problematic. That is, we're not seeing an unusual number of folks reacting to the question without disclosing more information (that they intentionally failed to disclose).

---

That's not surprising. There is always (nearly always more whenever there is something) more information.

When there is >0 (more than nothing) to disclose, the stimulus/response equation gets kinda complicated: “besides the fact that you did it those X times, are there any other reasons why you might react to this stimulus” This might be more realistic when the admissions are minimal, when the examinee can remember every single incident. When the examinee admits involvement at a level for which he cannot recount every single incident, there will always be a human tendency to underreport/minimize/make-a-good-impression, and we should generally expect that there is always more information regarding the issue.

So, more admissions - about something that was already admitted - are interesting to us polygraph examiners, because they are self-gratifying (I.e masturbation). What, however, is the real added incremental validity of those additional posttest admissions, compared to the signal value of the information that was already gained during the pretest? Sure there is more. There generally always is. People underreport. They underreport on psychological measures – that's why we try to build validity indicies into our personality inventories like the MMPI. It's why we polygraph people. They tend to minimize.

Whether we get the info in the pretest or posttest doesn't change the interpretable signal value of the information. If we get admissions during the pretest, we know we can generally get more admissions during the posttest. So what. What have we really accomplished, in terms of increased incremental validity, by attempting to test the limits of a behavior that was admitted during the pretest.

Our goal is to know enough (a lot) to make well informed decisions. A secondary goal is to get that information while sticking to practices that are scientifically and ethically defensible, without engaging in fiction or mythology. When we allow ourselves to assume we know everything, we are choosing to be duped – there is, of course, always more (empirical argument). The only way there is not more information is if the examinee is exaggerating.

To opponents of the polygraph, exaggeration = making false admissions to placate an unreasonable method (ethical argument).

When we take this approach we allow ourselves to be boxed in by both the empirical argument that pretending to know everything is an empirical fallacy (I.e, science fiction), and the ethical argument that examinees have to make false admissions and exaggerate their problems to pass a test that is cannot work the way we want to pretend it does.

Whether we want to base the viability and credibility of the polygraph profession on the silly assumption that we can know everything or the indefensible position of prompting people to have to make false admissions to place our unrealistic insistence that we can somehow know everything – that's something we ought to be having more conversations about.

quote:

---

I take a long time in my pre-employment pre-test. The reason supports Ray's comments. We don't really care about the answers so much as we want to know if a person will lie when there is a chance of negative consequences - something we worry about with men on the street. Does such conduct predict what they hope? I don't know, and in fact I doubt it.

---

So, it's not really a polygraph test ----- it's some other kind of litmus test...???

Let think carefully about this argument.

quote:

---

Our psychologists appreciate polygraph because they are shooting blindly, and many times, some of the info we discover guides their interview. That is, we get "stuff" they normally wouldn't and that helps them.

---

Not argument on this. However, if this is all we want, then we are vulnerable to the bogus-pipeline accusations of our opponents and detractors.

The answer to bogus-pipeline accusations is to make sure that we base our assumptions, methods, and dialog on arguments and constructs that are empirically, logically, and ethically defensible.

In other words we have to actually believe in polygraph science and ourselves as scientists in order to advance ourselves through self-scrutiny. Unless we either 1) don't believe ourselves or the polygraph to be capable of advancement, or 2) believe think the polygraph and our present methods are completely adequate and not in need of advancement.

.02

r

Quotation for the day:
“I never give in to the temptation to be difficult just for the sake of being difficult. That would be too ridiculous.” ---- Jacques Derrida

posted 08-15-2009 09:54 PM         

Dr. Honts' paper, specifically attacks two out of three of the relevant questions in the LEPET. What is being done at DACA or anywhere else for that mattter, to address these issues? Why would Don K or any other federal instructor still be teaching the LEPET if they feel it is so severely flawed?

posted 08-16-2009 12:33 AM            

I don't think we are suggesting that the LEPET is severely flawed.

As always, there is fine print, that says 1) do not re-broadcast without express written permission from major league baseball, and 2) these statements are the views of the authors only, and do not necessarily represent the views of the food chain.

The LEPET may do a good job serving the needs of the federal agencies.

What I am suggesting is that the only way to know everything is when there is nothing.

I don't know that I'd hang it all on one person to fix all the shortcomings of a technique for all types of agencies.

One thing that can be done is to stimulate discussion regarding LEPET target selection and question formulation among smart field practitioners and others who have some influence on policies development activities.

The paper at Honts' website attempts to make some thoughtful recommendations on that, beginning with the idea that it would be wise to begin to select targets that have actuarial relevance to the outcomes of concern - successful police officer training outcomes (completion of training), and police officer performance in the field (absence of serious problems).

Regarding the drug question in particular, there is little actuarial evidence of the contribution of drug use history to either training or police performance outcomes. However, it is unclear whether that is because people with serious drug histories are effectively excluded from police training and police work.

We have suggested that the face validity of the notion that attempting or pretending to know "everything" about a person's drug use history is suspect. We have also suggested that police psychologist/evaluators and police personnel administrators might be best able to achieve their goals of increased incremental validity re drug use history by emphasizing the gathering of information that illustrates the difference between various levels of drug use (e.g., complete abstinence, experimental use, recreational use, instrumental/medicinal use, dependent/addicted use). Complete abstinence and experimental use may be easy to identify, because the answer is going to be either "none" or a very small (single digit) number of incidents for which the examinee will likely recall each and every time. Recreational (non-instrumental, non-medicinal, but still non-addicted)) drug use will be more difficult to completely delineate, because recreational drug use can still easily amount to too many incidents to recall and report every single one, coupled with the fact that people tend to underreport when they want to make a favorable impression (e.g. when they want a job).

We could go the route of attempting to verify that we know "everything," that the examinee is not "intentionally withholding any information about his involvement in..." The problems are: 1) it is impossible to know everything, 2) the probability that someone reports perfectly the exact volume of use is zero, and 3) people will overreport or exaggerate if they are externally motivated to do so - and that fuels the arguments of our detractors and opponents because of the false admission problem.

The good news is that medicinal/instrumental, and addicted/dependent use might be more easily inferred because it will not as easily be curtailed or stopped - it will more likely be ongoing into more recent periods of time. For example, a police applicant who has used marijuana within the last year is not very impressive. An applicant who stopped his recreational drug use only a year ago is also not impressive, because that give the impression that he stopped primarily because he decided to start applying for employment in law enforcement. A few more years of abstinence (three or five years, or whatever the hiring policies may be) make it a little easier for a police psychologist/evaluator to begin to formulate an opinion about the difference between recreational/non-medicinal use and patterns of use that represent a form of dysfunctional coping strategy that indicates someone may not be ideally suited to work in law enforcement.

In other words, we have recommended that attempting to learn the total volume of use (i.e., intentionally withholding any information about...) may not add much incremental validity, while attempting to verify non-involvment within a reasonable period of time can add incremental validity to conclusions about an applicant's judgment regarding illegal drugs. While obsessing over silly numbers games may not be productive, the entire history and pattern of use can still be excavated in some degree of searching detail during the pretest.

The application packet question is a little more complicated. It appears to violate the basic principle that an RQ describe the examinee's involvement in a behavioral concern. I'm guessing that those application forms for federal law enforcement positions are explicitly clear about the information they require, and that falsifying the application may itself be a punishable offense. Again, we can expect that people will tend to underreport unless they are externally motivated to exaggerate. Application forms probably ask about criminal history, drug history, employment history, and personal history. There is a lot of room for underreporting in all of those areas, unless a person has very little or nothing to report.

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 08-16-2009 12:51 AM            

I don't think anyone disputes the idea that we get more information during the pretest and the posttest.

What I am arguing is that the pursuit of more information for the mere sake of more information is not the smartest way to use the polygraph to add incremental validity to personnel screening and other risk management decisions. It would be wiser to focus our inquiries on the kinds of information that risk-prediction and risk-management researchers have identified as most indicative of risk.

What I am also arguing is that we don't give up on polygraph validity and polygraph science - that we don't simply retreat like empirical cowards to the illusion of comfort behind an argument that rests solely on the bogus-pipeline effect or inarticulate arguments about "utility."

We will have a more satisfying future by focusing an articulate discussion about incremental validity resulting from the informed use of information that is actuarially relevant to risk prediction and risk management decisions. We will also have a brighter future if we believe in our own science and don't shy away from the difficult questions about the possible meaning of our questions - that way we can actually ensure and increase the decision support capacity of the test result itself. If the results don't matter or can't be used, then all we have is a bogus pipeline.

I suspect the polygraph can be more accurate than many of us think, but that it may require we examine the types of things that can and cannot be tested effectively.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 08-16-2009 12:16 PM         

All good points.

Lets look at the LEPET drug question. I agree that it would be very difficult for most people to give a difinitive number for the frequency of Marijuana use in High School or College. Frankly, I(we)don't care if the number is 10 or 1000. I doubt that the actual number of uses would have much bearing on this persons ability to perform the job provided it was not determined to be excessive.(Uh Oh- ADA Alert!)

What I do care about, is the subjects willingness to be honest and take responsibility for past behaviors. That has everything to do with being able to perform the job. A subject may not remember how often he smoked pot but he sure as hell knows if he has or has not as well as when it was. He should also have a very good idea as to the last time he fired one up.

If the subject used pot while in High School and is willing to lie about it now, he is probably not the kind of guy I want riding shotgun with me on the graveyard shift.

To overlook this area or not ask about it in the pre-test(because it is too broad) would be irresponsible on our part.

My point here is that yes, we can use the polygraph to determine if a subject is witholding information about prior drug use. If we try to use the polygraph to determine definitive numbers, we have just used Skip's Torque Wrench to snap off the head of the bolt and defeated ourselves at what we initially set out to do.

I think the answer to all of this is based on how well you conduct yourself in the pre-test interview.

Ted

posted 08-16-2009 07:09 PM            

quote:

---

It would be wiser to focus our inquiries on the kinds of information that risk-prediction and risk-management researchers have identified as most indicative of risk.

---

posted 08-17-2009 03:56 PM            

If we don't accurately inform our consumers (referral sources, administrators, and policy makers) about the torque capabilities of the bolts we twist, or the torque range of the wrenches we use, then we might as well pee in our own pool (mixed metaphor), because they will have been taught, by us, to believe in some form of polygraph science that ain't real and will be vulnerable to accusations of pseudo science.

The way to turn the ship around is to stop steaming straight ahead and drop anchor long enough to initiate an important conversation about these complications. Then, we'll have to take the time to define an approach that meets the information needs of the referring agents and remains consistent with good science.

This is off point:

quote:

---

I agree that it would be very difficult for most people to give a difinitive number for the frequency of Marijuana use in High School or College. Frankly, I(we)don't care if the number is 10 or 1000. I doubt that the actual number of uses would have much bearing on this persons ability to perform the job provided it was not determined to be excessive.

---

Define excessive.

What really matters is that the evaluator can use the information to make informed decisions about the persons' stability, judgment, resiliency, coping abilities, etc.

quote:

---

What I do care about, is the subjects willingness to be honest and take responsibility for past behaviors. That has everything to do with being able to perform the job. A subject may not remember how often he smoked pot but he sure as hell knows if he has or has not as well as when it was. He should also have a very good idea as to the last time he fired one up.

---

No argument that most people will know more or less exactly when they last fired one up.

Of course, for police officers who work undercover, the ability to lie convincingly may also be strongly correlated with job performance.

quote:

---

If the subject used pot while in High School and is willing to lie about it now, he is probably not the kind of guy I want riding shotgun with me on the graveyard shift.

---

But without evidence (empirical evidence from research) that recreational marijuana use as a juvenile, or lying about it as an adult, is correlated with police training problems or police performance problems, we are still engaged in a values-based (non-evidence) based approach, and you are still describing a litmus test (Lying about something you believe to be important) not a polygraph test of a behavior (drug use history) that is correlated with police training and performance outcomes.

OK, lets pretend we have two potential partners for you on the graveyard patrol shift, with all else being assumed equal. One used marijuana no more than 10 times as a juvenile, the other was a recreational marijuana user during his high school and college years – which could conceivably amount to approximately 1000 incidents of marijuana use. Recreational use, by definition, means that the pattern of use was non-medicinal/non-instrumental, and does not appear to be a coping strategy. The first guy (10x) minimizes and says it's only a couple times, the second (~1000x) tells the truth. Remember: all else is assumed equal for this discussion – so lets assume it's been more than five years since either person last used any marijuana. Which do you want riding shotgun with you? What if all else is not equal. What if the first guy (10x minimizer) was a talented athlete and A student with good disciplined study habits during high school and college, while the second guy (~200x) coasted on his abilities without great effort, didn't compete in sports and got above average grades based mostly on ability alone. Which one do you want for your partner now? Is it really the lying that makes the difference?

I would argue that it is not. 10X looks to me like experimental use. 1000X looks more like recreational use, but could also represent a coping behavior (instrumental use). In the above argument we assumed these things to be equal.

If we are looking for a litmus test, regarding one's tendency to lie about important things, why not inquire about marital infidelity? Surely, if someone would betray the trust of the one person they are closest to in the entire world, they might be at increased risk for betraying or letting down a duty partner during a very stressful patrol incident? Right? Of course, we'll have to separate cheating on non-marital partners – because that doesn't matter as much does it.

For applicants with previous law enforcement experience, we could probably ask about lying under oath. Surely any police officer that has ever lied under oath should be considered unsuitable for any future work in the law enforcement arena. Now, what about someone who had lied under oath, but tells the truth about it at the polygraph, when it matters most to the new opportunity? Would telling the truth about one's history of lying under oath rectify one's candidacy for future employment in law enforcement?

My point is that if we want the polygraph to be taken seriously as a science, then we have to be more willing to evaluate our tendency to engage in value-based pontifications. If you want the polygraph to be regarded as anything more than a ruse, then we have to do things that actually matter, describe why it matters, and use evidence to support the argument.

The article that Ted is referring to was submitted to a police psychologists journal.

----> end of deconstructionist rant

.02

r

posted 08-17-2009 05:33 PM            

quote:

---

If we don't accurately inform our consumers (referral sources, administrators, and policy makers) about the torque capabilities of the bolts we twist, or the torque range of the wrenches we use, then we might as well pee in our own pool (mixed metaphor), because they will have been taught, by us, to believe in some form of polygraph science that ain't real and will be vulnerable to accusations of pseudo science.

---

Ray, you're assuming our consumers (in the local government) see polygraph as more than an interrogation prop. I've solved major cases for investigators and they still think it's hocus pocus and voodoo. They don't care about those issues. They just want admissions. I've had chiefs as an audience. I've asked if they'd read anything, and the answer is always the same: Only admissions are meaningful, so use the Xerox machine if it'll get them what they think they want and need. They want those of us who are low on the HR totem pole not to meddle in HR issues.

Think about it: we can't get people to follow the ADA hiring rules. Do you think they care about question formulation? No, most don't. That's an unfortunate reality.

posted 08-17-2009 07:01 PM            

Ray so well stated that the drug issue, as it is used at present is sort of an intuition dowsing indicator----and I've ranted for yrs about such, only without the sophistication and due diligence Ray has put forth---but it seems to me that it is a lead sled for a test and that in my opinion values-based test questions are indeed a (just one) achilles heel of applicant screening. No one wants to place their lives in the hands of a drug addict---can we agree on this ? Can we agree that substance abuse in general is the single greatest social rot, next to dancing contests on reality tv? Can we also agree, without great defensiveness, that among law enforcement professionals, chronic and destructive alcohol abuse is as pervassive as the divorce rate amongst lawman and women? Cops, precops, and intel personnel getting smashed---isn't that what the drug question is really after? With GREAT respect to the fine lawpersons on this board, I must admit great eye-rolling on my part when there is talk about expected "honesty" from whom you are patrolling alongside, and talk of "integrity behind the badge." It' like a mothers' rally where the speakers are expousing the need for mothers to be nice to their kids----it's a given. I just don't see the integrity part as being the end result, nor do I appreciate the "you're not a cop and you wouldn't understand the trust"---which is well, all a little cryptic and even dodgy. Ironically, the main stated point from lawperson examiners is that trustworthiness is the flagship of the test---the overall theme. Yet a classic (and very savvy)control question is "do you consider yourself a trustworthy person?" So, the drug issue must be more about whether applicants need to get hammered when they have stress than it is about truthfulness and integrity. I dunno.If I were in an even higher cynical mood, I'd even go so far as to say that the drug issue is for the polygraph examiner, "a wage earner moment"---as it gets juicey info. "You stole your grandmother's oxy's when you threw your back out?" wow. Wait'll they hear this one up at HR. And for the HR and the shrinks, it may be something for them to brag about--- their ability to source/staff a good polygraph unit. It's like Ray's old story about the monkeys getting squirted with ice cold water (I'll let him retell that behavioral gem.)

The drug question has become so much of a liability (NOT the admissions mind you) that one can theorize that it would make a better control question, as even among folks with little to no drug use, the signal value is always there. Ooops, we just stepped on a slick slope. GM, the poem writing, weight lifting, star of his own variety hour "The Frank Burns Show," has an occasionally really good barb. He sees polygraph as a whole as a values-based testing modality, where we compare apples to apples in a CQ test. While he has always missed the great---dare I say wonderous---aspect of the field, his glass half-empty approach to polygraph, reveals a glass---by his appraisal--- that is not only half-empty, but has slimy, pizza-sauce-finger-prints with floaters around the rim.

I've said it for years----the best non-polygraph thing that could happen to the polygraph field is an affordible drug AND alcohol test that can test substance types and volumes over years and years of use and/or abuse.

posted 08-18-2009 06:08 PM            

Better drug testing would free up a target for more focus on the above---imo.

posted 08-18-2009 07:23 PM            

quote:

---

Ray, you're assuming our consumers (in the local government) see polygraph as more than an interrogation prop.

---

I'm assuming that if we don't see the polygraph as anything more than an interrogation prop, then our consumers won't either.

I'm also assuming that if we don't believe in the decision support value of accurate and reliable results from a scientific polygraph test, and if we don't believe in ourselves as good scientists, then our consumers won't either. We cannot expect to be taken seriously as scientists without learning a little bit more about science. Similarly, we can't expect to make any improvements in the polygraph without discarding old arcane ideas and incorporating new and better ones. To neglect this and simply go with the statust quo - because it's always been done... or some other tired rationale - is to gamble the future of our profession entirely on the interrogation aspects of the polygraph.

So, we can let the uninformed steer the ship according to their whims, and we can limit our own intelligence by neglecting to thing past our own hyperbole, then we can try quell the energy of our critics and opponents by baffling them with BS - and we risk running the thing into the ground the way we did in the 1980s.

Have you ever wondered whether polygraph examiners actually believed in what they were doing back then? I'm guessing they did. Why? Because they were probably decent people who couldn't have done it any other way. That doesn't mean they were right, or that there methods were good, only that they probably believed in it - for whatever it was worth. The real concern is this: how did they manage to believe in something that looks, in retrostpect, so silly? Answer: they didn't think critically about it, and simply endorsed our hyperbole as valid and accurate. We may be less at risk for repeating this mistake, but we'd be negligent to get complacent. People are not entirely satisfied with the polygraph. Neither are we. Our main choice is whether or not to become complacent about it, or to continue to pursue advances and improvements in our knowledge, instrumentation, and methods.

The long term answer is maintain an active dialog about a good scientific approach to the polygraph, continue to improve our standards of practice and professionalism, and show our critics and the community that the polygraph is just another version of good scientific testing. But this will only work if we actually believe in the polygraph as good science, and ourselves as good scientists.

.02

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 08-18-2009 07:27 PM            

quote:

---

On an aside (recent reflection and summary)------cops/precops/intel who compulsively steal, lie pathologically, have fits of rage, sexual addictions, hate women and/or minorities----are the greatest risk to the public's trust.

---

Interesting how those look a lot like some of the targets we attempted to recommend in the police preemployment paper.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)