Wow, Ray. That was some pretty lame.

MagicSteve is right on.

And you know it.

Ray, please...

For the love of God, tell us, the great unwashed.

How accurate -- in terms of a percentage -- is a PCSOT sexual history polygraph "test"?

The American Polygraph Association on test accuracy, 1997-2012:

Polygraph testing is 98.6% accurate. 

The American Polygraph Association on polygraph accuracy, 2012-2015:

Polygraph testing is, on average, 89% accurate for specific-issue exams and 85% accurate for multiple-issue exams. 

APA chairman Raymond Nelson on polygraph accuracy, 2016:

"The test is probably more accurate than some people wish, and also probably less accurate than some people wish." 

Dr. Richardson

I believe you may have confused a low poster probability with test sensitivity. 

It is an easy mistake and I assume it is accidental. As I'm sure you known, test sensitivity is a descriptive proportion that does not change whether the frequency count is small or large. 

What you seem to be referring to is the posterior probability when screening for an issue for which the frequency count is small relative to the population. 

As you know, some interesting things can happen to the posterior under a strong prior - particularly the proportion of true positives and false positives. 

But the actual test sensitivity level - the proportion of actual positive cases that are classified as positive - is not affected by the prior or frequency. 

RN

Drew Richardson wrote on Jan 14^th, 2016 at 5:43pm:


What you appear to be saying here is that the scoring of the chart is influenced by the examiner's impression of the test subject... am I getting the wrong impression? And, if that is the case, then what does that say about the validity of polygraph examinations? 

Also, how does that work for sexual history polygraphs, when there is no known wrong-doing?

Dr. Richardson,

You and I, and a number of polygraph examiners, share the same concern about the importance of objective analytic models in credibility assessment, and the potential hazards associated with subjective analytic methods. 

Just so that I’m not feeling that this is some simple game of “punk-the-chump,” - as if this is a real discussion ...

we should remember that the lack of an “exact” or “precise” statistical estimate for an input parameter (test sensitivity) does not prevent the use of Bayesian analysis. The point for Bayesians seems to be to define what we have to start with, and then use the test or experimental data and Bayes' theorem to improve on that.

Bayesians could simply treat the test sensitivity parameter as a random variable. Bayesians often quantify unknown parameters using anything from blunt or thin available information to subjective belief (a hunch or suggestion) to markov chains or stochastic simulation or monte carlo methods, and other solutions to estimate the unknown parameter. 

Or we could simply use a known input parameter.

If one starts with a single hypothesis or theoretical assumption that some recordable changes in physiological activity are loaded onto different types of test stimuli as a function of deception or truth-telling (or concealed information) relative to the investigation target stimuli (NAS/NRC, 2003 wrote “ … discriminates deception and truth at rates significantly greater than chance…”).

Sure, anyone can choose to reject the data and evidence, but that does not change the data and evidence that support a conclusion that the recorded physiological data can and does discriminate deception and truth-telling at rates significantly greater than chance.

The subject/hunch argument is entertaining for some, but if mere subjective intuition were driving the polygraph test then we would need to somehow explain the effectiveness of contemporary statistical algorithms. It’s more likely that there is something important in the recorded physiological data.

The null hypothesis to all of this is that all physiology is random and not correlated with deception or truth-telling (or concealed information) in any way.

The theory that some recordable differences in physiology are loaded onto different test stimuli as a function of deception is not completely lost on other. Your own research on Brain Fingerprinting and the Concealed Information Test is basically premised on the same overarching idea that some noisy but real changes in physiology are loaded in response to different types of test stimuli (key or target stimuli and the other stimuli) . Other groups of scientists have also begun to study the possibility of using data from different test configurations and different physiological recording sensors with statistical or machine algorithms designed to indentify subtle differences for credibility assessment tasks. 
 
Having a working theory and some some sensors that have been shown to discriminate deception and truth in some noisy way, we still don’t know the test sensitivity do we? 

Remember that Bayesian analysis can also be used to study unknown parameters such as priors and other things. 

Or maybe we merely have to calculate a multinomial distribution of the combined scores from an array of sensors under the null-hypothesis that all data are random with how many presentations of the investigation target stimuli. And we could calculate the density of any test score against the random multinomial distribution. Then we could proceed to determine statistically whether the recorded data do or do not appear to be loaded systematically or randomly.   

But from a Bayesian perspective, we could use the multinomial density as the input parameter for our Bayesian analysis. 

But we still need to be clear about the fact that we are talking about a posterior probability (still conditional on the prior as an input parameter) in the end.

Of course the prior is often another  known parameter, but that does not seem to stop Bayesians. Bayesians would use available information to quantify the unknown parameter, or simply treat it as a random variable. We could also use Bayes' theorem to improve refine our knowledge about the unknown prior. 

Have a nice day.

RN

Can someone explain how you get a reading of deceptiveness out of a chart, when the only thing that is charted is basic physiological responses? Out of all these discussions, no one has come even close to addressing the fact that one physiological response is no different from another. I think we all agree, 100 percent, that the exams measure physiological responses to questions.

If there is no difference between one chartable response and another, then how do you know if someone is being deceptive? 

You don't. Remember the whole 'you can't measure something that doesn't exist' thing I went on and on about? This is what I mean. Scientifically, the difference between the responses, as recorded by a polygraph machine (or any machine known to man), is not detectable. You can't find someone to be 'deceptive' when you have absolutely zero clue what 'deceptive' looks like scientifically. No one is addressing the fact that the reason that polygraph 'results' are essentially null and void, and the 'test' holds so little validity both scientifically and legally, is because no one can tell the difference between what a guilty, or deceptive, or frightened, or excited, or angry, or aroused (you get the gist) response looks like.

Literally, this is the core of why polygraph examinations are so ill regarded. The examinations simply do not do what they are purported to do. Plain and simple. 

I have to admit, though... they are darned good at taking blood pressure readings, measuring breathing, heart rate, galvanic skin conductivity readings, and the like... darned good.   

Dr. Richardson,

Thank you for the reply and discussion.

What your concerns and experience with manual scoring of polygraph data actually seem to speak to is this: a)  the importance of more standardized, perhaps even more automated, test administration procedures that can remain objective and robust against the influence of human intuition/fatigue/bias/etc. during test administration, and b) the importance of structured, perhaps even automatic, algorithmic analytic models that are similarly more objective robust against the influence of human intuition/fatigue/bias/etc., leading to greater reproducibility of analytic results and a greater ability to incorporate new emerging knowledge and information to possibly improve the precision of an analytic result. 

Without clearly structured algorithmic methods the analytic process will continually subject to human factors and also to the need to train and develop those difficult-to-quantify "clinical" or expert judgement skills with every new class of professionals (which does not fully address the perishable skill problem). 

These are issues that have been in consideration in testing and diagnostic work since at least Meehl (1954) and Nunnally (1967), and are certainly worth discussion. 

The alternative what Meehl referred to as "mechanized" (antiquated term, but it makes the point) is to rely on less structured forms of professional or clinical judgement, for which the trend in the evidence has been somewhat clear for a long time.

Despite the clarity of evidence, learning to rely on structured solutions can introduce existential conflicts to minds of some professionals. Also, there are is a need for ethical discussion around what some have called the "sky-net problem" when allowing mechanistic (machine learning) processes to make decisions about humans. The ethical discussion here involves both what we should expect of structured machine processes, and also how to implement them in a manner for which human decisions remain human processes. 

RN

I will also be away from home for the next week.

Safe travels