Page Index Toggle Pages: 1 ... 11 12 [13] 14 15 ... 25 ReplyAdd Poll Send TopicPrint
Very Hot Topic (More than 25 Replies) Texas sex offender & mandatory polygraph (Read 314023 times)
Paste Member Name in Quick Reply Box Dan Mangan
God Member
*****
Offline



Posts: 652
Joined: Jul 31st, 2014
Re: Texas sex offender & mandatory polygraph
Reply #180 - Jan 14th, 2016 at 12:41am
Mark & QuoteQuote Print Post  
Wow, Ray. That was some pretty lame.

MagicSteve is right on.

And you know it.
  
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Raymond Nelson
Guest


Re: Texas sex offender & mandatory polygraph
Reply #181 - Jan 14th, 2016 at 1:16am
Mark & QuoteQuote Print Post  
Dan,

I'm quite sure you enjoyed MagicSteve.

As to your question about a layman's version of the accuracy of the test, how's this:


(to placate your science-phobia and math-phobia)

the test is probably more accurate that some people wish, and also probably less accurate than some people wish, and, of course, there is no such thing as a perfect test of any kind. 

-----

But if we are going to claim to be experts we have to eventually get beyond the simplistic and reductionistic hyperbole and quantify our knowledge about test accuracy. Then we will have to compare those quantified estimates to observed data and evidence and continue to refine them so that test accuracy estimates can hopefully increasingly converge with what we observe in the real world. 

Neglecting to do this we will probably see what we've seen in the past... a continued impulse to expect or pretend deterministic perfection (or near perfection) where this is not realistic - for which your published study claiming ~100% accuracy is a good example.

Alongside that we can expect continued aggravation around the inability to achieve deterministic perfection (or physical measurement - subject only to measurement error), and continued avoidance of real discussion about test accuracy and the probabilistic meaning of test results.

We can also expect continued straw man criticisms around the inability to actually measure some single physiological index for deception (even though absolutely nobody today seems to be claiming to do that).

In the absence of probabilistic thinking we will see people attempt to make a living be selling confidence and bravado. 

We will also see continued frustration and aggravation from others who notice the posture of unrealistic overconfidence, over-reliance on bravado, and lack of credible explanations. For example: how many people still believe that the effectiveness of the polygraph requires that the examinee is naive about the test, or that the effectiveness of the test requires misinformation (which only seems to create a sense of permission to make up more nonsense)? How many people still believe that the effectiveness of the polygraph requires that people believe it to be infallible? (does anybody today actually believe the polygraph to be infallible?)

And, because perfection is not going to be observed in reality people will be both avoidant and embarassed by the test result itself. People will also intuitively begin to make decisions and compromises to optimize the kinds of outcomes that are most useful in a practical sense (while avoiding the test result) and also minimize the cost function (there is a Bayesian concept for you) associated with the types of errors that are considered the least tolerable. 

But in the absence of realistic attempts to quantify our knowledge about test accuracy all of this gets driven by personality instead of data, and professional opinions will continue to  be overly subjective and un-replicable - as if professionals simply offer the opinion or conclusion that they are paid to offer. 

Some day hopefully somebody can have a real discussion about these very important and very interesting issues. 

But its not going to happen in the context of a lot of dramatic hyperbole and aggression because that kind of contempt will seem to be a barrier to any development or exchange of knowledge. 

Peace,

rn
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Dan Mangan
God Member
*****
Offline



Posts: 652
Joined: Jul 31st, 2014
Re: Texas sex offender & mandatory polygraph
Reply #182 - Jan 14th, 2016 at 2:09am
Mark & QuoteQuote Print Post  
Ray, please...

For the love of God, tell us, the great unwashed.

How accurate -- in terms of a percentage -- is a PCSOT sexual history polygraph "test"?

  
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Ex Member
God Member
*****
Offline



Posts: 710
Joined: Dec 9th, 2012
Re: Texas sex offender & mandatory polygraph
Reply #183 - Jan 14th, 2016 at 2:55am
Mark & QuoteQuote Print Post  
Dan Mangan wrote on Jan 14th, 2016 at 2:09am:
For the love of God, tell us, the great unwashed.

Ha! you crack me up sometimes Dan. Yes, you may indeed shed the loin cloth! Probabilistic thinking tries to go beyond traditional logic as in Boolean algebra and instead uses probabilistic expressions to address the problems of uncertainty and lack of evidence. Despite the tools Raymond mentioned, practitioners will have to have some working knowledge of statistics and probability to engage in these types of esoteric analyses. I am not aware of much research on PCSOT Sexual History Exam accuracy--correct me if I am wrong about that.

*Raymond, by the way, where I can learn about Ipsative-Z as it applies to scoring?
« Last Edit: Jan 14th, 2016 at 5:08am by Ex Member »  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Dan Mangan
God Member
*****
Offline



Posts: 652
Joined: Jul 31st, 2014
Re: Texas sex offender & mandatory polygraph
Reply #184 - Jan 14th, 2016 at 2:37pm
Mark & QuoteQuote Print Post  
The American Polygraph Association on test accuracy, 1997-2012:

Polygraph testing is 98.6% accurate.  Cool

The American Polygraph Association on polygraph accuracy, 2012-2015:

Polygraph testing is, on average, 89% accurate for specific-issue exams and 85% accurate for multiple-issue exams.  Roll Eyes

APA chairman Raymond Nelson on polygraph accuracy, 2016:

"The test is probably more accurate than some people wish, and also probably less accurate than some people wish."  Cheesy
  
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Drew Richardson
Especially Senior User
*****
Offline



Posts: 427
Joined: Sep 7th, 2001
Re: Texas sex offender & mandatory polygraph
Reply #185 - Jan 14th, 2016 at 3:15pm
Mark & QuoteQuote Print Post  
Dan et al,

The problem with assessing accuracy of polygraph screening is that there is virtually no construct validity to the testing paradigms, e.g., the National Academies of Sciences Polygraph Screening study was forced to use specific issue criminal studies (which have their own distinct construct validity issues) to draw conclusions because of a (almost complete)) lack of screening studies to include in its pool of examined studies.  

There is a world of difference between the fishing expedition we know as polygraph screening and a specific incident criminal exam for which it is at least known that a given crime took place.

Prior to the NAS report probably the best studies done were those of two former DoDPI researchers, Charles Honts and Shelia Reed in the late 1980s and early 1990s, resp.  

As I recall the former concluded that the chance of catching a spy with a screening exam was something on the order of two percent (I think far less, but further damning the process is the fact that Charles was, at least initially, prevented from publishing the study because of the results and conclusions contained therein).  Charles, if you happen to still follow APO and see this post, please chime in and elaborate.

Sheila was the principal researcher involved in developing DoD’s Test for Espionage and Sabotage (TES), a format used widely for many years (perhaps still is in some form).  Upon considerable reflection, Shelia many years ago concluded that this exam format was not valid and that its practice in the field should be immediately discontinued.  But once again, the (federal) polygraph community refused to listen to its own expert.

The underlying considerations for the PCSOT are essentially the same as for other screening applications.  I see no reason and no likelihood that it is any more valid as a diagnostic test than the TES, etc.
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Raymond Nelson
Guest


Re: Texas sex offender & mandatory polygraph
Reply #186 - Jan 14th, 2016 at 4:52pm
Mark & QuoteQuote Print Post  
Dr. Richardson

I believe you may have confused a low poster probability with test sensitivity. 

It is an easy mistake and I assume it is accidental. As I'm sure you known, test sensitivity is a descriptive proportion that does not change whether the frequency count is small or large. 

What you seem to be referring to is the posterior probability when screening for an issue for which the frequency count is small relative to the population. 

As you know, some interesting things can happen to the posterior under a strong prior - particularly the proportion of true positives and false positives. 

But the actual test sensitivity level - the proportion of actual positive cases that are classified as positive - is not affected by the prior or frequency. 

RN
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Drew Richardson
Especially Senior User
*****
Offline



Posts: 427
Joined: Sep 7th, 2001
Re: Texas sex offender & mandatory polygraph
Reply #187 - Jan 14th, 2016 at 5:43pm
Mark & QuoteQuote Print Post  
Raymond,

Although I have made reference to Bayesian considerations of screening exams many times, the major point of my last post is that one can not determine the test sensitivity of a screening exam (examining a situation in which no probed-for crime or behavior is known to have been committed) by looking at specific issue criminal tests (either involving single or multiple issue(s) in which a crime/behavior is known to have been committed).  The considerations for both examiner and examinee are different; the examiner pre-test assertions to examinee are different in nature and scope, etc.  Confusing one with the other leads to construct validity issues in experimental design.


Although not mentioned in the last post, I think it is quite likely in the real world of criminal specific incident testing that perceived test sensitivity is higher/artificially inflated in a manner not available with screening exams, i.e., I think it is highly likely that examiners are influenced by more-likely-correct-than-wrong investigative hunches passed on by case agents/investigating detectives.
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box MagicSteve
Guest


Re: Texas sex offender & mandatory polygraph
Reply #188 - Jan 14th, 2016 at 5:54pm
Mark & QuoteQuote Print Post  
Drew Richardson wrote on Jan 14th, 2016 at 5:43pm:
I think it is highly likely that examiners are influenced by more-likely-correct-than-wrong investigative hunches passed on by case agents/investigating detectives.


What you appear to be saying here is that the scoring of the chart is influenced by the examiner's impression of the test subject... am I getting the wrong impression? And, if that is the case, then what does that say about the validity of polygraph examinations? 

Also, how does that work for sexual history polygraphs, when there is no known wrong-doing?
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Drew Richardson
Especially Senior User
*****
Offline



Posts: 427
Joined: Sep 7th, 2001
Re: Texas sex offender & mandatory polygraph
Reply #189 - Jan 14th, 2016 at 6:20pm
Mark & QuoteQuote Print Post  
MagicSteve,

Yes, I do believe that polygraph examiners can be and perhaps likely are influenced by non-exam considerations such as the investigative hunch that I referred to in my last post.  This influence may well be reflected in the conduct of all phases of the exam to include the intonation of the words in exam questions as posed to an examinee during the in-test phase of the examination.

In general the investigative-hunch influence would not apply to screening exams, but if such a hunch were passed on by a court appointed psychologist (meeting on a regular basis with a convicted sexual offender) to an examiner, the aforementioned considerations would apply. The difference being that the court appointed psychologist may well have less of a basis for the hunch than the criminal investigator bringing a suspect for an exam.  I'll leave that for others to ponder/argue.
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Raymond Nelson
Guest


Re: Texas sex offender & mandatory polygraph
Reply #190 - Jan 15th, 2016 at 2:56pm
Mark & QuoteQuote Print Post  
Dr. Richardson,

You and I, and a number of polygraph examiners, share the same concern about the importance of objective analytic models in credibility assessment, and the potential hazards associated with subjective analytic methods. 

Just so that I’m not feeling that this is some simple game of “punk-the-chump,” - as if this is a real discussion ...

we should remember that the lack of an “exact” or “precise” statistical estimate for an input parameter (test sensitivity) does not prevent the use of Bayesian analysis. The point for Bayesians seems to be to define what we have to start with, and then use the test or experimental data and Bayes' theorem to improve on that.

Bayesians could simply treat the test sensitivity parameter as a random variable. Bayesians often quantify unknown parameters using anything from blunt or thin available information to subjective belief (a hunch or suggestion) to markov chains or stochastic simulation or monte carlo methods, and other solutions to estimate the unknown parameter. 

Or we could simply use a known input parameter.

If one starts with a single hypothesis or theoretical assumption that some recordable changes in physiological activity are loaded onto different types of test stimuli as a function of deception or truth-telling (or concealed information) relative to the investigation target stimuli (NAS/NRC, 2003 wrote “ … discriminates deception and truth at rates significantly greater than chance…”).

Sure, anyone can choose to reject the data and evidence, but that does not change the data and evidence that support a conclusion that the recorded physiological data can and does discriminate deception and truth-telling at rates significantly greater than chance.

The subject/hunch argument is entertaining for some, but if mere subjective intuition were driving the polygraph test then we would need to somehow explain the effectiveness of contemporary statistical algorithms. It’s more likely that there is something important in the recorded physiological data.

The null hypothesis to all of this is that all physiology is random and not correlated with deception or truth-telling (or concealed information) in any way.

The theory that some recordable differences in physiology are loaded onto different test stimuli as a function of deception is not completely lost on other. Your own research on Brain Fingerprinting and the Concealed Information Test is basically premised on the same overarching idea that some noisy but real changes in physiology are loaded in response to different types of test stimuli (key or target stimuli and the other stimuli) . Other groups of scientists have also begun to study the possibility of using data from different test configurations and different physiological recording sensors with statistical or machine algorithms designed to indentify subtle differences for credibility assessment tasks. 
 
Having a working theory and some some sensors that have been shown to discriminate deception and truth in some noisy way, we still don’t know the test sensitivity do we? 

Remember that Bayesian analysis can also be used to study unknown parameters such as priors and other things. 

Or maybe we merely have to calculate a multinomial distribution of the combined scores from an array of sensors under the null-hypothesis that all data are random with how many presentations of the investigation target stimuli. And we could calculate the density of any test score against the random multinomial distribution. Then we could proceed to determine statistically whether the recorded data do or do not appear to be loaded systematically or randomly.   

But from a Bayesian perspective, we could use the multinomial density as the input parameter for our Bayesian analysis. 

But we still need to be clear about the fact that we are talking about a posterior probability (still conditional on the prior as an input parameter) in the end.

Of course the prior is often another  known parameter, but that does not seem to stop Bayesians. Bayesians would use available information to quantify the unknown parameter, or simply treat it as a random variable. We could also use Bayes' theorem to improve refine our knowledge about the unknown prior. 

Have a nice day.

RN
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Drew Richardson
Especially Senior User
*****
Offline



Posts: 427
Joined: Sep 7th, 2001
Re: Texas sex offender & mandatory polygraph
Reply #191 - Jan 15th, 2016 at 4:48pm
Mark & QuoteQuote Print Post  
Good Morning, Raymond.  Thank you for the continued discussion.  I will soon be leaving home and be away for a couple of days, but thought I would address a couple of points that you made in your last post.

Quote:
The subject/hunch argument is entertaining for some, but if mere subjective intuition were driving the polygraph test then we would need to somehow explain the effectiveness of contemporary statistical algorithms. It’s more likely that there is something important in the recorded physiological data.


The investigative hunch impact that I referred may well lead to apriori examiner expectations about examinee guilt, and I don't believe scoring algorithms shed any particular light on the issue.  Admittedly scoring/conclusions drawn (either from manual scoring or from algorithms applied to digital data) are quite reliable. A class of second week polygraph students can more or less end up with similar scores for a set of charts.  But as I pointed out before, scoring is downstream of the damage done by the hunch and the expectations it provides, e.g., damage caused through word intonation in presented exam questions.  All scoring indicates is that we can reliably score a damaged process, but not necessarily recognize that we are dealing with a damaged process.

The examiner hunch is but one of several potential influences (I believe one of the most likely to occur in a field criminal setting) leading to artificial (unrelated to the polygraph exam itself) expectations. My first exposure to this phenomenon was early in basic polygraph training.

In the simulated-crime polygraph examinations we conducted, my classmates and I were confronted with more guilty/deceptive examinees than innocent/truthful examinees.  This was arranged intentionally by the instructional staff so as to give us practice conducting interrogations.  As one would predict, our overall scoring was reliable and correct with the quickly developed expectation and perceived probability of facing a guilty examinee.  What was initially surprising, but now quite understandable, was that when we encountered an innocent examinee after a string of guilty examinees was that our across the class scoring was again quite reliable, but we were nearly all reliably wrong with false positive results.  This happened on several occasions.

So, in summary, I don't believe the notion of examiner expectations is merely entertaining or a mere distraction from the business of applying inferential statistics to a data set.  The ability to successfully apply a scoring algorithm with a simulated crime in the absence of the aforementioned influencing investigator hunch really says nothing about its influence in the field or our ability to successfully apply such algorithms in that setting.

You mention one oft quoted statement from the NAS polygraph study:

Quote:
(NAS/NRC, 2003 wrote “ … discriminates deception and truth at rates significantly greater than chance…”).


The vast majority of the studies looked at were simulated-crime studies published in the polygraph literature having little bearing on understanding/estimating the validity of field exams and virtually no bearing on field screening exams.  As you may or may not know, the initial goal of the NAS/NRC studies was to evaluate screening validity in connection with the sponsor's (DOE) desire to implement screening more fully in the national lab system.  As the members of the panel quickly discovered and politely made mention of, the quality of the studies given them was not only lacking but more troubling, with the initial goal in mind, was that the applicability to field screening was almost nil.  All of this is not in conflict with the assessment that you quote.

You mention in passing concealed information.  At some point I would like to direct our attention and future discussion there.  Although there is a great disconnect between our knowledge of brain biology and deception, such is not the case for memory encoding and retrieval.

Have a very pleasant weekend.  

Best, Drew
« Last Edit: Jan 15th, 2016 at 5:18pm by Drew Richardson »  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box MagicSteve
Guest


Re: Texas sex offender & mandatory polygraph
Reply #192 - Jan 15th, 2016 at 5:43pm
Mark & QuoteQuote Print Post  
Can someone explain how you get a reading of deceptiveness out of a chart, when the only thing that is charted is basic physiological responses? Out of all these discussions, no one has come even close to addressing the fact that one physiological response is no different from another. I think we all agree, 100 percent, that the exams measure physiological responses to questions.

If there is no difference between one chartable response and another, then how do you know if someone is being deceptive? 

You don't. Remember the whole 'you can't measure something that doesn't exist' thing I went on and on about? This is what I mean. Scientifically, the difference between the responses, as recorded by a polygraph machine (or any machine known to man), is not detectable. You can't find someone to be 'deceptive' when you have absolutely zero clue what 'deceptive' looks like scientifically. No one is addressing the fact that the reason that polygraph 'results' are essentially null and void, and the 'test' holds so little validity both scientifically and legally, is because no one can tell the difference between what a guilty, or deceptive, or frightened, or excited, or angry, or aroused (you get the gist) response looks like.

Literally, this is the core of why polygraph examinations are so ill regarded. The examinations simply do not do what they are purported to do. Plain and simple. 

I have to admit, though... they are darned good at taking blood pressure readings, measuring breathing, heart rate, galvanic skin conductivity readings, and the like... darned good.   Roll Eyes
  
Back to top
 
IP Logged
 
Paste Member Name in Quick Reply Box Dan Mangan
God Member
*****
Offline



Posts: 652
Joined: Jul 31st, 2014
Re: Texas sex offender & mandatory polygraph
Reply #193 - Jan 15th, 2016 at 6:02pm
Mark & QuoteQuote Print Post  
MagicSteve, while chart interpretation and polygraph scoring rules are, generally speaking, relatively well defined, here's an interesting excerpt from a document that speaks to the larger scope of the polygraph "test" and subsequent evaluation process. (Note: The document is reportedly included with some polygraph software.)

-----------------------------------------------------------------------


    There are verbal and non-verbal cues commonly observed in examinees who prove to be deceptive. The examiner should be observant during the entire testing process for these non-chart related markers of deception:

        The examinee made an attempt(s) to avoid taking the polygraph examination
        The examinee was late arriving for the examination without a legitimate excuse
        The examinee tries to limit the length of the polygraph session
        The examinee expresses distrust or non-belief in polygraph
        The examinee tries to dominate the conversation and talks incessantly
        The examinee complains of some physical ailment or medical condition prior to being asked about his health and physical condition
        The examinee is quick to volunteer information regarding medications and then asks “will that effect the test”
        The examinee tries to oversell his honesty / truthfulness / character / reputation, etc.
        The examinee gives excuses why he might fail the examination
        The examinee’s story is absurd, illogical or in direct conflict with case facts
        The examinee provides little details regarding critical parts of his story
        The examinee uses memory qualifiers to excess when answering questions
        The examinee answers relevant questions with half-truths
        The examinee minimizes the seriousness of the allegation / crime
        The examinee blames the person making the allegations or victim and provides reasons why he has been wrongfully accused
        The examinee avoids answering direct questions about the relevant issues with “yes” or “no” and provides evasive answers to those questions
        The examinee answers with a question
        The examinee tries to buy thinking time before answering relevant questions
        The examinee uses defensive statements when asked a direct question
        The examinee exhibits excessive physical indicators of unconscious stress relief such as yawning, stretching, knuckle cracking, throat clearing, sniffling, burping, etc.
        The examinee is “overly” anything
        The examinee deviates from his norm at critical times
        The examinee exhibits clusters of non-verbal cues
        The examinee makes small admissions designed to cloud the relevant issue
        The examinee exhibits an unusual interest or knowledge about polygraph

------------------------------------------------------------------
You can learn more about the genesis of the document here:

https://antipolygraph.org/blog/2015/11/15/leaked-documents-further-confirm-polyg...
  
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Raymond Nelson
Guest


Re: Texas sex offender & mandatory polygraph
Reply #194 - Jan 15th, 2016 at 6:08pm
Mark & QuoteQuote Print Post  
Dr. Richardson,

Thank you for the reply and discussion.

What your concerns and experience with manual scoring of polygraph data actually seem to speak to is this: a)  the importance of more standardized, perhaps even more automated, test administration procedures that can remain objective and robust against the influence of human intuition/fatigue/bias/etc. during test administration, and b) the importance of structured, perhaps even automatic, algorithmic analytic models that are similarly more objective robust against the influence of human intuition/fatigue/bias/etc., leading to greater reproducibility of analytic results and a greater ability to incorporate new emerging knowledge and information to possibly improve the precision of an analytic result. 

Without clearly structured algorithmic methods the analytic process will continually subject to human factors and also to the need to train and develop those difficult-to-quantify "clinical" or expert judgement skills with every new class of professionals (which does not fully address the perishable skill problem). 

These are issues that have been in consideration in testing and diagnostic work since at least Meehl (1954) and Nunnally (1967), and are certainly worth discussion. 

The alternative what Meehl referred to as "mechanized" (antiquated term, but it makes the point) is to rely on less structured forms of professional or clinical judgement, for which the trend in the evidence has been somewhat clear for a long time.

Despite the clarity of evidence, learning to rely on structured solutions can introduce existential conflicts to minds of some professionals. Also, there are is a need for ethical discussion around what some have called the "sky-net problem" when allowing mechanistic (machine learning) processes to make decisions about humans. The ethical discussion here involves both what we should expect of structured machine processes, and also how to implement them in a manner for which human decisions remain human processes. 

RN

I will also be away from home for the next week.

Safe travels
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1 ... 11 12 [13] 14 15 ... 25
ReplyAdd Poll Send TopicPrint
Texas sex offender & mandatory polygraph

Please type the characters that appear in the image. The characters must be typed in the same order, and they are case-sensitive.
Open Preview Preview

You can resize the textbox by dragging the right or bottom border.
Insert Hyperlink Insert FTP Link Insert Image Insert E-mail Insert Media Insert Table Insert Table Row Insert Table Column Insert Horizontal Rule Insert Teletype Insert Code Insert Quote Edited Superscript Subscript Insert List /me - my name Insert Marquee Insert Timestamp No Parse
Bold Italicized Underline Insert Strikethrough Highlight
                       
Change Text Color
Insert Preformatted Text Left Align Centered Right Align
resize_wb
resize_hb







Max 200000 characters. Remaining characters:
Text size: pt
More Smilies
View All Smilies
Collapse additional features Collapse/Expand additional features Smiley Wink Cheesy Grin Angry Sad Shocked Cool Huh Roll Eyes Tongue Embarrassed Lips Sealed Undecided Kiss Cry
Attachments More Attachments Allowed file types: txt doc docx ics psd pdf bmp jpe jpg jpeg gif png swf zip rar tar gz 7z odt ods mp3 mp4 wav avi mov 3gp html maff pgp gpg
Maximum Attachment size: 500000 KB
Attachment 1:
X