Stathis and Marinakis’ “Shadows into Light”: A Failure of Peer-Review

In January 2020, the Journal of Child Sexual Abuse published online an article by Marigo J. Stathis and Maria M. Marinakis purporting to show that the Computer Voice Stress Analyzer (CVSA), a quack device marketed as a lie detector by the so-named National Institute for Truth Verification (NITV), is highly useful for interrogating pedophiles who use the internet in the commission of crimes, producing no false positives and a 100% confession rate from those who failed.

The authors of “Shadows into Light: The Investigative Utility of Voice Analysis with Two Types of Online Child-Sex Predators” declared to the journal that they have no conflicts of interest to report. However, an investigation by AntiPolygraph.org reveals that both the article’s primary author and the source of the data used for the study have ties to NITV that call into question the article’s credibility.

The Journal of Child Sexual Abuse is published by Taylor & Francis, who on their web page on “competing interest” or “conflict of interest” state: “Competing interests can be financial or non-financial in nature. To ensure transparency, any associations which can be perceived by others as a competing interest must also be declared.”

Marigo J. Stathis

Stathis did not disclose that she is the recipient of the National Association of Computer Voice Stress Analysts’ (NACVSA’s) “Chapman Award” for an article she co-authored with James L. Chapman, who until his death in 2012 was a member of the NACVSA board of directors.

NACVSA is an arm of NITV, which has used the article Stathis co-authored with Chapman to claim in marketing copy that “according to an independent, peer-reviewed, published 18-year study” CVSA “has an accuracy rate of 98%.” However, an earlier investigation by AntiPolygraph.org revealed that the Chapman & Stathis article was published in an obscure publication of the Ukrainian government that has dubious claim to being peer-reviewed and that, in any event, the study was not based on NITV’s CVSA, but rather a different voice stress analyzer whose manufacturer went out of business some three decades ago.

In January 2013, Stathis, a resident of Maryland, gave a presentation at NITV’s Advanced Continuing Education Course in Fort Lauderdale, Florida on the topic of “Current VSA Research.”

Bob McCarty,
NACVSA Chapman Award recipients Bob McCarty, Chad Jeansonne, and Marigo Stathis

In January 2015, Stathis was again in Florida for NITV’s Advanced Examiners Conference held at Disney’s Coronado Springs Resort, where she was photographed with other winners of NACVSA’s Chapman Award.

In January 2017, Stathis was yet again in Florida for NITV and NACVSA’s 29th Annual Professional Development and Continuing Education Conference, where she gave a lecture. NITV’s description of the event characterizes her as a “valued member”:

Some of the highlights of the conference were lectures by valued members who have advanced knowledge in specialized areas. One such person was Marigo Stathis, a neuroscientist who co-authored a peer-reviewed field study of the CVSA with Professor James Chapman. Marigo has a deep understanding of the scientific workings of the CVSA, but she’s also great at translating the data into everyday language. Her lecture discussed the protocols of the Chapman study and described how they discovered that the CVSA has an accuracy rate of 99.69%, a precision rate of 99.67%, and a verified confession rate of 96.4 %.

In response to a request for comment, Stathis denied having a “close and continuing” relationship with NITV and affirmed that “[a]ll of [her] efforts pertaining to the ‘Shadows into Light’ article were of a pro-bono nature…”

Nonetheless, a reasonable person might conclude that Stathis’ above-documented association with NITV constitutes a conflict of interest that should have been declared.

In “Shadows into Light,” Stathis and Marinakis write that “[t]he data contained in this paper originates from a southeastern US Internet Crimes Against Children (ICAC) Task Force, an organization that successfully uses both advanced interviewing techniques and investigative tools when eliciting disclosures.” Stathis and Marinakis do not specify which ICAC Task Force produced the data on which they rely, but AntiPolygraph.org’s investigation confirms that it is the Central Florida ICAC Task Force, and that the key person involved was former detective sergeant Jerry W. Crotty II of the Manatee County Sheriff’s Office.

Jerry W. Crotty II

Like Stathis, Crotty is a recipient of NACVSA’s “Chapman Award” (for 2015):

2015 – D/Sgt. Jerry W. Crotty II: Jerry is with the Manatee Co. Sheriff’s Office (FL) and was the recipient of the fourth annual Professor James L. Chapman Award for Excellence. He developed innovative strategies using the CVSA to protect the safety and welfare of children in Florida and elsewhere. Jerry is assigned to the Federal Internet Crimes Against Children Task Force (ICAC), and he introduced the CVSA as a screening tool during investigations of ‘Traveler and Pornography Cases.’ His efforts have resulted in more accurately determining the past predatory histories of these offenders, and also has directly led to the identification and recovery of multiple ‘live victims’ of sexual predators. As a direct result of Jerry’s innovation and creativity the CVSA has been acquired by many ICAC Task Forces across the USA, which have also adopted Jerry’s specialized interviewing approach.

In February 2018, six months after the cutoff date for Stathis & Marinakis’ data sample, Detective Sergeant Crotty “retired to avoid a demotion and discipline.” Shortly thereafter, NITV hired Crotty as “Director of Law Enforcement Operations.”

That Crotty is the source of Stathis and Marinakis’ data is confirmed by Crotty himself in an online biographical sketch:

His work is…the subject of an article published in the Journal of Child Sex Abuse, where his use of the CVSA was analyzed in online sex traveler and child porn offenders. Using this technique, he was able to recover 87 unknown live victims of child sex abuse.

Concordantly, in “Shadows into Light,” Stathis and Marinakis write, “…as a result of voice stress analysis procedures, 87 previously undiscovered live victims were identified.”

There can be little doubt but that Stathis and Marinakis’ data is based on CVSA examinations conducted primarily, and perhaps exclusively, by Jerry Crotty.

In response to a request for comment, Stathis stated, “I received the data relevant to ‘Shadows into Light’ in 2017 from a Southeastern US ICAC team that consisted of several individuals who originally approached me in 2016, none of whom was associated with or worked for any forensic or truth verification software/hardware company at that time.”

Nonetheless, a reasonable person might conclude that Stathis and Marinakis’ data source’s association with NITV constitutes a conflict of interest that should have been declared.

Stathis & Marinakis’ “Shadows into Light” is also beset by methodological flaws and omissions that make it difficult to generalize the authors’ conclusions to any other population:

  • What were the total arrest figures for the ICAC unit during the relevant period?
  • How many arrestees refused all interrogation?
  • How many were interrogated without CVSA, and what were their admission/confession rates?
  • How were the CVSA charts scored? Manually by the operator? Or automatically by the CVSA software?
  • How many CVSA operators were involved? (Evidence suggests that the number is one, but the authors should have included this data point.)

Stathis and Marinakis write that “[t]his study’s de-identified raw data can be furnished upon reasonable request from the corresponding author.” In order to better understand their research findings, AntiPolygraph.org requested this data by email to the corresponding author, Marigo Stathis, on 22 July 2020 and received no reply. However, in October 2020, following AntiPolygraph.org’s request for comment for this article, Stathis wrote:

…we have appropriately considered your initial July 22, 2020 request for the study’s raw data. However, we additionally require a detailed description of how you intend to use the data. As the corresponding co-author, I will then be able to forward your request to the agencies that comprise the US ICAC task force (that generated the data) with the stated reasonable potential use of such data in order to get their consent to release the data. The latter will be contingent on having met specific criteria re a written limited rights data use agreement, with appropriate protection of the data and defined limited reasonable use.

Given that Stathis did not reply to our request for the study’s data, only addressing it more than two months later when asked for comment for this article, her claim that she needs more information to entertain our request seems disingenuous.

The late astronomer and science communicator Carl Sagan famously observed that extraordinary claims require extraordinary evidence. Stathis and Marinakis’ claim that CVSA—a scientifically baseless device that NITV itself has acknowledged in court “is not capable of lie detection”—produced no false positives and a 100% confession rate from those who failed is an extraordinary claim indeed.

Regrettably, the evidence of Stathis and Marinakis’ “Shadows into Light” is murky.

NITV Falsely Claims Peer-Reviewed Study Shows CVSA Is Highly Accurate

NITV’s logo (misappropriating the Great Seal of the United States)

The so-named National Institute for Truth Verification (NITV), which markets a putative lie detector called the Computer Voice Stress Analyzer (CVSA), and its associated group, the National Association of Computer Voice Stress Analysts (NACVSA),1 have in recent years claimed that a 2012 independent peer-reviewed study shows that CVSA has an accuracy rate on the order of 97% or higher. This despite the fact that NITV has admitted in court that “the CVSA is not capable of lie detection.”

An investigation by AntiPolygraph.org reveals that:

  1. The study cited by NITV/NACVSA has dubious claim to being peer-reviewed;
  2. The study did not involve NITV’s CVSA, but rather a different voice stress analyzer called the PSE-101 that was marketed by Dektor Counterintelligence and Security, Inc., a company that has been defunct for some three decades;
  3. The study is not “independent” in that its primary author had an undisclosed close and continuing relationship with NITV/NACVSA.

The citation for the study touted by NITV and NACVSA is:

Chapman, James L. and Marigo Stathis. “Field Evaluation of Effectiveness of VSA (Voice Stress Analysis) Technology in a US Criminal Justice Setting,” Criminalistics and Court Expertise, 2012 Annual Edition, Number 57.

However, a search of research libraries returns no publication titled Criminalistics and Court Expertise. As James R. Wygant noted in a 2014 commentary in the American Polygraph Association’s APA Magazine, “no trace of a publication by that name can be found on the Internet, although copies of the article itself are available.”

It turns out that the article appeared in a yearbook of the Ukrainian Ministry of Justice whose Russian title is Криминалистика и судебная экспертиза (Kriminalistika i sudebnaya ekspertiza). This periodical published Chapman and Stathis’ article in Russian under the title, “Оценка эффективности технологии VSA (Voice Stress Analysis ) на основе практики уголовного судопроизводства в США” (“Otsenka effektivnosti tekhnologii VSA (Voice Stress Analysis ) na osnove praktiki ugolovnogo sudoproizvodstva v SSHA”).

James L. Chapman (1942-2012)

AntiPolygraph.org has obtained copies of the Chapman & Stathis study in both English and Russian.

It may seem odd that American researchers would choose to publish in an obscure Ukrainian yearbook in a language that is not their own.

Writer Bob McCarty helps to solve this mystery. In Chapter 22 of a self-published, fawning book about CVSA titled The Clapper Memo2 that is based in part on “dozens of phone calls over a three-year period” with the study’s primary author, James L. Chapman, McCarty reveals that Chapman and Stathis unsuccessfully attempted to have their article published in “a few reputable criminology-oriented, scientific journals in North America” and that it was rejected by at least two such journals.

Marigo J. Stathis

Only after these rejections did Chapman and Stathis resort to submitting their article to the Ukrainian periodical that ultimately published it.

The article, which is referred to by NITV/NACVSA as “the Chapman study,” suffers serious methodological flaws that should have prevented it from being accepted by any peer-reviewed publication worthy of the name. One such flaw is that Chapman put his thumb on the scale by excluding cases where a confession was unlikely to be obtained, noting (at p. 12 of the English version of his study), “Contract criminals were excluded from this particular analysis, as this special type of offender is known to reject confessions, due to organized crime affiliations, etc., regardless of what their VSA results or the evidence indicate.” There is no scientific rationale for such exclusion.

Notably, the Chapman study does not mention CVSA even once. It instead refers only to “voice stress analysis.” This is because the voice stress analysis “tests” that form the basis for this study were performed by Chapman not with NITV’s CVSA device, but rather with Dektor Counterintelligence and Security’s “Psychological Stress Evaluator.”

In The Clapper Memo, McCarty discloses that Chapman “wrote a lengthy white paper” about the results of voice stress analysis “tests” that he conducted over the course of his career. AntiPolygraph.org has obtained portions of this “white paper,” which ran over 100 pages, that Chapman faxed to John J. Palmatier, who was then researching voice stress analysis, in 1993. In that original paper, Chapman writes, at p. 27:

This field study was drawn from the total number of cases processed with a Psychological Stress Evaluator, PSE-101, by James L. Chapman, Criminologist, during the eighteen years between 1971 and 1989. The procedure in each case consisted of briefing by the requesting agency, suspect interviews, questioning, and the processing of requestioning through a PSE-101, manufactured by the [sic] Dektor Counterintelligence and Security, Inc., Savannah, GA.

The fact that Chapman’s field study was based on his use of the PSE-101 and not the CVSA is unjustifiably omitted from the article published in Ukraine.

Chapman had an undisclosed conflict of interest that helps explain why he would obfuscate what specific voice stress analyzer he used.

At the time of his death on 17 April 2012, Chapman was a member of NACVSA’s Board of Executive Directors. His affiliation with NITV/NACVSA is older than that, however. Chapman’s name appears on NITV’s letterhead as a member of its Board of Directors in a letter dated 11 May 1998.3

James L. Chapman clearly had a close and continuing relationship with NITV/NACVSA. Nevertheless, in a 2011 phone call to Vancouver Sun columnist David Baines, Chapman averred that he had “no real connection” to NITV. Baines writes in a column published on 14 March 2011:

Chapman told me he has just completed a 19-year study confirming that the CVSA device is 96.4-per-cent accurate. He also said the study has been peer reviewed, but when I asked for a copy, he said it has not yet been published, and until it is, he could not provide one.

Chapman purposefully misled Baines by claiming that his study confirmed anything about CVSA.

Nine months later, Chapman delivered a presentation about his study at NITV’s 2012 Annual Advanced Continuing Education Course held 16-20 January 2012 at Fort Lauderdale, Florida. On its website, NITV falsely characterized Chapman’s study as a “19-year field study of CVSA”:

NITV’s misrepresentation of the Chapman study has continued and remains ongoing. For instance, a press release dated 11 January 2013 that currently appears on NITV’s web site falsely proclaims:

A newly published research study in the 2012 annual edition of the scientific journal Criminalistics and Court Expertise reports the accuracy rate of the Computer Voice Stress Analyzer (CVSA®) is greater than 95%, an assertion long made by the system’s manufacturer.

and

Professor Chapman used the CVSA to conduct the research and the results achieved were highly consistent throughout the period the study’s data were collected. The study’s findings revealed the CVSA, when used as an investigative support tool, can accurately predict whether a person under investigation is being truthful or deceptive. The study’s findings are supported by scientifically-accepted statistical models, and by the 96.4% validated confession rate Professor Chapman attained during the course of the 18-year study.

A 3-page synopsis of the Chapman report currently available on the NITV website similarly falsely claims that it “is based upon actual CVSA examinations.”

More recently, NITV repeated its false claims about the Chapman study in federal court in a 2018 lawsuit against Dektor Corporation (not to be confused with Dektor Counterintelligence and Security) and Arthur Herring III when at para. 50 of its Complaint, NITV falsely attested that “a 2012 peer reviewed published study…of the CVSA showed its error rate to be less than 1%….”

And in an announcement for a CVSA training session that was to be held from 10-14 August 2020 (the week that this article is being published), NITV proclaims in bold letters, “With the recently published 18-year study validating the accuracy of the CVSA at 98%, isn’t it time to acquire the latest in truth verification technology?”

In 2020, there is no peer-reviewed research showing that NITV’s CVSA works at better-than-chance levels of accuracy.

NITV managing member Charles Humble did not respond to a request for comment for this article.

  1. The close nexus between the NITV and the NACVSA is reflected in the fact that a single entity, Voice Biometrics, LLC, holds the trademarks for “NITV,” “National Institute for Truth Verification,” “CVSA,” “NACVSA,” and “National Association of Computer Voice Stress Analysts.” In addition, at the time of writing, the websites for both the NITV and the NACVSA share a common IP address: 35.209.16.223. []
  2. McCarty received the NACVSA’s 2013 “Professor James L. Chapman Award for Excellence” for writing this book. []
  3. That letter, from then NITV Executive Director David A. Hughes to Michael G. Adsit of the Canastota, New York Police Department, explained that if that department wished to purchase CVSA equipment and training, its chief would have to sign a letter stating that “the Canastota Police Department will not allow anyone outside of their trained examiners, and specifically Rome Lab, to use or have access to the CVSA and all proprietary interest will be protected.” The U.S. Air Force laboratory at Rome, New York, was at the time conducting research into voice stress analysis and published a report in 2002. []