When Sources Lie: Why You Can’t Rely on Confidence or Consistency

Editor’s Note: How do reporters know when their sources are telling the truth? In the collapse of Rolling Stone’s November story on campus rape, a gullible reporter relied far too heavily on the unchallenged word of an alleged victim. Heinrich Böhmke, a legal adviser to the African Network of Centers for Investigative Reporting, takes a look back at the case and warns that journalists should learn from the law and apply a tough standard even when sources are convincing, confident, and have consistent stories. 

In December 2014, Sean Woods received the call editors dread the most. A Rolling Stone reporter told him she no longer stood by her story of a horrible rape supposedly committed by frat boys at the University of Virginia.

The feature, published a few weeks earlier, broke readership records at the iconic magazine. It caused a national uproar. It is not hard to see why. The story recounts the dramatic allegations of a young woman, lured by her date to a fraternity house, there to be gang-raped on broken glass by seven young initiates. The line the story took was that the victim’s experience was emblematic of a “culture” that female students faced on campuses across the United States. This heinous crime was further compounded by the defensive, almost dismissive, response of University authorities.

Sabrina Erdely is an experienced contributor to Rolling Stone. Her last story, “A Rape on Campus,” went through a fact checker, an editor, and legal counsel. Other writers, however, notably in the Washington Post, had no difficulty exposing her exposé. At best, it was shoddy, prejudicial journalism with the underlying claim still intact. Fair to middling, Erdely was taken in by a teenage attention seeker, which the elementary techniques of investigative reporting could have detected with ease. At worst, Jackie’s account was an inherently improbable hoax. The reason Erdely recounted it at all was that it confirmed her ideological and “gothic” expectation of how entitled, rich, male undergraduates were apt to behave. As Cathy Young put it, Erdely’s story flowed from the pen of one possessed of “the troubling zealotry of advocates for whom believing rape claims is somewhat akin to a matter of religious faith.”

In due course, the story unraveled in newspapers and blogs across the United States. Rolling Stone approached the dean of the Columbia School of Journalism with a simple and acute brief. Would Steve Coll help them understand how they had got things so terribly wrong? The review was published in early April 2015. It found that the writer and editors were far too deferential to the single pseudonymous source, the victim, Jackie. Unwisely sensitive to her emotional well-being, Rolling Stone permitted Jackie to dictate the scope of the investigation and even their own questions to her.

Erdely further failed to verify crucial facts such as that the alleged rapist actually existed or that Jackie’s contemporaneous claims to friends matched what she was saying now. Erdely’s breach of press protocol did not end there. In what a Washington Post writer called an “unfathomable deceit,” Erdely and her editor misattributed dialogue and unfairly deprived the targets of her investigation of an opportunity to reply to — or discount — intensely damaging accusations.

The Coll review has itself attracted edifying commentary about the pitfalls of narrative journalism and the shaming by “activists” of those who correctly pointed out the story’s glaring holes. Amidst the condemnation of Rolling Stone, there is a sense that this is a learning moment for the profession as a whole. While some of the faults of the piece flow from Erdely’s unique and perhaps even malicious failings as the author, other errors are common to many in the profession. The error that has attracted the least commentary so far is the idea that Jackie’s credibility was legitimately enhanced because her detailed story stayed consistent over its retellings to the journalist and because of her “confident” demeanor during these narrations. Many a reporter has fallen into these two holes before.

The Coll review states that Erdely “remembered being ‘a bit incredulous’ about the vividness of some of the details Jackie offered, such as the broken glass from the smashed table. Yet Jackie had been ‘confident, she was consistent.’” Erdely repeats the quality of consistency in Jackie’s versions of events in an interview to Slate. This is what convinced her. All those details, staying the same, with every interview.

The fact-checker, who spent four hours on the phone reviewing Jackie’s story, was also impressed by her vivid consistency. “She wasn’t just answering, ‘Yes, yes, yes,’ she was correcting me.”

In ascribing credibility to a story because it stays consistent, investigative journalists are committing what amounts to a fallacy in evidentiary terms. Muckraking is the younger profession and journalists are well advised to take heed of the extreme caution, amounting to an exclusionary rule, with which the older profession of lawyering treats prior consistent statements. Contrary to common sense, when a witness repeats an accusation he has made on a prior occasion, this does not increase the credibility of his claim.  That witness could be consistently lying or consistently telling the truth. Mere consistency does not tell whether the scale is rigged or not.

Of course, if a witness is accused of recently fabricating a version, the fact that he has long been making the same claim is a fair rebuttal of this imputation. An account given immediately after an event also carries a certain weight. But judges and juries are usually spared the information that a complainant has, thereafter, repeated their allegations in detail. This is because logically, self-corroboration, sheds no light on credibility.

Strange as it may seem at first glance, while prior consistency does not augment credibility, prior inconsistency does validly impeach or discredit a witness. There certainly were material and conspicuous inconsistencies in Jackie’s story, if only Erdely had bothered to find them.

Demeanor is another notoriously unreliable measure of a person’s honesty. Studies consistently show that Erdely had odds little better than flipping a coin in discerning whether Jackie was telling the truth from her “confidence.” Confidence means as little as being coy. Indeed, Erdely reads confidence as increasing Jackie’s credibility during some of their interactions, while Jackie’s fear and reticence during other encounters does exactly the same thing.  This underscores the unreliable nature of demeanor as an objective sign of truth-telling.

None of this means that demeanor is worthless. Experienced journalists often have “a feeling” about a story. The feeling arises from a source’s tears or smiles, their straightforward manner of speaking, co-operation, concessions, and that elusive quality we call genuineness. The most careful journalists, though, do not hand out the wreath of genuineness based on demeanor alone. Even if reporters do not forensically analyse their structure of belief in these terms, they have also performed checks for contradiction, reliability of observation, and bias. They have thought really hard about the inherent plausibility of an allegation and sought corroboration for tales that go against the way the world ordinarily works. All of this combines with a good demeanor, in the alchemy of investigative reporting, to produce the “feeling” that a source can be trusted.

There are huge risks, however, if the feeling of genuineness arises only from the inconsolable weeping or, alternatively, the steady-eye and welcoming handshake of a source.  Such a story may well be a captive of stereotype, pop-psychological sleuthing, or an Erdelian confirmation bias that has kicked in.

Investigative reporting is an inherently contentious business. If the target is important and well resourced, if the stakes are high, there will be significant, coordinated, and biased blowback. The greater the impact of a piece, the shriller the shrieks of indignation, the sterner the letters from lawyers and the greater sanctimony from scrutinizing commentators. When that kind of heat arrives, it is far better to have forensically examined the facts of your story yourself before publication than to have your adversaries do so afterwards. The Coll review of the retracted Rolling Stone report has supplied useful new guidelines by which the dreaded recantation phone call may be avoided.  It turns out that old law books on evidence may be consulted, too, for additional, cross-over insights on how (not) to assess the credibility of a source.

A version of this article was previously published in the Premium Times (Nigeria).

Heinrich Böhmke is an adviser to the African Network of Centers for Investigative Reporting. He works as a trainer and director at the Specialised Skills Institute of South Africa, and has investigated and prosecuted corruption and sexual misconduct cases.