But researcher says psychology publishing still improving

A new study adds further criticism to the “replication crisis” in scientific research.

Researchers from a variety of universities looked at “164 quantitative papers published from 2009 to 2018 in 54 journals in the social and behavioural sciences,” according to the summary in the Nature article. The team “attempted replications of 274 claims of positive results” but found only about half could be replicated.

The researchers found that many published findings did not consistently hold up when tested again, although the exact replication rate varied depending on how success was measured.

For Michael Inzlicht, a professor and one of the study’s authors, the findings matter because psychology research often reaches the public long before its reliability is fully understood.

“Psychology constantly reaches the public through the media. If a finding doesn’t replicate, media headlines about it are closer to entertainment than knowledge, yet people make real decisions based on them,” Inzlicht told The College Fix via email on May 31.

The consequences extend beyond attention-grabbing headlines. Psychology research also influences how mental illness is treated, how schools operate, and how policymakers approach social problems.

“The bigger stake is clinical,” the University of Toronto professor said. “Psychology sits at the heart of how we treat mental illness. If the underlying work isn’t credible, why trust the treatment recommendations that flow from it?”

Still, Inzlicht cautions against interpreting the findings as evidence that psychology is fundamentally broken.

“It’s more complicated than that,” he said. “A lot (though far from all) of older work doesn’t hold up, but the field saw the problem and has been self-correcting since the mid-2010s.”

According to Inzlicht, psychology has undergone substantial reforms during the past decade. Researchers now commonly preregister studies, share data publicly, and conduct larger, more rigorous experiments.

“There have been tremendous improvements. The field looks nothing like it did ten years ago,” Inzlicht said.

Failed replications reveal weaknesses in some published findings, but they also demonstrate science’s ability to identify and correct its own mistakes.

Inzlicht sees that process as one of science’s greatest strengths.

“Despite everything psychology got wrong, I’m still in love with science,” he said. “The errors were caught and are being fixed by psychologists themselves, from the inside.”

His hope is that readers ultimately trust psychology more, not less.

“A field willing to audit itself in public is more trustworthy than one that claims it never erred,” Inzlicht said.

Humanities scholar says problem lies in society’s obsession with studies

The former dean of humanities at the University of Utah says people put too much stock into various studies and how to improve themselves.

“I think the questions to ask are ones that I don’t see others asking, which is: why is psychological research so dominant in our culture?” Hollis Robbins, a professor at the University of Utah and a fellow at Harvard University, told The College Fix via email. “Why are so many people obsessed with studies about people?”

Robbins is among those who believe the conversation should extend beyond individual research findings and focus on the culture that produces and consumes them.

“I don’t have much confidence in any studies,” Robbins told the College Fix in a May 30 email. “I think there is a massive overproduction of ‘studies’ for reasons I don’t fully understand.”

However, Robbins remains skeptical about society’s growing dependence on research findings.

“I think people read studies to know what to think when they might better think for themselves.”

MORE: Psychology associations have lost their credibility