Why Models are More Important Than Replications

There’s a replication crisis in psychology! No there’s not! Wait, there is! All this back-and-forth can feel exhausting (well, to me at least) but the take-home message for me is this: psychological science can and should be improved, but the issues are far wider and deeper than just “why won’t findings replicate?”

Open science, metascience, and other initiatives in psychology are important - the field should engage in continuous quality improvement (to borrow a phrase from the business-types): reflecting upon how it does it work, what it means, and what goals are worth pursuing. But fixating on whether findings replicate is just another example of psychology’s tendency to over-focus on statistical results, rather than the broader, messier, but ultimately more rewarding process of building, testing, and refining models.

Berna Devezer (a GREAT follow on social media, if you aren’t already) and Erkan Buzbas have a wonderful article out on what a model-centric psychological science would look like, and why it is superior to results-centric science. Definitely worth a read, if for no other reason than to enjoy this banger:

“Viewing results as the epistemic target of science and the principal unit of progress underlies problematic patterns in scientific culture: Unrealistic expectation that each study should generate new scientific facts; overreliance on exclusively post hoc use of ad hoc statistics and their misuse as sole arbiters of scientific truths; desire to make scientific discoveries without systematic exploration; loss of long-term, iterative assessment of scientific progress; and wishful thinking that fixing a set of results reported in the literature will improve science” (Devezer & Bukbas, 2023, pp. 189-190).

Amen.