Addressing the theory crisis in psychology

Klaus Oberauer, Stephan Lewandowsky

Research output: Contribution to journalReview article

Abstract

A worrying number of psychological findings are not replicable. Diagnoses of the causes of this “replication crisis,” and recommendations to address it, have nearly exclusively focused on methods of data collection, analysis, and reporting. We argue that a further cause of poor replicability is the often weak logical link between theories and their empirical tests. We propose a distinction between discovery-oriented and theory-testing research. In discovery-oriented research, theories do not strongly imply hypotheses by which they can be tested, but rather define a search space for the discovery of effects that would support them. Failures to find these effects do not question the theory. This endeavor necessarily engenders a high risk of Type I errors—that is, publication of findings that will not replicate. Theory-testing research, by contrast, relies on theories that strongly imply hypotheses, such that disconfirmation of the hypothesis provides evidence against the theory. Theory-testing research engenders a smaller risk of Type I errors. A strong link between theories and hypotheses is best achieved by formalizing theories as computational models. We critically revisit recommendations for addressing the “replication crisis,” including the proposal to distinguish exploratory from confirmatory research, and the preregistration of hypotheses and analysis plans.

Original languageEnglish
Pages (from-to)1596-1618
Number of pages23
JournalPsychonomic Bulletin and Review
Volume26
Issue number5
DOIs
Publication statusPublished - 1 Oct 2019

Fingerprint

Psychology
Research
Publications
Research Design
Testing

Cite this

Oberauer, Klaus ; Lewandowsky, Stephan. / Addressing the theory crisis in psychology. In: Psychonomic Bulletin and Review. 2019 ; Vol. 26, No. 5. pp. 1596-1618.
@article{90d5b78ae37847f7b990f745b9b88494,
title = "Addressing the theory crisis in psychology",
abstract = "A worrying number of psychological findings are not replicable. Diagnoses of the causes of this “replication crisis,” and recommendations to address it, have nearly exclusively focused on methods of data collection, analysis, and reporting. We argue that a further cause of poor replicability is the often weak logical link between theories and their empirical tests. We propose a distinction between discovery-oriented and theory-testing research. In discovery-oriented research, theories do not strongly imply hypotheses by which they can be tested, but rather define a search space for the discovery of effects that would support them. Failures to find these effects do not question the theory. This endeavor necessarily engenders a high risk of Type I errors—that is, publication of findings that will not replicate. Theory-testing research, by contrast, relies on theories that strongly imply hypotheses, such that disconfirmation of the hypothesis provides evidence against the theory. Theory-testing research engenders a smaller risk of Type I errors. A strong link between theories and hypotheses is best achieved by formalizing theories as computational models. We critically revisit recommendations for addressing the “replication crisis,” including the proposal to distinguish exploratory from confirmatory research, and the preregistration of hypotheses and analysis plans.",
keywords = "Computational modeling, Hypothesis testing, Preregistration, Replication, Scientific inference",
author = "Klaus Oberauer and Stephan Lewandowsky",
year = "2019",
month = "10",
day = "1",
doi = "10.3758/s13423-019-01645-2",
language = "English",
volume = "26",
pages = "1596--1618",
journal = "Psychonomic Bulletin & Review",
issn = "1069-9384",
publisher = "Springer",
number = "5",

}

Addressing the theory crisis in psychology. / Oberauer, Klaus; Lewandowsky, Stephan.

In: Psychonomic Bulletin and Review, Vol. 26, No. 5, 01.10.2019, p. 1596-1618.

Research output: Contribution to journalReview article

TY - JOUR

T1 - Addressing the theory crisis in psychology

AU - Oberauer, Klaus

AU - Lewandowsky, Stephan

PY - 2019/10/1

Y1 - 2019/10/1

N2 - A worrying number of psychological findings are not replicable. Diagnoses of the causes of this “replication crisis,” and recommendations to address it, have nearly exclusively focused on methods of data collection, analysis, and reporting. We argue that a further cause of poor replicability is the often weak logical link between theories and their empirical tests. We propose a distinction between discovery-oriented and theory-testing research. In discovery-oriented research, theories do not strongly imply hypotheses by which they can be tested, but rather define a search space for the discovery of effects that would support them. Failures to find these effects do not question the theory. This endeavor necessarily engenders a high risk of Type I errors—that is, publication of findings that will not replicate. Theory-testing research, by contrast, relies on theories that strongly imply hypotheses, such that disconfirmation of the hypothesis provides evidence against the theory. Theory-testing research engenders a smaller risk of Type I errors. A strong link between theories and hypotheses is best achieved by formalizing theories as computational models. We critically revisit recommendations for addressing the “replication crisis,” including the proposal to distinguish exploratory from confirmatory research, and the preregistration of hypotheses and analysis plans.

AB - A worrying number of psychological findings are not replicable. Diagnoses of the causes of this “replication crisis,” and recommendations to address it, have nearly exclusively focused on methods of data collection, analysis, and reporting. We argue that a further cause of poor replicability is the often weak logical link between theories and their empirical tests. We propose a distinction between discovery-oriented and theory-testing research. In discovery-oriented research, theories do not strongly imply hypotheses by which they can be tested, but rather define a search space for the discovery of effects that would support them. Failures to find these effects do not question the theory. This endeavor necessarily engenders a high risk of Type I errors—that is, publication of findings that will not replicate. Theory-testing research, by contrast, relies on theories that strongly imply hypotheses, such that disconfirmation of the hypothesis provides evidence against the theory. Theory-testing research engenders a smaller risk of Type I errors. A strong link between theories and hypotheses is best achieved by formalizing theories as computational models. We critically revisit recommendations for addressing the “replication crisis,” including the proposal to distinguish exploratory from confirmatory research, and the preregistration of hypotheses and analysis plans.

KW - Computational modeling

KW - Hypothesis testing

KW - Preregistration

KW - Replication

KW - Scientific inference

UR - http://www.scopus.com/inward/record.url?scp=85073584937&partnerID=8YFLogxK

U2 - 10.3758/s13423-019-01645-2

DO - 10.3758/s13423-019-01645-2

M3 - Review article

VL - 26

SP - 1596

EP - 1618

JO - Psychonomic Bulletin & Review

JF - Psychonomic Bulletin & Review

SN - 1069-9384

IS - 5

ER -