Meta-analysis to integrate effect sizes within an article: Possible misuse and Type I error inflation

Taiji Ueno, Greta M Fastrich, Kou Murayama

Research output: Contribution to journalArticlepeer-review

40 Citations (Scopus)

Abstract

In recent years an increasing number of articles have employed meta-analysis to integrate effect sizes of researchers' own series of studies within a single article ("internal meta-analysis"). Although this approach has the obvious advantage of obtaining narrower confidence intervals, we show that it could inadvertently inflate false-positive rates if researchers are motivated to use internal meta-analysis in order to obtain a significant overall effect. Specifically, if one decides whether to stop or continue a further replication experiment depending on the significance of the results in an internal meta-analysis, false-positive rates would increase beyond the nominal level. We conducted a set of Monte Carlo simulations to demonstrate our argument, and provided a literature review to gauge awareness and prevalence of this issue. Furthermore, we made several recommendations when using internal meta-analysis to make a judgment on statistical significance.

Original languageEnglish
Pages (from-to)643-654
Number of pages12
JournalJournal of Experimental Psychology
Volume145
Issue number5
DOIs
Publication statusPublished - 1 May 2016
Externally publishedYes

Fingerprint

Dive into the research topics of 'Meta-analysis to integrate effect sizes within an article: Possible misuse and Type I error inflation'. Together they form a unique fingerprint.

Cite this