Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias? A new GLO Discussion Paper by GLO Fellow Abel Brodeur and colleagues.

Discussion Paper, News, Post, Post-22

A new GLO Discussion Paper finds that pre-registered studies that have a complete pre-analysis plan are significantly less p-hacked.

Abel Brodeur

GLO Discussion Paper No. 1147, 2022

Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias? – Download PDF
by Brodeur, Abel & Cook, Nikolai M. & Hartley, Jonathan S. & Heyes, Anthony

GLO Fellow Abel Brodeur

Author Abstract: Randomized controlled trials (RCTs) are increasingly prominent in economics, with pre-registration and pre-analysis plans (PAPs) promoted as important in ensuring the credibility of findings. We investigate whether these tools reduce the extent of p-hacking and publication bias by collecting and studying the universe of test statistics, 15,992 in total, from RCTs published in 15 leading economics journals from 2018 through 2021. In our primary analysis, we find no meaningful difference in the distribution of test statistics from pre-registered studies, compared to their non-pre-registered counterparts. However, pre-registered studies that have a complete PAP are significantly less p-hacked. These results point to the importance of PAPs, rather than pre-registration in itself, in ensuring credibility.

Featured image: Mika-Baumeister-on-Unsplash

JUST PUBLISHED
Vol. 35, Issue 4, October 2022: Journal of Population Economics (JOPE): 15 articles
https://link.springer.com/journal/148/volumes-and-issues/35-4
Just released: CiteScore of JOPE moves up from 3.9 (2020) to 6.5 (2021)! LINK
Similar, its Impact Factor is now 4.7 (2021) after 2.8 (2020)! LINK

GLO Discussion Papers are research and policy papers of the GLO Network which are widely circulated to encourage discussion. Provided in cooperation with EconStor, a service of the ZBW – Leibniz Information Centre for Economics, GLO Discussion Papers are among others listed in RePEc (see IDEAS,  EconPapers). Complete list of all GLO DPs – downloadable for free.

The Global Labor Organization (GLO) is an independent, non-partisan and non-governmental organization that functions as an international network and virtual platform to stimulate global research, debate and collaboration.

Ends;