Star wars: The empirics strike back

Journals favor rejections of the null hypothesis. This selection upon results may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the AER, JPE and QJE, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a camel shape with abundant p-values above :25, a valley between :25 and :10 and a bump slightly under :05. Missing tests are those which would have been accepted but close to being rejected (p-values between :25 and :10). We show that this pattern corresponds to a shift in the distribution of p-values: between 10% and 20% of marginally rejected tests are misallocated. Our interpretation is that researchers might be tempted to inflate the value of their tests by choosing the specification that provides the highest statistics. Note that Inflation is larger in articles where stars are used in order to highlight statistical significance and lower in articles with theoretical models.

Data and Resources

Additional Info

Field Value
Source https://shs.hal.science/halshs-00710122
Author Brodeur, Abel, Lé, Mathias, Sangnier, Marc, Zylberberg, Yanos
Maintainer CCSD
Last Updated May 15, 2026, 15:14 (UTC)
Created May 15, 2026, 15:14 (UTC)
Identifier halshs-00710122
Language en
Rights https://about.hal.science/hal-authorisation-v1/
contributor Paris-Jourdan Sciences Economiques (PSE) ; École normale supérieure - Paris (ENS-PSL) ; Université Paris Sciences et Lettres (PSL)-Université Paris Sciences et Lettres (PSL)-Institut National de la Recherche Agronomique (INRA)-École des hautes études en sciences sociales (EHESS)-École nationale des ponts et chaussées (ENPC)-Centre National de la Recherche Scientifique (CNRS)
creator Brodeur, Abel
date 2012-06-15T00:00:00
harvest_object_id 4e35ea1a-660f-4685-9785-f4d7902aaad5
harvest_source_id 3374d638-d20b-4672-ba96-a23232d55657
harvest_source_title test moissonnage SELUNE
metadata_modified 2025-10-01T00:00:00
relation https://amu.hal.science/hal-01447851v1
set_spec type:UNDEFINED