Pre-registration and Strategic Private Experimentation

• In Event: Information Transmission

Abstract

We develop a model to explain pre-registration, the practice of empirical researchers describing significant parts of their planned analyses prior to any data collection. To do so, we consider a model in which a Sender attempts to persuade a Receiver of the truth of some hypothesized state of the world. The Sender can sequentially learn the outcome of noisy experiments, each of which are informative about the state of the world. The Sender's information acquisition is private and the Receiver does not observe the outcome of an experiment unless the Sender chooses to reveal it. This creates an opportunity for `p-hacking' where Sender may perform a large number of tests and reveal only those with favorable outcomes. We augment this private experimentation game with a very weak form of pre-registration with no enforcement and no Sender commitment power. Specifically, the Sender sends cheap talk messages about her intended experimentation plan prior to observing the outcomes of any experiments. We show that the private experimentation game with no pre-registration possesses many equilibria with very different levels of p-hacking. However, when this problem is most severe, the Sender's expected payoff is higher under equilibria that at least somewhat limit her own ability to p-hack. Finally, provided that the Sender and Receiver share a common language for experimentation plans, pre-registration tends to select the Sender-optimal equilibrium from the underlying private experimentation game.

Authors

• Zion Little, Washington University in St. Louis

• Jacob M. Montgomery, Washington University in St. Louis

• Keith E. Schnakenberg, Washington University