Confidence in RDDs: Change Detection through Bayesian Change Point Analysis

• In Event: Robustness and Sensitivity to Modeling Assumptions

Abstract

Regression Discontinuity Designs (RDD) are widely popular in the social sciences as they can recover local average treatment effects under a few assumptions. However, concerns about natural variation and noise in the data have led to skepticism of their results. We propose using Change Point Analysis (CPA) as a tool within the RDD framework to explore variation in the running variable and increase the credibility of RDD findings. By incorporating CPA, researchers may efficiently identify and evaluate discontinuities in the raw data without the need for pre-specification. The ability of CPA to correctly identify the theory-driven discontinuity should lend credibility to the design, while finding other discontinuities may signal noisy data or other threats to identification. We demonstrate the robustness of CPA in detecting meaningful breakpoints through empirical applications and simulations. We argue that CPA assists in visualizing data and validating changes against theoretical expectations. We leave practitioners with a sequential workflow to aid their research using RDDs.

Author

• Bryant James Moy, New York University