Assessing the Robustness of Post-estimation Quantities of Interest

• In Event: Robustness and Sensitivity to Modeling Assumptions

Abstract

Applied research in political science often produces post-estimation quantities of interest, such as marginal effects or predicted probabilities, to report the impact of a key variable of interest. An important question is the robustness of these results to changes in the underlying data, which is closely connected to the key concept of external validity.

We outline a computationally efficient framework for doing so that can be applied to generalized linear models, models with random effects, and/or clustered standard errors. Using our approach, researchers can estimate their model once and then can quickly approximate the impact of techniques such as bootstrapping and leave-one-out cross-validation without re-estimating the model. Additionally, one can approximate the impact of the "worst case" scenario of the most influential observations being deleted.

Methodologically, we build upon recent work in econometrics to approximate the impact of removing a set of observations and extend their technique to cover common quantities of interest used in political science. We derive analytical expressions for the approximate impact of deleting a set of observations for generalized linear models, random effects models, and common types of robust standard errors used in political science.

We will use simulation studies to examine when our approximate method works well and also explore a number of existing studies. We suspect that, across many types of studies, small perturbations of the underlying data can lead to considerably different conclusions.

Authors

• Ian Delabie, University of Pittsburgh

• Max Goplerud, University of Texas at Austin