Multiple Imputation for Large Multidimensional Data with Linear Constraints

• In Event: Advances and Challenges in the Measurement of Latent Traits

Abstract

The use of multiple imputation for missing data in empirical studies has become increasingly popular in recent years. However, existing multiple imputation methods encounter significant challenges when applied to large, hierarchical, multidimensional datasets, especially those with linear aggregation constraints. This paper introduces a novel multiple imputation method specifically designed to address these challenges. Our method utilizes singular multivariate normal distributions within an Expectation Maximization algorithm, combined with a Parallel-Sequential Imputation scheme, to effectively handle large and complex datasets that include linear aggregation constraints. Testing on real datasets demonstrates that our method achieves up to twice the accuracy and is an order of magnitude faster than leading alternative methods. We apply our method to estimate a panel data model of average weekly wages, demonstrating that our method produces estimates that are unbiased and as efficient as estimates based on datasets with no missing values.

Author

• Jian Cao, Trinity College, Dublin