Constructing Machine Learning Index: Cases of Populism and Nationalism

• In Event: Advances and Challenges in the Measurement of Latent Traits

Abstract

It is cumbersome and costly to manually construct indices for complex political concepts such as democracy, nationalism, or populism. A single researcher cannot accomplish the task alone. Moreover, the aggregation equation that links indicators needs to undergo rigorous tests to justify its suitability. Nevertheless, many rising and young scholars are invested in comparative studies of important political phenomena such as populism and nationalism, conceptualizing them using indices, and comparing them across regions within a country or across countries. This paper compares regression based white-box machine learning algorithms and black-box algorithms, such as random forest and support vector machines, to illustrate how to select variables and build an index for nationalism using machine learning algorithms. I evaluate these algorithms based on their predictive power and interpretability, which assesses how well the model contributes to theory-building. The paper also replicates Di Cocco and Monechi’s populism score with different machine learning algorithms and compares their performances. The paper also lays out a procedure and considerations for applied researchers to build machine learning indices of social and political concepts. Finally, the paper generalizes these findings to assist scholars construct indices for other complex concepts.

Author

• Peitong Jing, University of Notre Dame