Explainable Non-linear Models for Predicting Political Instability

• In Event: Novel Applications of Machine Learning

Abstract

Political scientists who traditionally focused on parameter estimation have recently put more emphasis on developing prediction models. On that front, advanced machine learning algorithms can be a powerful tool for making more accurate predictions. However, they have some shortcomings. First, it has been reported that simple linear models often outperform non-linear machine learning models in predictions. This is especially so when the sample size is small, which is very common in political science research. Small sample size often invites overfitting problems for machine learning models, resulting in low accuracy for out-of-sample predictions. Second, many advanced machine learning models, deep learning models as a typical example, are uninterpretable. The relation between the input and the output is “black-boxed,” even for model developers.
In this paper, we developed and propose non-linear forecasting models using advanced machine learning algorithms that can, compared to traditional linear forecasting models, more accurately predict forecasting results and are also explainable. We use the dataset of Goldstone et al. (2010) and Bowlsby et al. (2019) to show our non-linear models outperform Goldstone et al.’s linear logistic regression model in predicting political instability. To make the comparison clear cut, we used the same features (independent variables) as Goldstone et al.’s. We also show that our models are explainable; that is, the logical relationships between the input (features) and the output (the predictive result) are specified. In other words, unlike many sophisticated machine learning models, our models are not “black-boxed.”
We used a decision tree as our primary algorithm for our non-linear prediction models. Decision tree models are interpretable models that have proved effective in various prediction models, including political predictions. However, they are prone to overfitting problems. To mitigate the overfitting problem, we used ensemble techniques, namely, gradient boosting and random forest methods to combine a large number of decision trees for predictions.
We also used more advanced machine learning algorithms, such as deep neural networks, to make more accurate predictions. In these models, where input-output relations are black-boxed, we applied explainable AI (XAI) techniques to make our models interpretable. XAI is a rapidly developing research field in AI and machine learning. In this paper, we applied two of the most used XAI techniques, namely, LIME and SHAP.
Initial results are promising. Our non-linear prediction models have mostly outperformed Goldstone et al.’s model in sensitivity and specificity scores. Among them, ensembled decision tree models attained better scores. All the models are also interpretable.
Although our models’ applicability to other types of political data remains to be seen, we believe our models can serve as a basis for interpretable non-linear political prediction models.

Authors

• Budrul Ahsan, Philips Japan

• Sota Kato, The Tokyo Foundation for Policy Research

• Iku Yoshimoto, Univresity of Tokyo

• HIROSHI OKAMOTO, The University of Tokyo

• Takafumi Nakanishi, Musashino University