Stages of Affective Elite Polarization in Parliamentary Speeches

• In Event: Poster Session: Posters in Political Methodology

Abstract

This project develops a novel way to measure of the affective elite polarization. We use sentiment analysis of the existing corpus of parliamentary speeches and aim to create a time-series EU-wide dataset of affective polarization occurring in the last two decades. We employ pre-trained generative machine learning software that pinpoints elite attitudes towards different in- and out-groups, Then, these attitudes are combined into an index of affective polarization.
The affective polarization became a cornerstone of both party politics and regime change scholarship over the last few years. Scholars of party politics have shown that the excessive level of affective polarization among democratic adversaries and their partisans hampers the quality of the democratic process in various stages through different mechanisms Schedler 2023; Turkenburg and Janssen 2023; Druckman et al. 2020; Reiljan 2020; Iyengar et al. 2019; McCoy, Rahman, and Somer 2018; Iyengar, Sood, and Lelkes 2012). The literature on democratic breakdowns also noticed that before the democratic decay reveals itself, polarization among politicians grows (Wunsch and Blanchard 2023; Lührmann 2021; Levitsky and Ziblatt 2018; Bermeo 2016; Linz 1978) and they start to demonstrate anti-democratic traits such as publicly prioritizing their victory over the importance of democratic institutions or threatening their opponents and journalists.
At the same time, affective polarization among elites remains understudied because recent scholarship perceived the affect as a mass phenomenon while treating elites from an ideological or policy perspective ( see, for instance, Druckman et al. 2020; Iyengar et al. 2019; Banda and Cluverius 2018; Hetherington 2001). Also, this project would not have been feasible even a year ago. The sentiment or any kind of context analysis would require such an amount of training and manual coding, that a single project would rarely include more than one country (Abercrombie and Batista-Navarro 2020).
We overcome this issue by using the pre-trained generative AI, namely ChatGPT 4. It has several crucial advantages over the earlier supervised machine learning techniques, to say nothing of earlier methods of dictionary-based analysis. First of all, it does not require extensive training, made similar projects by computer science experts (Abercrombie and Batista-Navarro 2020). Second, it is capable of moving across language contexts and even does not require additional training when doing so. Third, as a result of our trials, among other versions of pre-trained gen AI, the ChatGPT4 has proven to be the only software capable of differentiating between important and non-important mentions of political figures in parliamentary speeches. Also, it can differentiate between emotions regarding the issue and emotions towards the opponent per se. It has also proven to be able to articulate in spoken language the reasons for its interpretations. Finally, in comparative studies of different tools for sentiment analysis, the pre-trained gen AI has shown even higher accuracy rates than the previous generations of neural language models trained for a specific task under study (Le Mens and Gallego 2023, Kheiri and Karimi 2023).
Thus, it now became technically possible to create a large cross-country dataset on emotions in parliamentary speeches. We use the ParlaMint dataset of European Parliamentary speeches and have at this stage started with the Hungarian case: first, it constitutes the case of democratic backsliding, second, it is the challenging instance of a non-Indoeuropean language. ChatGPT autonomously identifies instances where politicians criticize each other and then measures their attitude toward their counterparts. It does this not merely based on words but on context and articulates in spoken language the reasons for its interpretations. In addition, we retrieve the information on who is speaking, who is addressed, etc.
To enable this, we have developed an algorithm that enables artificial intelligence to extract and analyze information from parliamentary speeches. One R script extracts these speeches from existing XML databases and converts them into plain text. Another API R script with a custom-written set of functions then feeds them – together with a prompt that also gives context knowledge – speaker`s name and date – to ChatGPT one by one, automatically adding context: country, speaker's name, and year. The results are then compiled into a comprehensive table.
The most challenging part was crafting a prompt for ChatGPT that ensures near-perfect accuracy. During the talk, I will showcase the dataset of Hungarian speeches in the years 2010-2021, and will demonstrate the accuracy measurements. As of now, we have a pilot set of 525 mentions based on the 100 speeches, which also are the randomly selected 3% of Hungarian cases.

Authors

• Gennadii Iakovlev, European University Institute (EUI)

• Emil Kamalov, European University Institute