Decoding Polarization: Classification Algorithm for Politically Charged Visuals

• In Event: The Role of Emotions in Political Communication

Abstract

Visual information surrounds us in day-to-day life, it also impacts the way how we interpret social and political world. Increasingly, research in public policy turns to the questions that require analyzing how visual communication is presented and received. Rapid growth in accessibility of computational resources allows us to involve more computational tools, but often such involvement still needs supervision by humans (people still need to look at the images and code them manually prior to performing data processing and data analysis) which can be costly, both in terms of time and money (recruitment and training of coders or validating coders’ inputs). Our project aims to propose an algorithm – a set of step-by-step instructions – of how to automatize and as a result optimize the process of visual information processing, particularly when it comes to visual data that contains political and often polarizing topics. The computational core of such algorithm is comprised of training and testing a classification model that will automatically identify whether an image elicits a polarized reaction. We utilize already collected data to demonstrate the performance of such an algorithm, but it can be adapted to other data as well, streamlining this process to answer any important question in a cheaper and faster way.

This project is motivated by three streams of research in the overlap of public policy and behavioral studies. First, do all people look at visuals and perceive visual information in the same way? It has been established that while perceiving visuals, the audience cannot be truly politically neutral (Gordon, 2004; Hummel, Thiemann and Lulcheva, 2008). Second, if we state that people evaluate visual information differently, that is likely largely because they pay attention to different parts of visuals (i.e. particularly, they may be getting triggered by different parts/different elements of the image). Finally, what can be included in the set of these triggering elements? This is essentially a question of what the minimal meaningful level of visual information is. While behavioral studies and experimental research often work with the entire image (using it as an assigned treatment) (Rock, 1983, Bar, 2004; Norman, 2002; Druckman, Peterson and Slothuus, 2013;), computational studies mostly concentrate on the individual objects (object recognition and image classification research) (Lempitsky and Zisserman, 2010; Shih, 2010; Joo et al., 2014; Torres, 2019; Williams, Casas and Wilkerson 2020, Torres and Cantu, 2022). But it is possible that the minimal level of meaningful information for a perceiver is a scene which in some cases can be reduced to a single object, but most likely will include multiple objects and elements, all together combined in what we call – points of composition (POC).

In this project, we work with visual data on a specific polarizing issue, immigration, and seek to introduce a new approach for examining visual data in a political context. First, we empirically identify meaningful elements of visuals – points of composition – on a set of images that were collected in our previous research. Second, we examine the way information recipients fixate on them and, as a result of these fixations, how they evaluate the entire image. In short, this allows us to derive the presence of the most triggering scenes in a given image. Finally, with this information, the ultimate goal is to build a computational algorithm that will automatically identify the polarizing nature of any given image. Even though we build the empirical work based on visuals about the specific topic of immigration, the algorithm is presumed to be generalizable for other contexts.

This project contributes to the studies of visual information strategically supplied by politicians, policymakers, or media outlets as well as to the growing research on political and affective polarization.

Authors

• Olga Gasparyan, Florida State University

• Elena Sirotkina, University of North Carolina at Chapel Hill