Analyzing Economic Views via Social Media Posts versus Survey Data

• In Event: Advances in Public Opinion Research Methods

Abstract

We measure public sentiment about the US economy using tweets from a random sample of survey respondents who provided their twitter IDs, as well as a sample of over 750,000 individuals matched from voter files to their twitter-ID. We weight the two samples to be representative of the US adult population. Using a supervised machine learning classifier, we measure the proportion of positive tweets each week from January 1, 2018, to December 31, 2022. We compare these Twitter-based measures of sentiment to Morning Consult’s daily survey-based measure of consumer sentiment over the same period. We test how highly correlated the measures are and how each responds to a number of objective indicators of economic performance. If social media and survey sentiment track each other closely, it opens up the possibility of using Twitter-based measures to study how economic sentiment responds to temporally fine-grained events, as well as how sentiment varies across different groups. If social media and survey sentiment differ, further analysis may help us to better understand what motivates social media users to post.

Each day, millions of Americans express public opinions via social media. Social media offers a way to observe opinion at fine-grained time intervals without resorting to expensive surveys. But if we are to measure sentiment from social media, we need to know how opinion expressed in social media compares to survey-based measures. Unlike opinions expressed in surveys, individuals posting on social media choose what to talk about and how. And unlike representative surveys, the opinions expressed on social media come from those motivated to share their views, rather than a random sample. Further, while we usually assume survey responses are truthful, social media posts are performative. Thus, there are ample reasons to expect the two measures to differ. Yet at base, we expect both to respond to available objective information.

Opinions about the economy expressed on twitter can also tell us what aspects of the economy people care about. In addition to providing information about a user's evaluation of the economy on a valence dimension, the text of the tweet can provide information about the aspect of the economy (e.g., inflation or unemployment) that they care most about. And they can provide instantaneous feedback about the effects of economic events: whether those events are changes in the stock market or reporting of new economic indicators by the federal government or more subtle changes happening in the economy. And with a large enough sample of tweets (or tweeters), we can examine how this varies by region or ethnicity or partisanship (conditional on being able to measure these characteristics of the tweeters).

In this paper we explore the possibility of using tweets to measure public sentiment about the economy. We first train a classifier on a large corpus of tweets collected via a keyword filter from 2014 thru 2023. To train the classifier we hand-label 30,000 tweets for whether they are: not relevant to the U.S. economy, relevant and positive, relevant and neutral, relevant and negative, relevant but not known if positive, neutral, or negative, and don't know if relevant. We then apply the classifier to tweets by 3029 YouGov respondents who have provided their twitter handles. This allows us to create a measure of Twitter based Economic Sentiment (TES). We compare this measure to a traditional survey based measure of economic sentiment to see if the measures are highly correlated, and move in similar manners. And we estimate time-series models of sentiment with both series to see if they respond to the same things.

In addition, we examine the specific topics mentioned in tweets about the economy. We particularly focus on mentions of inflation versus unemployment, and by examining tweets at different time periods corresponding to different presidential administrations, consider whether co-partisans of the president are more likely to tweet about positive economic news than partisans of the out-party. Our initial results on this are quite despositive. In 2022 both Democrats and Republicans were more likely to mention Inflation (the key economic issue of the period) than unemployment. But while Democrats were 7 times as likely to mention Inflation compared to unemployment; Republicans were 22 times more likely to mention inflation than unemployment. We note that this is consistent both with ``partisan cheerleading’’ and with partisan motivated reasoning (Republicans may have processed more negative news than Democrats). And we confirm that this is not a bias of Republicans to mention inflation over unemployment: in 2018 Republicans mentioned inflation only one-fourth as often as they mentioned unemployment (while Democrats mentioned inflation 1.67 times more than they mentioned unemployment).

Authors

• Suzanna Linn, Pennsylvania State University

• Patrick Y. Wu, American University

• Jason Greenfield, New York University

• Joshua A. Tucker, New York University

• Jonathan Nagler, New York University