Why Do Expressed Policy Preferences Differ between Surveys and Focus Groups?

• In Event: Beyond the Mean: New Directions in Elite Behaviour and Political Representation

Abstract

Research on political inequality often assumes that socio-demographic groups differ in their policy preferences. Such preference gaps are the precondition for arguments about unequal responsiveness (Elsässer and Schäfer 2023). Also research on elite misperception of public opinion uses preference gaps between groups as reference points for assessing the selective (in)accuracy of politicians’ estimates (Pilet et al. 2023). A common methodological approach in these literatures is survey-based preference measurement. However, political psychology suggests that weak and ambivalent policy preferences are widespread, casting doubt on the reliability of closed-survey items in capturing preferences.

To evaluate the validity of typical preference-gap measures, this paper compares individuals' positions across two contexts: surveys and focus groups. Preliminary research indicates significant differences in expressed positions between these settings (Fastenrath and Marx 2024; Lindsey 2023). Understanding the factors driving these differences is crucial, given that politicians often rely heavily on face-to-face interactions with voters to gauge public opinion (Henderson et al. 2023). Any biases introduced in such social settings could have substantial implications for representation.

The study is based on 20 focus groups with 5-6 participants each. The groups differ socio-demographically, but are internally homogenous. Two weeks prior to group discussions, participants receive a survey with preference items on three policy areas: redistributive policy, environmental policy, and gender equality. The focus groups are structured by the same thematic blocs. Independent coding of positions expressed in focus groups allows within-person comparisons of preference measured by both methods.

Additionally, the paper aims at understanding why positions may diverge and which method provides a more accurate image of participants’ authentic preferences. To this end, we administer an additional survey immediately after the group discussions. This elucidates if people update their preferences in the group discussions or whether they strategically misrepresent their positions in the social setting (and revert to their initial position in an anonymous survey). Participants are also asked directly in which setting they felt more comfortable expressing their "true" preferences. Finally, the study considers group-level factors such as rapport, homogeneity, and dominant positions as explanatory factors for preference change. This is complemented with a qualitative assessment of conversation dynamics that might reveal patterns of social desirability bias.

Authors

• Paul Marx, University of Bonn

• Alice Barth, University of Bonn