Tackling Acquiescence Bias in Opinion Surveys: Efficacy of Bipolar-Item Scales

• In Event: Measuring Public Opinion

Abstract

When respondents answer unipolar questions in Agree/Disagree ideological scales, they often agree with statements regardless of content, which is known as the acquiescence bias. This study proposes that such bias can be effectively reduced by bipolar-item scales. We employed simulations, randomized survey experiments, and an expanded item response theory (IRT) model on a large-scale online survey of Chinese citizen’s ideology. The results show that bipolar-item scales reflect the public ideological preferences more accurately than unipolar scales do, demonstrating greater predictive accuracy of true preferences and related variables. Moreover, we highlight the difficulty in converting responses between these two types of scale by statistical methods. Our findings emphasize the need for careful scale selection in survey, particularly in contexts prone to acquiescence bias, and offer insights for political scientists and practitioners aiming to conduct more accurate surveys on ideology.

Authors

• Ziyi Chen, Tsinghua University

• Yiqing Xu, Stanford University