Beyond Human Subjects: LLMs as Participants in Political Experimentation

• In Event: Novel Applications of Machine Learning

Abstract

We introduce an innovative framework that utilizes large language models (LLMs) to simulate political experiments. Our approach extends the groundwork established by Argyle et al. (2023), progressing from merely forecasting AI-generated survey responses to fully simulating political lab experiments. We demonstrate the efficacy of this methodology by replicating two distinct experiments from a seminal political science study by Terris and Tykocinski (2014), focusing on the concept of inaction inertia in international negotiations. Our simulations employ `silicon samples' generated by LLMs to create a detailed imitation of human behavior in controlled settings. The sophisticated relationships between real-world variables are closely re-generated by LLM-powered behavioral responses, underscoring the potential of LLMs to model complex human dynamics in political scenarios. This framework opens avenues for conducting artificial group experiments as pilot studies, exemplified by our use of LLMs for power analysis.

Author

• Selim Yaman, American University