Comparing Verbal and Non-verbal Emotions in Parliamentary Speeches

• In Event: Understanding Legislative Debates

Abstract

This study aims to demonstrate how to unmask the parliamentary information that does not remain in the written records. Many researchers use state-of-the-art scaling methods to extract latent dimensions in written parliamentary records. Although the minutes provide a basis for understanding what legislators discuss in parliament, they do not tell us the whole story. The parliamentary communication spaces are multi-dimensional. While non-verbal communication attracts scholarly attention, online videos of speech and debate at many parliaments remain substantially understudied.
We have launched an internet video retrieval system for the Japanese Diet to seek innovative ways to provide universal access to parliamentary information. Using the speech recognition system that automatically transcribes parliamentary speeches in plenary and committee meetings, our video retrieval system creates timestamp data to link parliamentary video feeds with the minutes of proceedings and partially plays the parliamentary video clips corresponding to the minutes through keyword search. Also, we have developed an integrated interface to detect emotions from text and video records by taking advantage of our video retrieval system. This study proposes alternative ways to analyze parliamentary discussion by comparing the emotions seen in speech and facial expressions.
A limitation of textual analysis that extracts sentiment from words is that if the speaker uses the word sarcastically, it may send the opposite message from the word. However, such sarcastic emotions may appear in facial expressions, and this study aims to contribute to the emotion analysis by comparing verbal and facial expressions and examining whether verbal and facial emotions are the same or different. We also examine the importance of parliamentary communication by clarifying the effects of such differences or no differences on the speaker’s success in parliamentary and electoral activities.

Author

• Mikitaka Masuyama, National Graduate Institute for Policy Studies