Using Natural Language Processing to Analyse the Current State of Political Discourse on Twitter

Abstract

Twitter is a major public social media platform with millions of daily users. The Twitter app makes it very easy to post short messages, with the flexibility of adding hashtags to highlight key topics. Consequently, through the Twitter API a user’s tweets, retweets, like count, comments, followers, and other aspects can be flexibly mined for data insights. Once extracted, textual data can be analyzed using various Machine Learning methods. Indeed, through processing and dissecting text, machine-based methods can identify linguistic patterns within this text. Natural Language Processing (NLP) is typically used to analyze various high-level categories that surround textual data such as the topic, the emotion, and the sentiment. Current literature identifies that twitter data is highly useful for analyzing the political sphere. Given the current polarization in the United States, Twitter data, combined with these methods, could help us understand areas of commonality. Indeed, through analyzing large amounts of both democratic and republican sentiments and emotions towards certain topics, it may be possible to determine similarities between the two and allow for some bipartisanship. In this analysis, the Twitter API was used to collect approximately 200 tweets per elected official from the House of Representatives, which were then cleaned and stored in a database. Next, these tweets were dissected through tokenizing. Following this, the emotion and topic of the tweet, as well as sentiment scores for the tweets were determined. Using Streamlit, data and graphs were visually displayed on a webapp. Results display that both gun violence and human rights have democrats' and republicans’ majority negative sentiment. These may be possible areas of cooperation and communication between the two parties to create policies for change.