Exploring the political pulse on Telegram

How does offline public perception compare with discussion on telegram groups, regarding the government's handling of specific issues in the country?

Overall, this research question requires a mixed-methods approach, combining data collected from online sources (Telegram groups) with data collected from offline sources (surveys). The analysis of both datasets helped determine the sentiment towards specific political topics and compare online and offline opinions.

Motivation

Unprecedentedly high ratings of a democratic leader! But what does the public really feel about the situation in the country? Are people freer with opinions when anonymous? These are the questions we wanted to find answers to. We wanted to look at what the public and political perception on and off social media was towards key issues of the country.

Why choose telegram as the social media platform to analyze? Telegram is a popular messaging app and a well-liked messaging service. It has over 70.48 million users in India, making it a potential source of valuable data. Telegram being an end-to-end encrypted service, is known for its security and privacy features. Political organizations and activists now frequently use Telegram to organize and mobilize their supporters.

We joined relevant (public) groups to collect data. It allows export of all historical messages in the group, making it suitable for study in a project of limited time frame and scope.

Analysis and results

Overview: The goal of the project was to conduct a comprehensive analysis of politically affiliated Telegram groups, with the aim of identifying the topics being discussed and extracting sentiment towards those topics. In order to compare attitudes between online and offline, the project also included the release of a survey to elicit offline opinions on the same subjects. The analysis of the Telegram groups involved collecting and clustering text message data. The subjects and sentiment contained in the text data have been identified and classified using machine learning and natural language processing techniques.

Data collection: Exported data from 20 official and unofficial telegram groups over time period of 2017-2023. Parties include BJP, AAP and Congress. Languages are English and Hindi.

Topic Modeling: Used BERTopic to cluster the aggregate message data and used multilingual sentence embeddings (paraphrase-multilingual-MiniLM-L12-v2) to process Hindi messages.

About BERTopic: it is a topic modeling technique that leverages BERT embeddings and a class-based TF-IDF to create dense clusters allowing for easily interpretable topics whilst keeping important words in the topic descriptions.

Main topics found were regarding repealed farmers' laws, Coronavirus situation and response to China.

=> Temporal topic modeling

2018: bank_delhi; evms; petrol oil price

2021: covid cases; farmers protest; vaccination; hospital beds; unity muslim religion

2022: punjab; ukraine war; education schools

=> Sentiment analysis of aggregate data:

Used zero-shot-classifiier to classify with respect to labels of positive, negative and neutral. The model is Multilingual mDeBERTa-v3-base-mnli-xnli. This multilingual model can perform natural language inference (NLI) on 100 languages and is therefore also suitable for multilingual zero-shot classification.

China	Farmers	Covid

Survey: Surveyed 65 people on above topics: 60% male, 38% female and 2% other.

Form: Link to open responses

Survey sentiments:

Covid	Male Covid	Female Covid

Farmers	Male Farmers	Female Farmers

China	Male China	Female China

Sentiment analysis of groupwise Telegram data:

BJP covid: 80% of negative sentiment and 20% positive.

BJP farmers: 50% negative and 50% positive.

Congress covid: 60% negative , 5% neutral and 35% positive

Congress farmers: 100% negative

Other groups covid: 80% negative, 5% neutral and 15% positive

Other groups farmers: 45% negative, 5% neutral and 50% positive

To show correlation between survey questions and text messages cosine similarity is used.

Cosine similarity:

Embeddings of each sentence of a topic and survey questions is obtained. Then the two embeddings are passed into the cosine function and an array is returned which contains cosine similarity values. For example, first column contains cosine similarity between first survey question and all messages of a particular topic.

Conclusion:

Overall, the survey results were much more negative than telegram results for all topics. This is expected due to the bias of politically affiliated groups promoting an agenda. We could also argue than people are more freely critical when anonymous, and the general public part of the telegram groups feel obligated to express a particular opinion, or fall in with the group's overall opinion.

We must also take into account the sampling bias (of the survey) with regard to age and economic position, which is uniform due to limited accessibilty.

The temporal variation in topics is evidence that the groups discuss affairs relevant to the period.

Future work:

An interesting way to take the project forward is to expand the analysis to other social media platforms. While Telegram can provide valuable insights, it is just one of many social media platforms used by politically affiliated groups. The concept of the project, comparing perceptions on and off social media towards political agendas, is generalisable. To gain a more comprehensive understanding of political discourse and sentiment, other platforms such as Twitter, Facebook, or Reddit can be used.

Researchers could also analyze the impact of current events. Political discourse and sentiment can be influenced by current events such as elections, social movements, or international crises. Analyzing how politically affiliated social media groups respond to these events could provide valuable insights into the relationship between online and offline political behavior.