Outline

• Introduction
• Motivation
• Related Works
• Methodology
• Dataset
• Experiment Set-Ups
• Results
• Conclusion and Future Work

Social media and 'X' aka Twitter

• Social media has been one of the most influential features in the internet era
• Integral part of lives, to communicate with people, information and news
• X (formerly known as Twitter) is a text based social media platform
• Over 500 million monthly active users worldwide use X
• Users can post tweets (text posts), like, comment and share content of other people.

What are Bots?

• Bots are automated programs and can be controlled by algorithms to post certain content or behave in a certain way.
• Initial uses of social media bots involved posting weather updates of a local area by automated accounts.
• As technology grew, bots were used for customer service, subtasks on X or even virtual assistants.
• Bots generate between 20.8% and 29.2% of the content posted to Twitter [1] and huge revenue through automated ads
• Have also been used maliciously throughout its history
• Spam Bots are used to post large volume of tweets to influence people in a certain way

An Example of a bot generated tweet vs that of a real user.

Usage of Bot Accounts

• Good useful cases of bot accounts (WeatherBot, Customer Query)
• Malicious purposes of bot usage
  • Spreading misinformation (Spam Bots)
  • Cyberbullying
  • Scams, Frauds
  • Cyber attacks
• During the COVID 19 pandemic, more than 40% of the accounts tweeting about opening were bots. [2]
• In context of the 2018 US Midterms, [3] classified 21.1 percent of the accounts as bots, which in turn generated 30.6 percent of the tweets
• Scam bot accounts have stolen over $90K from deceived users according to [4]

Results of Botometer [5] application when given a few user ids as input

Previous Work

• Profile properties:
  • follower/following count, other metadata
  • ML algorithms for classification
• Tweets:
  • NLP to process tweets by accounts
  • ML and Deep Learning for classification
• Graph Structure:
  • Leveraging graph structure to obtain features
  • Using GNNs to directly work on graphs

Prior Work Summary

Graph Neural Networks

• Used for deep node embeddings using structural as well as semantic information
• Message passing
  • Each node transmits its information to neighboring nodes
  • Information received from other nodes is aggregated and updated its own information
  • This process goes on iteratively for each node to learn more information
• Information can include node as well as edge information depending on the type of GNN algorithm

Different GNN algorithms

• Graph Convolutional Neural Networks (GCN) [11]
• Relational Graph Convolutional Neural Networks (RGCN) [12]
• GraphSAGE [13]
• Graph Attention Networks (GAT) [14]

All GNN algorithms make use of a feature vector to process node and edge information.

GCNs and RGCNs

• Based off of CNN which work on grid-data
• Has convolutions and filters of sizes to process particular nodes together
• Helps identify localized patterns with filters
• Can be robust because of variable filter size

RGCNs are extended versions meaning they can process different edge types along with nodes Edge information is also carried along during message passing

GraphSAGE and GAT

• GraphSAGE uses sampling and aggregation, meaning starts with a small subset for learning, and keeps expanding to include more node information
• Nodes are sampled i.e some selected for aggregation
• Flexibility in selecting aggregating function
• Leads to inclusion of local and global properties

• GAT uses attention mechanism, by assigning importances to neighboring node features and iteratively updating them through learning node information
• Can start learning at multiple random points (attention heads) simultaneously, facilitating independent learning

GNN Approaches Summary

Natural Language Processing

• RoBERTa: A Robustly Optimized BERT Pretraining Approach [15]
• Extended Pretraining and Data Scale:
  • RoBERTa extends BERT's pretraining time and uses ten times more data
  • 160GB of uncompressed text compared to 16GB used to train BERT
• Optimized Training Techniques:
  • Eliminates the Next Sentence Prediction (NSP) task,
  • Focuses on more dynamic masking of input tokens, and
  • Uses larger mini-batches
• Dimension Size:
  • 768 for the base models (RoBERTa-base)

Process Flowchart:
Multirelational Twitter Bot Detection using Graph Neural Networks

GitHub:

Dataset

Feature Vector

• For us, these will be tuples of data associated with a node in a graph (such a node corresponds to a user).
• Four types of information processed to obtain feature vectors:
  • User bio using NLP technique RoBERTa
  • Numerical properties of user profile like follower count, following count, number of active days, number of tweets, screen name length
  • Categorical properties of user profile like does account have a profile image, is the account verified, is the account protected
• This information is concatenated in the initial layer of the architecture to be used as GNN features.

Features Extracted from Dataset

GNN models

• Used pyTorch Geometric neural networks library for GNN models
• Defined each model as a combination of layers that processed and combined the various input features along with the GNN model
• Used the following hyper parameters for training through tuning:
  • embedding size = 32
  • dropout = 0.1
  • learning rate = 0.01
  • weight decay = 0.05
  • loss = Cross Entropy Loss
  • activation = ReLu

Evaluation Metrics

• Accuracy: Accuracy measures the proportion of correct predictions out of all predictions made.
  $Accuracy\; \; :=\; \frac{\text{Number of correctly classified instances}}{\text{Total Number of Instances}}$
• Precision: Precision measures the proportion of true positive predictions out of all positive
  predictions made.
  $Precision\; \; :=\; \frac{\text{True Positives}}{\text{True Positives + False Positives}}$
• Recall: Recall (or Sensitivity) measures the proportion of true positive predictions out of all actual positive cases.
  $Recall\; \; :=\; \frac{\text{True Positives}}{\text{True Positives + False Negatives}}$
• F1 Score: The F1 Score is the harmonic mean of precision and recall, balancing the two metrics
  $F_1-Score\; := \; 2\; \times \frac{\text{Precision}\times \text{Recall}}{\text{Precision + Recall}}$

Accuracy Results

Takeaways: GraphSAGE performs similar on all relations GAT for interaction relation is the best followed by RGCN for follower + following

Precision Results

Takeaways: GraphSAGE performs similar on all relations GAT for interaction relation is the best followed by RGCN for follower + following + interaction

Recall Results

Takeaways: GAT for follower + following + interaction relation is the best followed by RGCN for follower + following + interaction

F₁-Score Results

Takeaways: GraphSAGE performs similar on all relations GAT for interaction relation is the best followed by RGCN for follower + following

Conclusion

• Twitter bots can be used for good and malice, identifying bots is important.
• Various techniques can be used to identify bots:
  • User profile details and Machine Learning
  • Natural Language Processing
  • User relations and Graph Neural Networks
• Leveraged GNNs to understand complex relationships of the account social network.
• Combined text based data, user account details and graphs to develop strong classification models.
• Used accuracy, precision, recall and F1-score to evaluate the models
• Graph Attention Network model with interaction relation constructed by combining the post and like relation gave the highest accuracy of 73.83

Future Work

• Processing account tweets using NLP techniques and using them as features
• Leveraging Heterogeneous GCNs to understand the relationship between node accounts and tweets
• Using weighted edge graphs between users by determining strength of relations
• Sub-classification of bot accounts into categories