Towards Exposing Coordinating Inauthentic Groups on Social Media

Abstract

Narratives can influence people on social media, and coordinating their dissemination can amplify their effects, which may result in polarisation between communities. Misinformation can exacerbate this polarisation by causing misunderstandings, potentially encouraging the formation of echo chambers and filter bubbles, further hampering dialogue. Deliberate coordinated inauthentic behaviour (CIB) is a core element of disinformation campaigns and Strategic Information Operations (SIOs) that exploit these online phenomena. CIB has been used for ideological and political reasons to pollute our information environment with biased narratives, misleading and false information and propaganda, intensifying existing societal divisions to the extent that it can threaten national security. Prior research has focused on detecting and classifying entire campaigns (e.g., spam) and individual social bots (automated accounts that deceive and influence by appearing human) and botnets. The damage that information disorders causes to society is also well established, with real-world effects such as vaccine hesitancy, increased conspiratorial thinking and even mass violence. We seek to detect the groups of accounts coordinating their behaviour as part of SIOs, appealing to and recruiting unwitting users to promote their propaganda. First, however, we need to understand the context that CIB occurs in, which we investigate via two avenues: the information environment and the communication environment. The information environment consists of commercially encumbered social media data. This presents challenges for research due to a lack of transparency, which causes a trust deficit in the results of social media analyses. Opaque sampling biases result in filtered social media data streams that produce variations in data with identical boundary criteria. We present a novel process to examine these variations and demonstrate the method via systematic case studies, finding significant flow-on effects on social network analyses. The communication environment is replete with contentious online discussions, which are particularly vulnerable to information disorders. We detect and characterise the communication strategies of two polarised groups in a temporally phased investigation of an Australian bushfire discussion, and observe the effects of the strategies. Then, in a longitudinal study, we explore how multiple polarised groups reappear and align in differently themed discussions, finding the polarisation largely remains though the discussion themes can overlap. With this knowledge, we present and demonstrate our novel network-based approach to detect coordinating groups, focusing on identifying accounts that appear to cooperate with anomalously high levels of coincidental behaviour, artificially raising the voices of the few above the many. The method is generalised, applicable to major platforms, and is amenable to near real-time applications, which are vital to counter influence campaigns before they take hold. Further, we extensively validate the method with several political Twitter datasets, introducing techniques to move beyond manual inspection, which has been the dominant approach in the literature. The research presented in this thesis provides a solid foundation for future investigation of CIB, online polarisation, and trust in social media data.