Bias identification and mitigation in the social media ecosystem has been lately researched towards achieving a more efficient utilization of social media platforms for different stakeholders and purposes. Among these stakeholders, intelligence services worldwide, collectively called the Intelligence Community (IC), tend to use social media, supplementarily to their pre-extant disciplines, for monitoring areas of interest and identifying emerging social, political and security trends/threats. Over time, the IC has identified bias as the major impediment in information analysis, thus it has developed scientific and empirical methods for bias mitigation, in parallel to those developed by the information and communication technology (ICT) and artificial intelligence (AI) community. As it becomes apparent, it is to both communities’ interest to accurately trace bias and ideally eradicate or moderate its effects. This paper is an extension of a previously presented academic work, in which we drew systemic parallels between Intelligence Analysis (IA) and Twitter Analytics (TA), comparatively examined existing bias mitigating methodologies to identify similarities/dissimilarities, and subsequently investigated the viability of adopting and attuning methodologies from the first field to the latter. Furthermore, we proposed a novel framework for AI-augmented bias mitigation in the IC and simultaneously recommended on a theoretical level, methods and tools, already adapted by the ICT community, for efficiently supporting bias mitigation in each phase of the aforementioned framework. In the current paper, we extend our previous work by implementing the collection phase of the proposed framework on a real-world use case utilizing Telegram as a collection platform. We contribute new insights resulted from our experimentation with a tri-modal source selection approach in which human agents and Large Language Models (LLMs) are involved. The experiments were performed with data collected using one of the correspondingly suggested tools, engineering an equally represented, balanced dataset for the working case. © The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd. 2024.
