A Method for Generating Explanations of Offensive Memes Based on Multimodal Large Language Models

With the advancement of 5G technology, offensive speech has increasingly proliferated across social networks in the form of multimodal memes. Consequently, the detection and interpretive generation of offensive memes play a crucial role in enhancing content moderation effectiveness and maintaining a harmonious and healthy public discourse environment. Existing studies on the interpretive generation of offensive memes focus solely on the targets and content of offense, neglecting the societal background knowledge and metaphorical expressions embedded in memes. This oversight limits the ability to comprehensively and accurately interpret the meaning of offensive memes, thus constraining the applicability of interpretations. To address this challenge, we propose a method based on multimodal large language model for generating interpretations of offensive memes. By augmenting elements such as offense targets, the content of the offense, and metaphor recognition into the instruction tuning data, we can effectively improve the multimodal large model’s proficiency in interpretively generating offensive memes through instruction tuning. The experimental outcomes validate three key strengths of our method: first, it achieves a notable 19% enhancement in the BERTScore evaluation metric over baseline models; second, it incorporates comprehensive background knowledge pertinent to offensive metaphorical expressions within its interpretations; third, it exhibits strong generalization capabilities when handling previously unseen meme data. © 2024 Science Press. All rights reserved.