Combining Language Models and Linguistic Information to Label Entities in Memes

This paper describes the system we developed for the shared task "Hero, Villain and Victim: Dissecting harmful memes for Semantic role labeling of entities" organized in the framework of the Second Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situation (Constraint 2022). We present an ensemble approach combining transformer-based models and linguistic information, such as the presence of irony and implicit sentiment associated to the target named entities. The ensemble system obtains promising classification scores, with a macro F-score of 55%, resulting in a third place finish in the competition.