Image to Text Recognition for Detecting Human and Machine Altered News in Social Media

Fake news on social media platforms has become a pressing issue, undermining the reliability of factual information. Platforms like Twitter, Instagram, and Reddit lack measures to recognize falsified information that is circulated via images. The existing state of the art solutions can ascertain the origin of a text string generated by a neural language model, they encounter difficulties when it comes to recognizing human-made modifications to news content and detecting instances of fake news conveyed through images. To address this challenge, we propose a solution that utilizes the Optical Character Recognition (OCR) capabilities of the Google Cloud Vision API to extract text from news images. The extracted text is then cross-referenced with the New York Times (NYT) database to verify the authenticity of the news articles. Our testing on human-altered, fake, and real news images yielded a 84.54% accuracy in detecting falsified news articles. This paper offers a promising approach to combat misinformation in image-based news content shared on social media platforms, thereby contributing to the preservation of factual information integrity.