Enhancing Disaster Situation Awareness Through Multimodal Social Media Data: Evidence from Typhoon Haikui

Emergency situation awareness during sudden natural disasters presents significant challenges. Traditional methods, characterized by low spatial and temporal resolution as well as coarse granularity, often fail to comprehensively capture disaster situations. However, social media platforms, as a vital source of social sensing, offer significant potential to supplement disaster situational awareness. This paper proposes an innovative framework for disaster situation awareness based on multimodal data from social media to identify social media content related to typhoon disasters. Integrating text and image data from social media facilitates near real-time monitoring of disasters from the public perspective. In this study, Typhoon Haikui (Strong Typhoon No. 11 of 2023) was chosen as a case study to validate the effectiveness of the proposed method. We employed the ERNIE natural language processing model to complement the Deeplab v3+ deep learning image semantic segmentation model for extracting disaster damage information from social media. A spatial visualization analysis of the disaster-affected areas was performed by categorizing the damage types. Additionally, the Geodetector was used to investigate spatial heterogeneity and its underlying factors. This approach allowed us to analyze the spatiotemporal patterns of disaster evolution, enabling rapid disaster damage assessment and facilitating emergency response efforts. The results show that the proposed method significantly enhances situational awareness by effectively identifying different types of damage information from social sensing data.