Assessing Building Damage by Learning the Deep Feature Correspondence of Before and After Aerial Images

The damage caused by a natural disaster such as a hurricane, not only impacts human lives but can also be detrimental to the city's infrastructure and potentially cause the loss of historical buildings and essential records. Delivering an effective response requires quick and precise analyses concerning the impact of a disastrous event. With the current technological developments to acquire massive volumes of data and the recent advances in artificial intelligence and machine learning, now more than ever, disaster information integration and fusion have the potential to deliver enhanced situational awareness tools for humanitarian assistance and disaster relief efforts. Given the aerial images of a residential building taken before and after a natural disaster, recent applications of Convolutional Neural Networks (CNNs) work well when differentiating two types of damage (i.e., whether the structure is intact or destroyed) but underperform when trying to differentiate more damage levels. According to our findings: (1) including enough surrounding context provides essential visual clues that help the model better predict the building's level of damage and (2) learning the correspondence between the features extracted from pre- and post-imagery boosts the performance compared to a simple concatenation. We propose a two-stream CNN architecture that overcomes the difficulties of classifying the buildings at four damage levels and evaluate its performance on a curated, fully-labeled dataset assembled from open sources.