From Imagery to Insight: Applying Multimodal Large Language Models for Rapid Disaster Impact Assessment

Abstract

Natural hazards continue to pose persistent and growing threats to infrastructure systems and communities worldwide. In the aftermath of a disaster, rapid damage assessment is critical for directing emergency response and allocating resources effectively. However, first responders and decision-makers often lack reliable, real-time validation of disaster impacts, resulting in delays in deploying critical aid and support. Most current methods rely on manual inspections and individual surveys, which are time-consuming, inconsistent, and difficult to scale across large, affected areas. Recent advancements in artificial intelligence and vision-language models—particularly in Multimodal Large Language Models (MMLLMs)—offer the potential to automate image-based damage evaluation, making the process faster, more objective, and scalable. Drawing on established engineering guidelines for disaster damage assessment—such as those from STEER, FEMA, and ATC—a set of metadata fields was identified. These fields were used to annotate an expert-curated dataset of post-disaster imagery. Using prompt engineering and the Gemini API, automated damage assessments were generated from the imagery. The predicted outputs were then used to evaluate the performance of the MMLLM across post-disaster scenarios. A confusion matrix approach was applied to assess both binary and multi-class classification performance, from which accuracy, precision, and recall metrics were computed. This automated approach to damage analysis demonstrates strong potential to improve disaster response by enabling standardized assessments that can guide resource allocation and rescue operations more efficiently than traditional methods.