Uncertainty Quantification with Probabilistic Deep Learning Models to Predict Out-of-distribution Events

Abstract

This project investigates how probabilistic deep learning models—such as Bayesian Neural Networks (BNN) and Monte Carlo Dropout (MCD)—can quantify predictive uncertainty to detect unexpected air quality extremes. The core objective is to illustrate how uncertainty estimates can facilitate the identification of out-of-distribution events in real-time, thereby supporting more reliable calibration and decision-making for PM₂.₅ monitoring during severe wildfire episodes. Using the 2025 Los Angeles wildfire as a case study, the work focuses on calibrating PM₂.₅ measurements from low-cost Clarity sensors and demonstrating how model uncertainty increases when extreme wildfire smoke pushes concentrations beyond the conditions seen during training. Accuracy was measured using traditional values such as R² and RSME. As expected, the accuracy of the model is significantly greater for in-distribution data regardless of whether BNN or MCD is used. Understanding the uncertainty for PM₂.₅ predictions will allow government agencies to be better prepared for out-of-distribution events such as wildfires or industrial accidents, and ensure accuracy in models during rapidly evolving, high-risk environmental scenarios with limited ground truth coverage.