I will give an overview of the interdisciplinary project "Integration and visualization of multisource information for mesoscale meteorology: Statistical and cognitive approaches to visualizing uncertainty," which brings together atmospheric scientists, psychologists and statisticians at the University of Washington. The goals of the project are the development of statistical methods for probabilistic weather forecasting, ways of visualizing and communicating such probabilistic statements, and tools for users to implement the methods.
The state of the atmosphere is very complex, and is often represented by 10 million numbers or so. We are interested in a further degree of complexity: statements of uncertainty about future values of this very high-dimensional quantity. How to present and communicate such uncertainty is only partly a statistical problem; it is also, and perhaps mainly, a cognitive problem. It is complicated by the fact that users such as weather forecasters for the media, shipping or aviation work under intense time pressure with many interruptions, and already suffer from information overload; if the information is not well presented, they simply won't use it. Led by the psychologists, our group is addressing this by ethnographic field study of how forecasters actually use information and of the social structure of forecasting operations, and by cognitive experiments assessing the use of probabilistic information and different ways of displaying it. Results about effective ways to display complex probabilistic information could be useful for quantitative researchers more generally. I will outline some experiments that are currently under way to assess various ways of communicating uncertainty.
In this talk, I will consider in more detail the first goal of the project: the problem of calibrated and sharp probabilistic forecasting of a future meteorological quantity. By calibrated, we mean that if we define a predictive interval, such as a 90% probability interval, then on average in the long run, 90% of such intervals contain the true value. By "sharp," we mean that the distribution is more concentrated than forecast distributions from climatology (i.e. the marginal distribution) alone. UW Atmospheric Science Professor Cliff Mass and his group have developed an ensemble forecasting system for the Pacific Northwest based on a set of weather forecasting deterministic simulation models. He has established a clear relationship between between-model variability and forecast errors, but his forecast intervals are generally not calibrated; they are too narrow. This seems contradictory at first sight. We apply Bayesian model averaging to develop probability forecasts using Mass's ensemble. The theory of Bayesian model averaging explains both of Mass's main empirical findings: the spread-error relationship, and the fact that the intervals from the Mass ensemble are too narrow on average. We develop Bayesian model averaging forecasts and apply them to forecasts of sea-level pressure in the Pacific Northwest. The resulting forecasts are calibrated and sharp.
This work is in collaboration with Fadoua Balabdaoui, Tilmann Gneiting, Earl Hunt, Susan Joslyn, Michael Polakowski, and McLean Sloughter.
