Worsening heat waves and extreme natural calamities have made it more important than ever to accurately predict weather forecasts. AI is proving increasingly helpful with the involvement of big tech companies in the domain.

In the latest attempt to help with the global climate crisis, Google in collaboration with ECMWF has announced WeatherBench 2 (WB2), a benchmark for data-driven, global weather models. This is an update to the original benchmark introduced in 2020, which was based on initial, lower-resolution ML models.

Evaluating weather forecasts isn’t an easy task, because weather is a multifaceted problem. Different end-users are interested in different properties of forecasts, To help with this, the WB2 benchmark will progress the models by providing a reproducible framework for evaluating and comparing several methods. 

The main element of WB2 is an open-source evaluation framework through which users can forecast similar to other baselines. The sheer size of high resolution data required to evaluate is a challenge. Hence, Google built the evaluation code on Apache Beam which lets users split computation into small chunks and evaluate them. The code comes with a guide to help users get up to speed. 

Moreover, most of the data is provided by the developers on Google Cloud Storage in Zarr format at various resolutions including a copy of ERA5 dataset used to train most ML models. With this Google is making an effort to provide analysis-ready, cloud-optimized weather and climate datasets to the community.

On their webpage, Google also provides scores from several state-of-the-art models like DeepMind’s GraphCast and Huawei’s Pangu-Weather, a transformer-based model. Additionally, forecasts from ECMWF’s forecasting systems are included, representing some of the models.

With WB2, Google aims to strengthen the future of ML-based weather prediction. The company also has plans to add station observations, better datasets and include nowcasting as well as subseasonal-to-seasonal predictions to the benchmark.