Earthly Machine Learning

🎧 Abstract:
In this episode, we dive into GraphDOP, a novel data-driven forecasting system developed by ECMWF. Unlike traditional models, GraphDOP learns directly from Earth System observations—without relying on physics-based reanalysis. By capturing relationships between satellite and conventional observations, it builds a latent representation of Earth’s dynamic systems and delivers accurate weather forecasts up to five days ahead.

📌 Bullet points summary:

• GraphDOP is developed by ECMWF and operates purely on observational data, without physics-based (re)analysis or feedback.

• Produces skillful forecasts for surface and upper-air parameters up to five days into the future.

• Competes with ECMWF’s IFS for two-metre temperature (t2m), outperforming it in the Tropics at 5-day lead times.

• Can generate forecasts at any time and location—even where observational data is sparse—without using gridded ERA5 fields for training.

• Combines data from various instruments to create accurate joint forecasts of surface and tropospheric temperatures in the Tropics.

• Learns observation relationships that generalize well to data-sparse regions, with upper-level wind forecasts aligning closely with ERA5 even in low-coverage areas.

💡 The Big Idea:
GraphDOP reimagines weather forecasting by proving that pure observational data—when paired with intelligent modeling—can rival and even surpass traditional, physics-based systems in both speed and accuracy.

📚 Citation:
Alexe, Mihai, et al. "GraphDOP: Towards skilful data-driven medium-range weather forecasts learnt and initialised directly from observations." arXiv preprint arXiv:2412.15687 (2024). https://doi.org/10.48550/arXiv.2412.15687

🎧 Abstract:
In this episode, we explore DiffDA, a novel data assimilation approach for weather forecasting and climate modeling. Built on the foundations of denoising diffusion models, DiffDA uses the pretrained GraphCast neural network to assimilate atmospheric variables from predicted states and sparse observations—providing a data-driven pathway to generate accurate initial conditions for forecasts.

📌 Bullet points summary:

• Introduces DiffDA, a machine learning-based data assimilation method that leverages predicted states and sparse observations.

• Utilizes the pretrained GraphCast weather model, repurposed as a denoising diffusion model.

• Employs a two-phase conditioning strategy: on predicted states (training/inference) and sparse observations (inference only).

• Capable of generating assimilated global atmospheric data at 0.25° resolution.

• Demonstrates that initial conditions created via DiffDA retain forecast quality with a lead time degradation of at most 24 hours compared to top-tier assimilation systems.

• Enables autoregressive reanalysis dataset generation without full observation availability.

💡 The Big Idea:
DiffDA represents a step forward in data assimilation—merging the strengths of diffusion models and machine learning to produce accurate, observation-consistent initial conditions for future-focused forecasting.

📚 Citation:
Huang, Langwen, et al. "Diffda: a diffusion model for weather-scale data assimilation." arXiv preprint arXiv:2401.05932 (2024). https://doi.org/10.48550/arXiv.2401.059327

🎙️ Abstract:
Embedding physical constraints as inductive priors is key in AI weather forecasting models. Locality—a common prior—relies on local neural interactions like 3D convolutions or attention. ARCHESWEATHER challenges this norm by introducing global vertical interactions, improving efficiency without sacrificing accuracy.

📌 Bullet points summary:

• ARCHESWEATHER is a lightweight, efficient AI model trained at 1.5º resolution with minimal compute (a few GPU-days), offering low-cost inference and strong performance.

• The Cross-Level Attention (CLA) mechanism enables vertical atmospheric feature interactions, replacing 3D local attention with 2D horizontal attention and column-wise CLA in a 3D Swin U-Net with Earth-specific biases.

• Ensemble versions (MX4 and LX2) outperform or match IFS HRES and NeuralGCM in RMSE for 1–3 day forecasts on upper-air variables; it gains edge on wind variables at longer lead times.

• Fine-tuning on post-2007 ERA5 data yields modest gains, pointing to distributional shifts in the dataset.

• A convolutional head with bilinear upsampling avoids checkerboard artifacts, offering cleaner projections. The code is open-source.

💡 Big Idea:
ARCHESWEATHER shows that global vertical interactions via cross-level attention can outperform traditional locality-based models, paving a path toward more efficient, physically grounded weather forecasting systems.

📚 Citation:
Mukkavilli, S. Karthik, et al. "Ai foundation models for weather and climate: Applications, design, and implementation." arXiv preprint arXiv:2309.10808 (2023). DOI: 10.48550/arXiv.2405.14527

🌍 Abstract:
Surface-level weather is what matters most to the public—but it's also where feedback loops and complex interactions dominate. Land Surface Models (LSMs) capture these dynamics. Coupled with atmospheric models, they help forecast water, carbon, and energy fluxes. This study explores machine learning emulators as fast, accurate alternatives for ecLand, the ECMWF’s land surface scheme.

⚡ Bullet points summary:

• Three machine learning models—LSTM, XGB, and MLP—were evaluated as statistical emulators for ECLand to enable faster experimentation in land surface forecasting.

• All models showed strong accuracy, but LSTM excelled in long-range continental forecasts, XGB was robust across tasks, and MLP balanced accuracy and ease of use.

• Emulators offered significant runtime savings over traditional numerical models, boosting potential for quicker simulations and integration into data assimilation pipelines.

• Model strengths varied by scale and variable: XGB led in European soil water predictions, MLP scored highest in global accuracy, and LSTM improved snow cover forecasts on a continental scale.

• The study provides a clear comparison of model trade-offs, helping guide the choice of emulator based on accuracy needs, compute budget, and geographic focus.

💡 Big Idea:
Machine learning emulators can dramatically speed up land surface forecasting without compromising accuracy—empowering faster, more adaptable weather research and operations.

📚 Citation:
Wesselkamp, M., et al. Advances in Land Surface Model-based Forecasting: A Comparison of LSTM, Gradient Boosting, and Feedforward Neural Networks as Prognostic State Emulators in a Case Study with ECLand, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2024-2081, 2024.

DOI:https://doi.org/10.48550/arXiv.2411.11268

Abstract:

Existing machine learning models of weather variability are not formulated to enable assessment of their response to varying external boundary conditions such as sea surface temperature and greenhouse gases. Here we present ACE2 (Ai2 Climate Emulator version 2) and its application to reproducing atmospheric variability over the past 80 years on timescales from days to decades. ACE2 is a 450M-parameter autoregressive machine learning emulator, operating with 6-hour temporal resolution, 1° horizontal resolution and eight vertical layers. It exactly conserves global dry air mass and moisture and can be stepped forward stably for arbitrarily many steps with a throughput of about 1500 simulated years per wall clock day....

Citation:

Watt-Meyer, Oliver, et al. "ACE2: Accurately learning subseasonal to decadal atmospheric variability and forced responses." arXiv preprint arXiv:2411.11268 (2024).

DOI:https://doi.org/10.48550/arXiv.2405.13063

Abstract:Reliable forecasts of the Earth system are crucial for human progress and safety from natural disasters. Artificial intelligence offers substantial potential to improve prediction accuracy and computational efficiency in this field, however this remains underexplored in many domains. Here we introduce Aurora, a large-scale foundation model for the Earth system trained on over a million hours of diverse data...

Citation: Bodnar, Cristian, et al. "Aurora: A foundation model of the atmosphere." arXiv preprint arXiv:2405.13063 (2024).

Abstract:

Existing ML-based atmospheric models are not suitable for climate prediction, which requires long-term stability and physical consistency. We present ACE (AI2 Climate Emulator), a 200M-parameter, autoregressive machine learning emulator of an existing comprehensive 100-km resolution global atmospheric model. The formulation of ACE allows evaluation of physical laws such as the conservation of mass and moisture...

Citation: Watt-Meyer, Oliver, et al. "ACE: A fast, skillful learned global atmospheric model for climate prediction." arXiv preprint arXiv:2310.02074 (2023).

DOI:https://doi.org/10.48550/arXiv.2310.02074

DOI:https://doi.org/10.48550/arXiv.2308.15560
Abstract:

WeatherBench 2 is an update to the global, medium-range (1-14 day) weather forecasting benchmark proposed by Rasp et al. (2020), designed with the aim to accelerate progress in data-driven weather modeling. WeatherBench 2 consists of an open-source evaluation framework, publicly available training, ground truth and baseline data as well as a continuously updated website with the latest metrics and state-of-the-art model...

Citation: Rasp, Stephan, et al. "WeatherBench 2: A benchmark for the next generation of data‐driven global weather models." Journal of Advances in Modeling Earth Systems 16.6 (2024): e2023MS004019.

Abstract:

Ensemble forecasting is crucial for improving weather predictions, especially for forecasts of extreme events. Constructing an ensemble prediction system (EPS) based on conventional NWP models is highly computationally expensive. ML models have emerged as valuable tools for deterministic weather forecasts, providing forecasts with significantly reduced computational requirements and even surpassing the forecast performance of traditional NWP models. However, challenges arise when applying ML models to ensemble forecasting. Recent ML models, such as GenCast and SEEDS model, rely on the ERA5 EDA or operational NWP ensemble members for forecast generation. Their spatial resolution is also considered too coarse for many applications. To overcome these limitations, we introduce FuXi-ENS, an advanced ML model designed to deliver 6-hourly global ensemble weather forecasts up to 15 days. This model runs at a significantly increased spatial resolution of 0.25\textdegree, incorporating 5 atmospheric variables at 13 pressure levels, along with 13 surface variables. By leveraging the inherent probabilistic nature of Variational AutoEncoder (VAE), FuXi-ENS optimizes a loss function that combines the CRPS and the KL divergence between the predicted and target distribution, facilitating the incorporation of flow-dependent perturbations in both initial conditions and forecast. This innovative approach makes FuXi-ENS an advancement over the traditional ones that use L1 loss combined with the KL loss in standard VAE models for ensemble weather forecasting. Results demonstrate that FuXi-ENS outperforms ensemble forecasts from the ECMWF, a world leading NWP model, in the CRPS of 98.1% of 360 variable and forecast lead time combinations. This achievement underscores the potential of the FuXi-ENS model to enhance ensemble weather forecasts, offering a promising direction for further development in this field.

Zhong, Xiaohui, et al. "FuXi-ENS: A machine learning model for medium-range ensemble weather forecasting." arXiv preprint arXiv:2405.05925 (2024).

arXiv:2405.05925

Abstract : Fourier Neural Operators (FNOs) have proven to be an efficient and effective method for resolution-independent operator learning in a broad variety of application areas across scientific machine learning. A key reason for their success is their ability to accurately model long-range dependencies in spatio-temporal data by learning global convolutions in a computationally efficient manner. To this end, FNOs rely on the discrete Fourier transform (DFT), however, DFTs cause visual and spectral artifacts as well as pronounced dissipation when learning operators in spherical coordinates since they incorrectly assume a flat geometry. To overcome this limitation, we generalize FNOs on the sphere, introducing Spherical FNOs (SFNOs) for learning operators on spherical geometries. We apply SFNOs to forecasting atmospheric dynamics, and demonstrate stable auto\-regressive rollouts for a year of simulated time (1,460 steps), while retaining physically plausible dynamics. The SFNO has important implications for machine learning-based simulation of climate dynamics that could eventually help accelerate our response to climate change.

Citation: Bonev, Boris, et al. "Spherical fourier neural operators: Learning stable dynamics on the sphere." International conference on machine learning. PMLR, 2023

DOI: https://doi.org/10.48550/arXiv.2306.03838