Submitted as: development and technical paper 07 Jan 2022

Submitted as: development and technical paper | 07 Jan 2022

Review status: this preprint is currently under review for the journal GMD.

AIEADA 1.0: Efficient high-dimensional variational data assimilation with machine-learned reduced-order models

Romit Maulik1, Vishwas Rao1, Jiali Wang1, Gianmarco Mengaldo2, Emil Constantinescu1, Bethany Lusch1, Prasanna Balaprakash1, Ian Foster1, and Rao Kotamarthi1 Romit Maulik et al.
  • 1Argonne National Laboratory
  • 2National University of Singapore

Abstract. Data assimilation (DA) in the geophysical sciences remains the cornerstone of robust forecasts from numerical models. Indeed, DA plays a crucial role in the quality of numerical weather prediction, and is a crucial building block that has allowed dramatic improvements in weather forecasting over the past few decades. DA is commonly framed in a variational setting, where one solves an optimization problem within a Bayesian formulation using raw model forecasts as a prior, and observations as likelihood. This leads to a DA objective function that needs to be minimized, where the decision variables are the initial conditions specified to the model. In traditional DA, the forward model is numerically and computationally expensive. Here we replace the forward model with a low-dimensional, data-driven, and differentiable emulator. Consequently, gradients of our DA objective function with respect to the decision variables are obtained rapidly via automatic differentiation. We demonstrate our approach by performing an emulator-assisted DA forecast of geopotential height. Our results indicate that emulator-assisted DA is faster than traditional equation-based DA forecasts by four orders of magnitude, allowing computations to be performed on a workstation rather than a dedicated high-performance computer. In addition, we describe accuracy benefits of emulator-assisted DA when compared to simply using the emulator for forecasting (i.e., without DA). Our overall formulation is denoted AIAEDA (Artificial Intelligence Emulator Assisted Data Assimilation).

Romit Maulik et al.

Status: open (until 04 Mar 2022)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse

Romit Maulik et al.

Romit Maulik et al.


Total article views: 320 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
229 87 4 320 2 2
  • HTML: 229
  • PDF: 87
  • XML: 4
  • Total: 320
  • BibTeX: 2
  • EndNote: 2
Views and downloads (calculated since 07 Jan 2022)
Cumulative views and downloads (calculated since 07 Jan 2022)

Viewed (geographical distribution)

Total article views: 293 (including HTML, PDF, and XML) Thereof 293 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
Latest update: 27 Jan 2022
Short summary
In numerical weather prediction, data assimilation is frequently utilized to enhance the accuracy of forecasts from equation-based models. In this work – we use a machine learning framework that approximates a complex dynamical system given by the geopotential height. Instead of using an equation-based model, we utilize this machine-learned alternative to dramatically accelerate both the forecast and the assimilation of data thereby reducing need for large computational resources.