Submitted as: development and technical paper
06 Oct 2021
Submitted as: development and technical paper | 06 Oct 2021
Status: a revised version of this preprint is currently under review for the journal GMD.

A fast, single-iteration ensemble Kalman smoother for sequential data assimilation

Colin Grudzien1 and Marc Bocquet2 Colin Grudzien and Marc Bocquet
  • 1Department of Mathematics and Statistics, University of Nevada, Reno, Reno, Nevada, USA
  • 2CEREA, École des Ponts and EDF R&D, Île-de-France, France

Abstract. Ensemble-variational methods form the basis of the state-of-the-art for nonlinear, scalable data assimilation, yet current designs may not be cost-effective for reducing prediction error in online, short-range forecast systems. We propose a novel, outer-loop optimization of the ensemble-variational formalism for applications in which forecast error dynamics are weakly nonlinear, such as synoptic meteorology. In order to rigorously derive our method and demonstrate its novelty, we review ensemble smoothers that appear throughout the literature in a unified Bayesian maximum-a-posteriori narrative, updating and simplifying some results. After mathematically deriving our technique, we systematically develop and inter-compare all studied schemes in the open-source Julia package DataAssimilationBenchmarks.jl, with pseudo-code provided for these methods. This high-performance numerical framework, supporting our mathematical results, produces extensive benchmarks that demonstrate the significant performance advantages of our proposed technique. In particular, our single-iteration ensemble Kalman smoother is shown both to improve prediction / posterior accuracy and to simultaneously reduce the leading order cost of iterative, sequential smoothers in a variety of relevant test cases for operational short-range forecasts. This long work is thus intended to present our novel single-iteration ensemble Kalman smoother, and to provide a theoretical and computational framework for the study of sequential, ensemble-variational Kalman filters and smoothers generally.

Colin Grudzien and Marc Bocquet

Status: final response (author comments only)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on gmd-2021-306', Anonymous Referee #1, 26 Oct 2021
  • RC2: 'Comment on gmd-2021-306', Pavel Sakov, 07 Nov 2021
  • AC1: 'Authors' Response to Reviewers', Colin Grudzien, 24 Jan 2022

Colin Grudzien and Marc Bocquet

Colin Grudzien and Marc Bocquet


Total article views: 1,039 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
807 211 21 1,039 6 8
  • HTML: 807
  • PDF: 211
  • XML: 21
  • Total: 1,039
  • BibTeX: 6
  • EndNote: 8
Views and downloads (calculated since 06 Oct 2021)
Cumulative views and downloads (calculated since 06 Oct 2021)

Viewed (geographical distribution)

Total article views: 958 (including HTML, PDF, and XML) Thereof 958 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
Latest update: 20 May 2022
Short summary
Iterative optimization techniques, at the state-of-the-art of data assimilation, have largely focused on extending forecast accuracy to moderate- to long-range forecast systems. However, current methodology may not be cost-effective in reducing forecast errors in online, short-range forecast systems. We propose a novel optimization of these techniques for online, short-range forecast cycles, simultaneously providing an improvement in forecast accuracy and a reduction of the computational cost.