Articles | Volume 16, issue 14
https://doi.org/10.5194/gmd-16-4233-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-16-4233-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Development and technical paper
| 
27 Jul 2023
Development and technical paper |
| 27 Jul 2023
| 27 Jul 2023
Simplified Kalman smoother and ensemble Kalman smoother for improving reanalyses
Department of Meteorology, University of Reading, Reading, UK
National Centre for Earth Observation, University of Reading, Reading, UK
Ross Bannister
Department of Meteorology, University of Reading, Reading, UK
National Centre for Earth Observation, University of Reading, Reading, UK
Yumeng Chen
Department of Meteorology, University of Reading, Reading, UK
National Centre for Earth Observation, University of Reading, Reading, UK
Alison Fowler
Department of Meteorology, University of Reading, Reading, UK
National Centre for Earth Observation, University of Reading, Reading, UK
Department of Meteorology, University of Reading, Reading, UK
National Centre for Earth Observation, University of Reading, Reading, UK
Related authors
Samantha Petch, Bo Dong, Tristan Quaife, Robert P. King, and Keith Haines
Hydrol. Earth Syst. Sci., 27, 1723–1744, https://doi.org/10.5194/hess-27-1723-2023, https://doi.org/10.5194/hess-27-1723-2023, 2023
Short summary
Short summary
Gravitational measurements of water storage from GRACE (Gravity Recovery and Climate Experiment) can improve understanding of the water budget. We produce flux estimates over large river catchments based on observations that close the monthly water budget and ensure consistency with GRACE on short and long timescales. We use energy data to provide additional constraints and balance the long-term energy budget. These flux estimates are important for evaluating climate models.
Jozef Skákala, David Ford, Keith Haines, Amos Lawless, Matthew J. Martin, Philip Browne, Marcin Chrust, Stefano Ciavatta, Alison Fowler, Daniel Lea, Matthew Palmer, Andrea Rochner, Jennifer Waters, Hao Zuo, Deep S. Banerjee, Mike Bell, Davi M. Carneiro, Yumeng Chen, Susan Kay, Dale Partridge, Martin Price, Richard Renshaw, Georgy Shapiro, and James While
Ocean Sci., 21, 1709–1734, https://doi.org/10.5194/os-21-1709-2025, https://doi.org/10.5194/os-21-1709-2025, 2025
Short summary
Short summary
UK marine data assimilation (MDA) involves a closely collaborating research community. In this paper, we offer both an overview of the state of the art and a vision for the future across all of the main areas of UK MDA, ranging from physics to biogeochemistry to coupled DA. We discuss the current UK MDA stakeholder applications, highlight theoretical developments needed to advance our systems, and reflect upon upcoming opportunities with respect to hardware and observational missions.
Davi Mignac, Jennifer Waters, Daniel J. Lea, Matthew J. Martin, James While, Anthony T. Weaver, Arthur Vidard, Catherine Guiavarc'h, Dave Storkey, David Ford, Edward W. Blockley, Jonathan Baker, Keith Haines, Martin R. Price, Michael J. Bell, and Richard Renshaw
Geosci. Model Dev., 18, 3405–3425, https://doi.org/10.5194/gmd-18-3405-2025, https://doi.org/10.5194/gmd-18-3405-2025, 2025
Short summary
Short summary
We describe major improvements of the Met Office's global ocean–sea ice forecasting system. The models and the way observations are used to improve the forecasts were changed, which led to a significant error reduction of 1 d forecasts. The new system performance in past conditions, where subsurface observations are scarce, was improved with more consistent ocean heat content estimates. The new system will be of better use for climate studies and will provide improved forecasts for end users.
Ieuan Higgs, Ross Bannister, Jozef Skákala, Alberto Carrassi, and Stefano Ciavatta
EGUsphere, https://doi.org/10.48550/arXiv.2504.05218, https://doi.org/10.48550/arXiv.2504.05218, 2025
Short summary
Short summary
We explored how machine learning can improve computer models that simulate ocean ecosystems. These models help us understand how the ocean works, but they often struggle due to limited observations and complex processes. Our approach uses machine learning to better connect the parts of the system we can observe with those we cannot. This leads to more accurate and efficient predictions, offering a promising way to improve future ocean monitoring and forecasting tools.
Samantha Petch, Liang Feng, Paul Palmer, Robert P. King, Tristan Quaife, and Keith Haines
EGUsphere, https://doi.org/10.22541/essoar.173343481.12875858/v1, https://doi.org/10.22541/essoar.173343481.12875858/v1, 2025
Short summary
Short summary
The growth rate of atmospheric CO2 varies year to year, mainly due to land ecosystems. Understanding factors controlling the land carbon uptake is crucial. Our study examines the link between terrestrial water storage and the CO2 growth rate from 2002–2023, revealing a strong negative correlation. We highlight the key role of tropical forests, especially in tropical America, and assess how regional contributions shift over time.
Yumeng Chen, Lars Nerger, and Amos S. Lawless
EGUsphere, https://doi.org/10.5194/egusphere-2024-1078, https://doi.org/10.5194/egusphere-2024-1078, 2024
Short summary
Short summary
In this paper, we present pyPDAF, a Python interface to the parallel data assimilation framework (PDAF) allowing for coupling with Python-based models. We demonstrate the capability and efficiency of pyPDAF under a coupled data assimilation setup.
Yumeng Chen, Polly Smith, Alberto Carrassi, Ivo Pasmans, Laurent Bertino, Marc Bocquet, Tobias Sebastian Finn, Pierre Rampal, and Véronique Dansereau
The Cryosphere, 18, 2381–2406, https://doi.org/10.5194/tc-18-2381-2024, https://doi.org/10.5194/tc-18-2381-2024, 2024
Short summary
Short summary
We explore multivariate state and parameter estimation using a data assimilation approach through idealised simulations in a dynamics-only sea-ice model based on novel rheology. We identify various potential issues that can arise in complex operational sea-ice models when model parameters are estimated. Even though further investigation will be needed for such complex sea-ice models, we show possibilities of improving the observed and the unobserved model state forecast and parameter accuracy.
Ross Noel Bannister and Chris Wilson
EGUsphere, https://doi.org/10.5194/egusphere-2024-655, https://doi.org/10.5194/egusphere-2024-655, 2024
Preprint archived
Short summary
Short summary
Prior information is essential for the top-down estimation of CH4 surface fluxes. Errors in the prior are correlated in time/space, but accounting for correlations can be costly. We report on an efficient scheme to represent correlations in the inverse modelling system, INVICAT. The method is tested by assimilating CH4 observations using the scheme. Our findings show that accounting for spatio-temporal correlations improve CH4 flux estimates, demonstrating that the method should be further used.
Ieuan Higgs, Jozef Skákala, Ross Bannister, Alberto Carrassi, and Stefano Ciavatta
Biogeosciences, 21, 731–746, https://doi.org/10.5194/bg-21-731-2024, https://doi.org/10.5194/bg-21-731-2024, 2024
Short summary
Short summary
A complex network is a way of representing which parts of a system are connected to other parts. We have constructed a complex network based on an ecosystem–ocean model. From this, we can identify patterns in the structure and areas of similar behaviour. This can help to understand how natural, or human-made, changes will affect the shelf sea ecosystem, and it can be used in multiple future applications such as improving modelling, data assimilation, or machine learning.
Jiangshan Zhu and Ross Noel Bannister
Geosci. Model Dev., 16, 6067–6085, https://doi.org/10.5194/gmd-16-6067-2023, https://doi.org/10.5194/gmd-16-6067-2023, 2023
Short summary
Short summary
We describe how condensation and evaporation are included in the existing (otherwise dry) simplified ABC model. The new model (Hydro-ABC) includes transport of vapour and condensate within a dynamical core, and it transitions between these two phases via a micro-physics scheme. The model shows the development of an anvil cloud and excitation of atmospheric waves over many frequencies. The covariances that develop between variables are also studied together with indicators of convective motion.
Tobias Sebastian Finn, Charlotte Durand, Alban Farchi, Marc Bocquet, Yumeng Chen, Alberto Carrassi, and Véronique Dansereau
The Cryosphere, 17, 2965–2991, https://doi.org/10.5194/tc-17-2965-2023, https://doi.org/10.5194/tc-17-2965-2023, 2023
Short summary
Short summary
We combine deep learning with a regional sea-ice model to correct model errors in the sea-ice dynamics of low-resolution forecasts towards high-resolution simulations. The combined model improves the forecast by up to 75 % and thereby surpasses the performance of persistence. As the error connection can additionally be used to analyse the shortcomings of the forecasts, this study highlights the potential of combined modelling for short-term sea-ice forecasting.
Nicholas Williams, Nicholas Byrne, Daniel Feltham, Peter Jan Van Leeuwen, Ross Bannister, David Schroeder, Andrew Ridout, and Lars Nerger
The Cryosphere, 17, 2509–2532, https://doi.org/10.5194/tc-17-2509-2023, https://doi.org/10.5194/tc-17-2509-2023, 2023
Short summary
Short summary
Observations show that the Arctic sea ice cover has reduced over the last 40 years. This study uses ensemble-based data assimilation in a stand-alone sea ice model to investigate the impacts of assimilating three different kinds of sea ice observation, including the novel assimilation of sea ice thickness distribution. We show that assimilating ice thickness distribution has a positive impact on thickness and volume estimates within the ice pack, especially for very thick ice.
Samantha Petch, Bo Dong, Tristan Quaife, Robert P. King, and Keith Haines
Hydrol. Earth Syst. Sci., 27, 1723–1744, https://doi.org/10.5194/hess-27-1723-2023, https://doi.org/10.5194/hess-27-1723-2023, 2023
Short summary
Short summary
Gravitational measurements of water storage from GRACE (Gravity Recovery and Climate Experiment) can improve understanding of the water budget. We produce flux estimates over large river catchments based on observations that close the monthly water budget and ensure consistency with GRACE on short and long timescales. We use energy data to provide additional constraints and balance the long-term energy budget. These flux estimates are important for evaluating climate models.
Sukun Cheng, Yumeng Chen, Ali Aydoğdu, Laurent Bertino, Alberto Carrassi, Pierre Rampal, and Christopher K. R. T. Jones
The Cryosphere, 17, 1735–1754, https://doi.org/10.5194/tc-17-1735-2023, https://doi.org/10.5194/tc-17-1735-2023, 2023
Short summary
Short summary
This work studies a novel application of combining a Lagrangian sea ice model, neXtSIM, and data assimilation. It uses a deterministic ensemble Kalman filter to incorporate satellite-observed ice concentration and thickness in simulations. The neXtSIM Lagrangian nature is handled using a remapping strategy on a common homogeneous mesh. The ensemble is formed by perturbing air–ocean boundary conditions and ice cohesion. Thanks to data assimilation, winter Arctic sea ice forecasting is enhanced.
Joshua Chun Kwang Lee, Javier Amezcua, and Ross Noel Bannister
Geosci. Model Dev., 15, 6197–6219, https://doi.org/10.5194/gmd-15-6197-2022, https://doi.org/10.5194/gmd-15-6197-2022, 2022
Short summary
Short summary
In this article, we implement a novel data assimilation method for the ABC–DA system which combines traditional data assimilation approaches in a hybrid approach. We document the technical development and test the hybrid approach in idealised experiments within a tropical framework of the ABC–DA system. Our findings indicate that the hybrid approach outperforms individual traditional approaches. Its potential benefits have been highlighted and should be explored further within this framework.
Yumeng Chen, Alberto Carrassi, and Valerio Lucarini
Nonlin. Processes Geophys., 28, 633–649, https://doi.org/10.5194/npg-28-633-2021, https://doi.org/10.5194/npg-28-633-2021, 2021
Short summary
Short summary
Chaotic dynamical systems are sensitive to the initial conditions, which are crucial for climate forecast. These properties are often used to inform the design of data assimilation (DA), a method used to estimate the exact initial conditions. However, obtaining the instability properties is burdensome for complex problems, both numerically and analytically. Here, we suggest a different viewpoint. We show that the skill of DA can be used to infer the instability properties of a dynamical system.
Yumeng Chen, Konrad Simon, and Jörn Behrens
Geosci. Model Dev., 14, 2289–2316, https://doi.org/10.5194/gmd-14-2289-2021, https://doi.org/10.5194/gmd-14-2289-2021, 2021
Short summary
Short summary
Mesh adaptivity can reduce overall model error by only refining meshes in specific areas where it us necessary in the runtime. Here we suggest a way to integrate mesh adaptivity into an existing Earth system model, ECHAM6, without having to redesign the implementation from scratch. We show that while the additional computational effort is manageable, the error can be reduced compared to a low-resolution standard model using an idealized test and relatively realistic dust transport tests.
Irene Polo, Keith Haines, Jon Robson, and Christopher Thomas
Ocean Sci., 16, 1067–1088, https://doi.org/10.5194/os-16-1067-2020, https://doi.org/10.5194/os-16-1067-2020, 2020
Short summary
Short summary
AMOC variability controls climate and is driven by wind and buoyancy forcing in the Atlantic. Density changes there are expected to connect to tropical regions. We develop methods to identify boundary density profiles at 26° N which relate to the AMOC. We found that density anomalies propagate equatorward along the western boundary, eastward along the Equator and then poleward up the eastern boundary with 2 years lag between boundaries. Record lengths of more than 26 years are required.
Cited articles
Balmaseda, M. A., Mogensen, K., and Weaver, A. T.: Evaluation of the ECMWF
ocean reanalysis system ORAS4, Q. J. Roy. Meteor.
Soc., 139, 1132–1161, 2013. a
Bishop, C. H., Etherton, B. J., and Majumdar, S. J.: Adaptive Sampling with the
Ensemble Transform Kalman Filter. Part I: Theoretical Aspects, Mon.
Weather Rev., 129, 420–436,
https://doi.org/10.1175/1520-0493(2001)129<0420:ASWTET>2.0.CO;2, 2001. a, b
Bocquet, M. and Sakov, P.: An iterative ensemble Kalman smoother, Q.
J. Roy. Meteor. Soc., 140, 1521–1535,
https://doi.org/10.1002/qj.2236, 2014. a, b
Buizza, R., Poli, P., Rixen, M., Alonso-Balmaseda, M., Bosilovich, M. G.,
Brönnimann, S., Compo, G. P., Dee, D. P., Desiato, F., Doutriaux-Boucher,
M., Fujiwara, M., Kaiser-Weiss, A. K., Kobayashi, S., Liu, Z., Masina, S.,
Mathieu, P.-P., Rayner, N., Richter, C., Seneviratne, S. I., Simmons, A. J.,
Thépaut, J.-N., Auger, J. D., Bechtold, M., Berntell, E., Dong, B., Kozubek,
M., Sharif, K., Thomas, C., Schimanke, S., Storto, A., Tuma, M., Välisuo,
I., and Vaselali, A.: Advancing Global and Regional Reanalyses, B.
Am. Meteorol. Soc., 99, ES139–ES144,
https://doi.org/10.1175/BAMS-D-17-0312.1, 2018. a
Burgers, G., Van Leeuwen, P. J., and Evensen, G.: Analysis scheme in the
ensemble Kalman filter, Mon. Weather Rev., 126, 1719–1724, 1998. a
Cosme, E., Brankart, J.-M., Verron, J., Brasseur, P., and Krysta, M.:
Implementation of a reduced rank square-root smoother for high resolution
ocean data assimilation, Ocean Model., 33, 87–100,
https://doi.org/10.1016/j.ocemod.2009.12.004, 2010. a
Desroziers, G., Arbogast, E., and Berre, L.: Improving spatial localization in
4DEnVar, Q. J. Roy. Meteor. Soc., 142,
3171–3185, 2016. a
Dong, B. and Chen, Y.: code for paper “Simplified Kalman smoother and ensemble Kalman smoother for improving ocean forecasts and reanalyses”, Zenodo [code], https://doi.org/10.5281/zenodo.7675286, 2023. a
Evensen, G.: Sequential data assimilation with a nonlinear quasi-geostrophic
model using Monte Carlo methods to forecast error statistics, J.
Geophys. Res.-Oceans, 99, 10143–10162,
https://doi.org/10.1029/94JC00572, 1994. a
Evensen, G.: The ensemble Kalman filter for combined state and parameter
estimation, IEEE Contr. Syst. Mag., 29, 83–104,
https://doi.org/10.1109/MCS.2009.932223, 2009. a
Evensen, G. and van Leeuwen, P. J.: An Ensemble Kalman Smoother for Nonlinear
Dynamics, Mon. Weather Rev., 128, 1852–1867,
https://doi.org/10.1175/1520-0493(2000)128<1852:AEKSFN>2.0.CO;2, 2000. a, b
Evensen, G., Vossepoel, F. C., and van Leeuwen, P. J.: Data assimilation
fundamentals: A unified formulation of the state and parameter estimation
problem, Springer Nature, ISBN
9783030967093, 9783030967093, https://doi.org/10.1007/978-3-030-96709-3, 2022. a
Feng, L., Palmer, P. I., Bösch, H., and Dance, S.: Estimating surface CO2 fluxes from space-borne CO2 dry air mole fraction observations using an ensemble Kalman Filter, Atmos. Chem. Phys., 9, 2619–2633, https://doi.org/10.5194/acp-9-2619-2009, 2009. a
Hamill, T. M., Whitaker, J. S., Fiorino, M., and Benjamin, S. G.: Global
ensemble predictions of 2009’s tropical cyclones initialized with an
ensemble Kalman filter, Mon. Weather Rev., 139, 668–688, 2011. a
Houtekamer, P. L. and Mitchell, H. L.: Data assimilation using an ensemble
Kalman filter technique, Mon. Weather Rev., 126, 796–811, 1998. a
Houtekamer, P. L., Mitchell, H. L., Pellerin, G., Buehner, M., Charron, M.,
Spacek, L., and Hansen, B.: Atmospheric Data Assimilation with an Ensemble
Kalman Filter: Results with Real Observations, Mon. Weather Rev., 133,
604–620, https://doi.org/10.1175/MWR-2864.1, 2005. a
Khare, S. P., Anderson, J. L., Hoar, T. J., and Nychka, D.: An investigation
into the application of an ensemble Kalman smoother to high-dimensional
geophysical systems, Tellus A, 60,
97–112, https://doi.org/10.1111/j.1600-0870.2007.00281.x, 2008. a
MacLachlan, C., Arribas, A., Peterson, K. A., Maidens, A., Fereday, D., Scaife, A. A., Gordon, M., Vellinga, M., Williams, A., Comer, R. E., Camp, J., Xavier, P., and Madec, G.: Global
Seasonal forecast system version 5 (GloSea5): A high-resolution seasonal
forecast system, Q. J. Roy. Meteor. Soc., 141,
1072–1084, 2015. a
Pan, Y., Zhu, K., Xue, M., Wang, X., Hu, M., Benjamin, S. G., Weygandt, S. S.,
and Whitaker, J. S.: A GSI-Based Coupled EnSRF–En3DVar Hybrid Data
Assimilation System for the Operational Rapid Refresh Model: Tests at a
Reduced Resolution, Mon. Weather Rev., 142, 3756–3780,
https://doi.org/10.1175/MWR-D-13-00242.1, 2014. a
Petrie, R. E. and Dance, S. L.: Ensemble-based data assimilation and the
localisation problem, Weather, 65, 65–69, 2010. a
Pinnington, E., Quaife, T., Lawless, A., Williams, K., Arkebauer, T., and Scoby, D.: The Land Variational Ensemble Data Assimilation Framework: LAVENDAR v1.0.0, Geosci. Model Dev., 13, 55–69, https://doi.org/10.5194/gmd-13-55-2020, 2020. a
Raanes, P. N., Grudzien, C., and 14tondeu: nansencenter/DAPPER: Version 0.8 (v0.8), Zenodo [code], https://doi.org/10.5281/zenodo.2029296, 2018.
a
Ravela, S. and McLaughlin, D.: Fast ensemble smoothing, Ocean Dynam., 57,
123–134, https://doi.org/10.1007/s10236-006-0098-6, 2007. a
Sakov, P., Counillon, F., Bertino, L., Lisæter, K. A., Oke, P. R., and Korablev, A.: TOPAZ4: an ocean-sea ice data assimilation system for the North Atlantic and Arctic, Ocean Sci., 8, 633–656, https://doi.org/10.5194/os-8-633-2012, 2012. a
Skachko, S., Ménard, R., Errera, Q., Christophe, Y., and Chabrillat, S.: EnKF and 4D-Var data assimilation with chemical transport model BASCOE (version 05.06), Geosci. Model Dev., 9, 2893–2908, https://doi.org/10.5194/gmd-9-2893-2016, 2016. a
Uppala, S. M., Kållberg, P., Simmons, A. J., Andrae, U., Bechtold, V.
D. C., Fiorino, M., Gibson, J., Haseler, J., Hernandez, A., Kelly, G.,
and Woollen, J.: The ERA-40 re-analysis, Q. J. Roy. Meteor.
Soc., 131, 2961–3012, 2005. a
van Velzen, N., Altaf, M. U., and Verlaan, M.: OpenDA-NEMO framework for ocean
data assimilation, Ocean Dynam., 66, 691–702, 2016. a
Wang, X., Parrish, D., Kleist, D., and Whitaker, J.: GSI 3DVar-based
ensemble–variational hybrid data assimilation for NCEP Global Forecast
System: Single-resolution experiments, Mon. Weather Rev., 141,
4098–4117, 2013. a
Whitaker, J. S. and Hamill, T. M.: Ensemble data assimilation without perturbed
observations, Mon. Weather Rev., 130, 1913–1924, 2002. a
Zeng, Y. and Janjić, T.: Study of conservation laws with the local ensemble
transform Kalman filter, Q. J. Roy. Meteor.
Soc., 142, 2359–2372, 2016. a
Zhang, M. and Zhang, F.: E4DVar: Coupling an ensemble Kalman filter with
four-dimensional variational data assimilation in a limited-area weather
prediction model, Mon. Weather Rev., 140, 587–600, 2012. a
Zhu, Y., Todling, R., Guo, J., Cohn, S. E., Navon, I. M., and Yang, Y.: The
GEOS-3 Retrospective Data Assimilation System: The 6-Hour Lag Case, Mon.
Weather Rev., 131, 2129–2150,
https://doi.org/10.1175/1520-0493(2003)131<2129:TGRDAS>2.0.CO;2, 2003. a, b
Short summary
Traditional Kalman smoothers are expensive to apply in large global ocean operational forecast and reanalysis systems. We develop a cost-efficient method to overcome the technical constraints and to improve the performance of existing reanalysis products.
Traditional Kalman smoothers are expensive to apply in large global ocean operational forecast...
