Articles | Volume 17, issue 6
https://doi.org/10.5194/gmd-17-2427-2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-17-2427-2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
ParticleDA.jl v.1.0: a distributed particle-filtering data assimilation package
Department of Statistical Sciences, University College London, London, UK
Centre of Advanced Research Computing (ARC), University College London, London, UK
Matthew M. Graham
Centre of Advanced Research Computing (ARC), University College London, London, UK
Mosè Giordano
Centre of Advanced Research Computing (ARC), University College London, London, UK
Tuomas Koskela
Centre of Advanced Research Computing (ARC), University College London, London, UK
Alexandros Beskos
Department of Statistical Sciences, University College London, London, UK
The Alan Turing Institute, London, UK
Serge Guillas
Department of Statistical Sciences, University College London, London, UK
Centre of Advanced Research Computing (ARC), University College London, London, UK
The Alan Turing Institute, London, UK
Related authors
No articles found.
Ryuichi Kanai, Masashi Kamogawa, Toshiyasu Nagao, Alan Smith, and Serge Guillas
Nat. Hazards Earth Syst. Sci., 22, 849–868, https://doi.org/10.5194/nhess-22-849-2022, https://doi.org/10.5194/nhess-22-849-2022, 2022
Short summary
Short summary
The air pressure created by a tsunami causes a depression in the electron density in the ionosphere. The depression is measured at sparsely distributed, moving GPS satellite locations. We provide an estimate of the volume of the depression. When applied to the 2011 Tohoku-Oki earthquake in Japan, our method can warn of a tsunami event within 15 min of the earthquake, even when using only 5 % of the data. Thus satellite-based warnings could be implemented across the world with our approach.
Dimitra M. Salmanidou, Joakim Beck, Peter Pazak, and Serge Guillas
Nat. Hazards Earth Syst. Sci., 21, 3789–3807, https://doi.org/10.5194/nhess-21-3789-2021, https://doi.org/10.5194/nhess-21-3789-2021, 2021
Short summary
Short summary
The potential of large-magnitude earthquakes in Cascadia poses a significant threat over a populous region of North America. We use statistical emulation to assess the probabilistic tsunami hazard from such events in the region of the city of Victoria, British Columbia. The emulators are built following a sequential design approach for information gain over the input space. To predict the hazard at coastal locations of the region, two families of potential seabed deformation are considered.
Maryam Ilyas, Douglas Nychka, Chris Brierley, and Serge Guillas
Atmos. Meas. Tech., 14, 7103–7121, https://doi.org/10.5194/amt-14-7103-2021, https://doi.org/10.5194/amt-14-7103-2021, 2021
Short summary
Short summary
Instrumental temperature records are fundamental to climate science. There are spatial gaps in the distribution of these measurements across the globe. This lack of spatial coverage introduces coverage error. In this research, a methodology is developed and used to quantify the coverage errors. It results in a data product that, for the first time, provides a full description of both the spatial coverage uncertainties along with the uncertainties in the modeling of these spatial gaps.
Cited articles
Bannister, R. N.: A review of operational methods of variational and ensemble-variational data assimilation, Q. J. Roy. Meteor. Soc., 143, 607–633, https://doi.org/10.1002/qj.2982, 2017. a
Bengtsson, T., Bickel, P., and Li, B.: Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems, in: Probability and statistics: Essays in honor of David A. Freedman, 316–334, Institute of Mathematical Statistics, 2008. a
Berquin, Y. and Zell, A.: A physics perspective on LIDAR data assimilation for mobile robots, Robotica, 40, 862–887, https://doi.org/10.1017/S0263574721000850, 2022. a
Bezanson, J., Edelman, A., Karpinski, S., and Shah, V. B.: Julia: A Fresh Approach to Numerical Computing, SIAM Review, 59, 65–98, https://doi.org/10.1137/141000671, 2017. a
Bickel, P., Li, B., and Bengtsson, T.: Sharp failure rates for the bootstrap particle filter in high dimensions, in: Pushing the limits of contemporary statistics: Contributions in honor of Jayanta K. Ghosh, 318–329, Institute of Mathematical Statistics, 2008. a
Bocquet, M., Pires, C. A., and Wu, L.: Beyond Gaussian statistical modeling in geophysical data assimilation, Mon. Weather Rev., 138, 2997–3023, 2010. a
Buizza, C., Quilodrán Casas, C., Nadler, P., Mack, J., Marrone, S., Titus, Z., Le Cornec, C., Heylen, E., Dur, T., Baca Ruiz, L., Heaney, C., Díaz Lopez, J. A., Kumar, K. S., and Arcucci, R.: Data Learning: Integrating Data Assimilation and Machine Learning, J. Comput. Sci., 58, 101525, https://doi.org/10.1016/j.jocs.2021.101525, 2022. a
Burgers, G., van Leeuwen, P. J., and Evensen, G.: Analysis scheme in the ensemble Kalman filter, Mon. Weather Rev., 126, 1719–1724, 1998. a
Byrne, S., Wilcox, L. C., and Churavy, V.: MPI.jl: Julia bindings for the Message Passing Interface, Proceedings of the JuliaCon Conferences, 1, 68, https://doi.org/10.21105/jcon.00068, 2021. a, b
Carrassi, A., Bocquet, M., Bertino, L., and Evensen, G.: Data assimilation in the geosciences: An overview of methods, issues, and perspectives, Wiley Interdisciplinary Reviews: Climate Change, 9, e535, https://doi.org/10.1002/wcc.535, 2018. a
Chan, T. F., Golub, G. H., and LeVeque, R. J.: Updating formulae and a pairwise algorithm for computing sample variances, in: COMPSTAT 1982 5th Symposium held at Toulouse 1982: Part I: Proceedings in Computational Statistics, 30–41, Springer, 1982. a
Churavy, V., Godoy, W. F., Bauer, C., Ranocha, H., Schlottke-Lakemper, M., Räss, L., Blaschke, J., Giordano, M., Schnetter, E., Omlin, S., Vetter, J. S., and Edelman, A.: Bridging HPC Communities through the Julia Programming Language, arXiv e-prints, https://doi.org/10.48550/arXiv.2211.02740, 2022. a
Cotter, C., Crisan, D., Holm, D., Pan, W., and Shevchenko, I.: Data assimilation for a quasi-geostrophic model with circulation-preserving stochastic transport noise, J. Stat. Phys., 179, 1186–1221, 2020. a
Del Moral, P.: Feynman-Kac formulae, Springer, 2004. a
Dietrich, C. R. and Newsam, G. N.: Fast and exact simulation of stationary Gaussian processes through circulant embedding of the covariance matrix, SIAM J. Sci. Comput., 18, 1088–1107, 1997. a
Douc, R. and Cappé, O.: Comparison of resampling schemes for particle filtering, in: Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 64–69, IEEE, 2005. a
Doucet, A., Godsill, S., and Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering, Stat. Comput., 10, 197–208, https://doi.org/10.1023/A:1008935410038, 2000. a, b, c
Dunbar, O. R. A., Lopez-Gomez, I., Garbuno-Iñigo, A., Huang, D. Z., Bach, E., and long Wu, J.: EnsembleKalmanProcesses.jl: Derivative-free ensemble-based model calibration, J. Open Source Softw., 7, 4869, https://doi.org/10.21105/joss.04869, 2022. a
Evensen, G.: Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics, J. Geophys. Res.-Oceans, 99, 10143–10162, 1994. a
Evensen, G., Vossepoel, F. C., and van Leeuwen, P. J.: Fully Nonlinear Data Assimilation, 95–110, Springer International Publishing, Cham, ISBN 978-3-030-96709-3, https://doi.org/10.1007/978-3-030-96709-3_9, 2022. a
Farchi, A. and Bocquet, M.: Review article: Comparison of local particle filters and new implementations, Nonlin. Processes Geophys., 25, 765–807, https://doi.org/10.5194/npg-25-765-2018, 2018. a, b
Foreman-Mackey, D., Agol, E., Ambikasaran, S., and Angus, R.: Fast and Scalable Gaussian Process Modeling with Applications to Astronomical Time Series, Astronom. J., 154, 220, https://doi.org/10.3847/1538-3881/aa9332, 2017. a
Gailler, A., Hébert, H., Loevenbruck, A., and Hernandez, B.: Simulation systems for tsunami wave propagation forecasting within the French tsunami warning center, Nat. Hazards Earth Syst. Sci., 13, 2465–2482, https://doi.org/10.5194/nhess-13-2465-2013, 2013. a
Giordano, M., Klöwer, M., and Churavy, V.: Productivity meets Performance: Julia on A64FX, in: 2022 IEEE International Conference on Cluster Computing (CLUSTER), 549–555, https://doi.org/10.1109/CLUSTER51413.2022.00072, 2022. a, b
Gordon, N. J., Salmond, D. J., and Smith, A. F.: Novel approach to nonlinear/non-Gaussian Bayesian state estimation, in: IEE Proceedings F (Radar and Signal Processing), IET, 140, 107–113, 1993. a
Goto, C.: Equations of nonlinear dispersive long waves for a large Ursell number, Doboku Gakkai Ronbunshu, 1984, 193–201, 1984. a
Graham, M. M. and Thiery, A. H.: A scalable optimal-transport based local particle filter, arXiv preprint arXiv:1906.00507, 2019. a
Grudzien, C. and Bocquet, M.: A fast, single-iteration ensemble Kalman smoother for sequential data assimilation, Geosci. Model Dev., 15, 7641–7681, https://doi.org/10.5194/gmd-15-7641-2022, 2022. a
Grudzien, C., Merchant, C., and Sandhu, S.: DataAssimilationBenchmarks.jl: a data assimilation research framework., J. Open Source Softw., 7, 4129, https://doi.org/10.21105/joss.04129, 2022. a
Gusman, A. R., Sheehan, A. F., Satake, K., Heidarzadeh, M., Mulia, I. E., and Maeda, T.: Tsunami data assimilation of Cascadia seafloor pressure gauge records from the 2012 Haida Gwaii earthquake, Geophys. Res. Lett., 43, 4189–4196, https://doi.org/10.1002/2016GL068368, 2016. a, b
Hatfield, S.: samhatfield/letkf-speedy, Zenodo [code], https://doi.org/10.5281/zenodo.1198432, 2018. a
Jordán, A., Eyheramendy, S., and Buchner, J.: State-space Representation of Matérn and Damped Simple Harmonic Oscillator Gaussian Processes, Research Notes of the AAS, 5, 107, https://doi.org/10.3847/2515-5172/abfe68, 2021. a
Kondo, K. and Miyoshi, T.: Non-Gaussian statistics in global atmospheric dynamics: a study with a 10 240-member ensemble Kalman filter using an intermediate atmospheric general circulation model, Nonlin. Processes Geophys., 26, 211–225, https://doi.org/10.5194/npg-26-211-2019, 2019. a, b
Koskela, T., Giordano, M., Graham, M., and Giles, D.: Team-RADDISH/ParticleDA.jl: v1.1.0 (v1.1.0), Zenodo [code], https://doi.org/10.5281/zenodo.10814467, 2024. a
Leeuwen, P. J., Künsch, H. R., Nerger, L., Potthast, R., and Reich, S.: Particle filters for high‐dimensional geoscience applications: A review, Q. J. Roy. Meteor. Soc., 145, 2335–2365, https://doi.org/10.1002/qj.3551, 2019. a, b
Lei, J., Bickel, P., and Snyder, C.: Comparison of ensemble Kalman filters under non-Gaussianity, Mon. Weather Rev., 138, 1293–1306, 2010. a
Lorenz, E. N.: Deterministic Nonperiodic Flow, J. Atmos. Sci., 20, 130–141, https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2, 1963. a, b, c
Maeda, T., Obara, K., Shinohara, M., Kanazawa, T., and Uehira, K.: Successive estimation of a tsunami wavefield without earthquake source data: A data assimilation approach toward real-time tsunami forecasting, Geophys. Res. Lett., 42, 7923–7932, https://doi.org/10.1002/2015GL065588, 2015. a
Miyoshi, T., Kondo, K., and Imamura, T.: The 10,240-member ensemble Kalman filtering with an intermediate AGCM, Geophys. Res. Lett., 41, 5264–5271, https://doi.org/10.1002/2014GL060863, 2014. a, b
Molteni, F.: Atmospheric simulations using a GCM with simplified physical parametrizations. I: Model climatology and variability in multi-decadal experiments, Clim. Dynam., 20, 175–191, https://doi.org/10.1007/s00382-002-0268-2, 2003. a, b, c
Nadler, P., Arcucci, R., and Guo, Y. K.: Data assimilation for parameter estimation in economic modelling, Proceedings – 15th International Conference on Signal Image Technology and Internet Based Systems, SISITS 2019, 649–656, https://doi.org/10.1109/SITIS.2019.00106, 2019. a
Rackauckas, C. and Nie, Q.: DifferentialEquations.jl – a performant and feature-rich ecosystem for solving differential equations in Julia, J. Open Res. Softw., 5, 15, https://doi.org/10.5334/jors.151, 2017. a
Robbe, P.: GaussianRandomFields.jl: A package for Gaussian random field generation in Julia, GitHub [code], https://github.com/PieterjanRobbe/GaussianRandomFields.jl (last access: 31 August 2023), 2017. a
Ruzayqat, H., Er-Raiy, A., Beskos, A., Crisan, D., Jasra, A., and Kantas, N.: A lagged particle filter for stable filtering of certain high-dimensional state-space models, SIAM/ASA Journal on Uncertainty Quantification, 10, 1130–1161, 2022. a
Sanpei, A., Okamoto, T., Masamune, S., and Kuroe, Y.: A data-assimilation based method for equilibrium reconstruction of magnetic fusion plasma and its application to reversed field pinch, IEEE Access, 9, 74739–74751, 2021. a
Sunberg, Z., Lasse, P., Bouton, M., Fischer, J., Becker, T., Saba, E., Moss, R., Gupta, J. K., Dressel, L., Kelman, T., Wu, C., and Thibaut, L.: ParticleFilters.jl: Simple particle filter implementation in Julia, https://github.com/JuliaPOMDP/ParticleFilters.jl (last access: 27 February 2023), 2017. a, b
Thépart, J.-N., Vasiljevic, D., Courtier, P., and Pailleux, J.: Variational assimilation of conventional meteorological observations with a multilevel primitive-equation model., Q. J. Roy. Meteor. Soc., 119, 153–186, https://doi.org/10.1002/qj.49711950907, 1993. a
Vetra-Carvalho, S., van Leeuwen, P. J., Nerger, L., Barth, A., Altaf, M. U., Brasseur, P., Kirchgessner, P., and Beckers, J. M.: State-of-the-art stochastic data assimilation methods for high-dimensional non-Gaussian problems, Tellus A, 70, 1–38, https://doi.org/10.1080/16000870.2018.1445364, 2018. a, b
Short summary
Digital twins of physical and human systems informed by real-time data are becoming ubiquitous across a wide range of settings. Progress for researchers is currently limited by a lack of tools to run these models effectively and efficiently. A key challenge is the optimal use of high-performance computing environments. The work presented here focuses on a developed open-source software platform which aims to improve this usage, with an emphasis placed on flexibility, efficiency, and scalability.
Digital twins of physical and human systems informed by real-time data are becoming ubiquitous...