Articles | Volume 18, issue 2
https://doi.org/10.5194/gmd-18-361-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-18-361-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Methods for assessment of models
| 
22 Jan 2025
Methods for assessment of models |
| 22 Jan 2025
| 22 Jan 2025
Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecasts
Andy Richling
CORRESPONDING AUTHOR
Institute of Meteorology, Freie Universität Berlin, Carl-Heinrich-Becker Weg 6–10, 12165 Berlin, Germany
Jens Grieger
Institute of Meteorology, Freie Universität Berlin, Carl-Heinrich-Becker Weg 6–10, 12165 Berlin, Germany
Henning W. Rust
Institute of Meteorology, Freie Universität Berlin, Carl-Heinrich-Becker Weg 6–10, 12165 Berlin, Germany
Related authors
No articles found.
Andreas Trojand, Henning W. Rust, and Uwe Ulbrich
Nat. Hazards Earth Syst. Sci., 25, 2331–2350, https://doi.org/10.5194/nhess-25-2331-2025, https://doi.org/10.5194/nhess-25-2331-2025, 2025
Short summary
Short summary
The study investigates how the intensity of previous windstorm events and the time between two events affect the vulnerability of residential buildings in Germany. By analyzing 23 years of data, it was found that higher intensity of previous events generally reduces vulnerability in subsequent storms, while shorter intervals between events increase vulnerability. The results emphasize the approach of considering vulnerability in risk assessments as temporally dynamic.
Yan Li, Bo Huang, Chunping Tan, Xia Zhang, Francesco Cherubini, and Henning W. Rust
Hydrol. Earth Syst. Sci., 29, 1637–1658, https://doi.org/10.5194/hess-29-1637-2025, https://doi.org/10.5194/hess-29-1637-2025, 2025
Short summary
Short summary
Deforestation has a significant impact on climate, yet its effects on drought remain less understood. This study investigates how deforestation affects drought across various climate zones and timescales. Findings indicate that deforestation leads to drier conditions in tropical regions and wetter conditions in arid areas, with minimal effects in temperate zones. Long-term drought is more affected than short-term drought, offering valuable insights into vegetation–climate interactions.
Elena Xoplaki, Florian Ellsäßer, Jens Grieger, Katrin M. Nissen, Joaquim G. Pinto, Markus Augenstein, Ting-Chen Chen, Hendrik Feldmann, Petra Friederichs, Daniel Gliksman, Laura Goulier, Karsten Haustein, Jens Heinke, Lisa Jach, Florian Knutzen, Stefan Kollet, Jürg Luterbacher, Niklas Luther, Susanna Mohr, Christoph Mudersbach, Christoph Müller, Efi Rousi, Felix Simon, Laura Suarez-Gutierrez, Svenja Szemkus, Sara M. Vallejo-Bernal, Odysseas Vlachopoulos, and Frederik Wolf
Nat. Hazards Earth Syst. Sci., 25, 541–564, https://doi.org/10.5194/nhess-25-541-2025, https://doi.org/10.5194/nhess-25-541-2025, 2025
Short summary
Short summary
Europe frequently experiences compound events, with major impacts. We investigate these events’ interactions, characteristics, and changes over time, focusing on socio-economic impacts in Germany and central Europe. Highlighting 2018’s extreme events, this study reveals impacts on water, agriculture, and forests and stresses the need for impact-focused definitions and better future risk quantification to support adaptation planning.
Bjorn Stevens, Stefan Adami, Tariq Ali, Hartwig Anzt, Zafer Aslan, Sabine Attinger, Jaana Bäck, Johanna Baehr, Peter Bauer, Natacha Bernier, Bob Bishop, Hendryk Bockelmann, Sandrine Bony, Guy Brasseur, David N. Bresch, Sean Breyer, Gilbert Brunet, Pier Luigi Buttigieg, Junji Cao, Christelle Castet, Yafang Cheng, Ayantika Dey Choudhury, Deborah Coen, Susanne Crewell, Atish Dabholkar, Qing Dai, Francisco Doblas-Reyes, Dale Durran, Ayoub El Gaidi, Charlie Ewen, Eleftheria Exarchou, Veronika Eyring, Florencia Falkinhoff, David Farrell, Piers M. Forster, Ariane Frassoni, Claudia Frauen, Oliver Fuhrer, Shahzad Gani, Edwin Gerber, Debra Goldfarb, Jens Grieger, Nicolas Gruber, Wilco Hazeleger, Rolf Herken, Chris Hewitt, Torsten Hoefler, Huang-Hsiung Hsu, Daniela Jacob, Alexandra Jahn, Christian Jakob, Thomas Jung, Christopher Kadow, In-Sik Kang, Sarah Kang, Karthik Kashinath, Katharina Kleinen-von Königslöw, Daniel Klocke, Uta Kloenne, Milan Klöwer, Chihiro Kodama, Stefan Kollet, Tobias Kölling, Jenni Kontkanen, Steve Kopp, Michal Koran, Markku Kulmala, Hanna Lappalainen, Fakhria Latifi, Bryan Lawrence, June Yi Lee, Quentin Lejeun, Christian Lessig, Chao Li, Thomas Lippert, Jürg Luterbacher, Pekka Manninen, Jochem Marotzke, Satoshi Matsouoka, Charlotte Merchant, Peter Messmer, Gero Michel, Kristel Michielsen, Tomoki Miyakawa, Jens Müller, Ramsha Munir, Sandeep Narayanasetti, Ousmane Ndiaye, Carlos Nobre, Achim Oberg, Riko Oki, Tuba Özkan-Haller, Tim Palmer, Stan Posey, Andreas Prein, Odessa Primus, Mike Pritchard, Julie Pullen, Dian Putrasahan, Johannes Quaas, Krishnan Raghavan, Venkatachalam Ramaswamy, Markus Rapp, Florian Rauser, Markus Reichstein, Aromar Revi, Sonakshi Saluja, Masaki Satoh, Vera Schemann, Sebastian Schemm, Christina Schnadt Poberaj, Thomas Schulthess, Cath Senior, Jagadish Shukla, Manmeet Singh, Julia Slingo, Adam Sobel, Silvina Solman, Jenna Spitzer, Philip Stier, Thomas Stocker, Sarah Strock, Hang Su, Petteri Taalas, John Taylor, Susann Tegtmeier, Georg Teutsch, Adrian Tompkins, Uwe Ulbrich, Pier-Luigi Vidale, Chien-Ming Wu, Hao Xu, Najibullah Zaki, Laure Zanna, Tianjun Zhou, and Florian Ziemen
Earth Syst. Sci. Data, 16, 2113–2122, https://doi.org/10.5194/essd-16-2113-2024, https://doi.org/10.5194/essd-16-2113-2024, 2024
Short summary
Short summary
To manage Earth in the Anthropocene, new tools, new institutions, and new forms of international cooperation will be required. Earth Virtualization Engines is proposed as an international federation of centers of excellence to empower all people to respond to the immense and urgent challenges posed by climate change.
Madlen Peter, Henning W. Rust, and Uwe Ulbrich
Nat. Hazards Earth Syst. Sci., 24, 1261–1285, https://doi.org/10.5194/nhess-24-1261-2024, https://doi.org/10.5194/nhess-24-1261-2024, 2024
Short summary
Short summary
The paper introduces a statistical modeling approach describing daily extreme precipitation in Germany more accurately by including changes within the year and between the years simultaneously. The changing seasonality over years is regionally divergent and mainly weak. However, some regions stand out with a more pronounced linear rise of summer intensities, indicating a possible climate change signal. Improved modeling of extreme precipitation is beneficial for risk assessment and adaptation.
Yan Li, Bo Huang, and Henning W. Rust
Hydrol. Earth Syst. Sci., 28, 321–339, https://doi.org/10.5194/hess-28-321-2024, https://doi.org/10.5194/hess-28-321-2024, 2024
Short summary
Short summary
The inconsistent changes in temperature and precipitation induced by forest cover change are very likely to affect drought condition. We use a set of statistical models to explore the relationship between forest cover change and drought change in different timescales and climate zones. We find that the influence of forest cover on droughts varies under different precipitation and temperature quantiles. Forest cover also could modulate the impacts of precipitation and temperature on drought.
Johannes Riebold, Andy Richling, Uwe Ulbrich, Henning Rust, Tido Semmler, and Dörthe Handorf
Weather Clim. Dynam., 4, 663–682, https://doi.org/10.5194/wcd-4-663-2023, https://doi.org/10.5194/wcd-4-663-2023, 2023
Short summary
Short summary
Arctic sea ice loss might impact the atmospheric circulation outside the Arctic and therefore extremes over mid-latitudes. Here, we analyze model experiments to initially assess the influence of sea ice loss on occurrence frequencies of large-scale circulation patterns. Some of these detected circulation changes can be linked to changes in occurrences of European temperature extremes. Compared to future global temperature increases, the sea-ice-related impacts are however of secondary relevance.
Daniel Gliksman, Paul Averbeck, Nico Becker, Barry Gardiner, Valeri Goldberg, Jens Grieger, Dörthe Handorf, Karsten Haustein, Alexia Karwat, Florian Knutzen, Hilke S. Lentink, Rike Lorenz, Deborah Niermann, Joaquim G. Pinto, Ronald Queck, Astrid Ziemann, and Christian L. E. Franzke
Nat. Hazards Earth Syst. Sci., 23, 2171–2201, https://doi.org/10.5194/nhess-23-2171-2023, https://doi.org/10.5194/nhess-23-2171-2023, 2023
Short summary
Short summary
Wind and storms are a major natural hazard and can cause severe economic damage and cost human lives. Hence, it is important to gauge the potential impact of using indices, which potentially enable us to estimate likely impacts of storms or other wind events. Here, we review basic aspects of wind and storm generation and provide an extensive overview of wind impacts and available indices. This is also important to better prepare for future climate change and corresponding changes to winds.
Edmund P. Meredith, Uwe Ulbrich, and Henning W. Rust
Geosci. Model Dev., 16, 851–867, https://doi.org/10.5194/gmd-16-851-2023, https://doi.org/10.5194/gmd-16-851-2023, 2023
Short summary
Short summary
Cell-tracking algorithms allow for the study of properties of a convective cell across its lifetime and, in particular, how these respond to climate change. We investigated whether the design of the algorithm can affect the magnitude of the climate-change signal. The algorithm's criteria for identifying a cell were found to have a strong impact on the warming response. The sensitivity of the warming response to different algorithm settings and cell types should thus be fully explored.
Alberto Caldas-Alvarez, Markus Augenstein, Georgy Ayzel, Klemens Barfus, Ribu Cherian, Lisa Dillenardt, Felix Fauer, Hendrik Feldmann, Maik Heistermann, Alexia Karwat, Frank Kaspar, Heidi Kreibich, Etor Emanuel Lucio-Eceiza, Edmund P. Meredith, Susanna Mohr, Deborah Niermann, Stephan Pfahl, Florian Ruff, Henning W. Rust, Lukas Schoppa, Thomas Schwitalla, Stella Steidl, Annegret H. Thieken, Jordis S. Tradowsky, Volker Wulfmeyer, and Johannes Quaas
Nat. Hazards Earth Syst. Sci., 22, 3701–3724, https://doi.org/10.5194/nhess-22-3701-2022, https://doi.org/10.5194/nhess-22-3701-2022, 2022
Short summary
Short summary
In a warming climate, extreme precipitation events are becoming more frequent. To advance our knowledge on such phenomena, we present a multidisciplinary analysis of a selected case study that took place on 29 June 2017 in the Berlin metropolitan area. Our analysis provides evidence of the extremeness of the case from the atmospheric and the impacts perspectives as well as new insights on the physical mechanisms of the event at the meteorological and climate scales.
Robert Polzin, Annette Müller, Henning Rust, Peter Névir, and Péter Koltai
Nonlin. Processes Geophys., 29, 37–52, https://doi.org/10.5194/npg-29-37-2022, https://doi.org/10.5194/npg-29-37-2022, 2022
Short summary
Short summary
In this study, a recent algorithmic framework called Direct Bayesian Model Reduction (DBMR) is applied which provides a scalable probability-preserving identification of reduced models directly from data. The stochastic method is tested in a meteorological application towards a model reduction to latent states of smaller scale convective activity conditioned on large-scale atmospheric flow.
Noelia Otero, Oscar E. Jurado, Tim Butler, and Henning W. Rust
Atmos. Chem. Phys., 22, 1905–1919, https://doi.org/10.5194/acp-22-1905-2022, https://doi.org/10.5194/acp-22-1905-2022, 2022
Short summary
Short summary
Surface ozone and temperature are strongly dependent and their extremes might be exacerbated by underlying climatological drivers, such as atmospheric blocking. Using an observational data set, we measure the dependence structure between ozone and temperature under the influence of atmospheric blocking. Blocks enhanced the probability of occurrence of compound ozone and temperature extremes over northwestern and central Europe, leading to greater health risks.
Felix S. Fauer, Jana Ulrich, Oscar E. Jurado, and Henning W. Rust
Hydrol. Earth Syst. Sci., 25, 6479–6494, https://doi.org/10.5194/hess-25-6479-2021, https://doi.org/10.5194/hess-25-6479-2021, 2021
Short summary
Short summary
Extreme rainfall events are modeled in this study for different timescales. A new parameterization of the dependence between extreme values and their timescale enables our model to estimate extremes on very short (1 min) and long (5 d) timescales simultaneously. We compare different approaches of modeling this dependence and find that our new model improves performance for timescales between 2 h and 2 d without affecting model performance on other timescales.
Jana Ulrich, Felix S. Fauer, and Henning W. Rust
Hydrol. Earth Syst. Sci., 25, 6133–6149, https://doi.org/10.5194/hess-25-6133-2021, https://doi.org/10.5194/hess-25-6133-2021, 2021
Short summary
Short summary
The characteristics of extreme precipitation on different timescales as well as in different seasons are relevant information, e.g., for designing hydrological structures or managing water supplies. Therefore, our aim is to describe these characteristics simultaneously within one model. We find similar characteristics for short extreme precipitation at all considered stations in Germany but pronounced regional differences with respect to the seasonality of long-lasting extreme events.
Carola Detring, Annette Müller, Lisa Schielicke, Peter Névir, and Henning W. Rust
Weather Clim. Dynam., 2, 927–952, https://doi.org/10.5194/wcd-2-927-2021, https://doi.org/10.5194/wcd-2-927-2021, 2021
Short summary
Short summary
Stationary, long-lasting blocked weather patterns can lead to extreme conditions. Within this study the temporal evolution of the occurrence probability is analyzed, and the onset, decay and transition probabilities of blocking within the past 30 years are modeled. Using Markov models combined with logistic regression, we found large changes in summer, where the probability of transitions to so-called Omega blocks increases strongly, while the unblocked state becomes less probable.
Alexander Pasternack, Jens Grieger, Henning W. Rust, and Uwe Ulbrich
Geosci. Model Dev., 14, 4335–4355, https://doi.org/10.5194/gmd-14-4335-2021, https://doi.org/10.5194/gmd-14-4335-2021, 2021
Short summary
Short summary
Decadal climate ensemble forecasts are increasingly being used to guide adaptation measures. To ensure the applicability of these probabilistic predictions, inherent systematic errors of the prediction system must be adjusted. Since it is not clear which statistical model is optimal for this purpose, we propose a recalibration strategy with a systematic model selection based on non-homogeneous boosting for identifying the most relevant features for both ensemble mean and ensemble spread.
Cited articles
Balmaseda, M. A., Mogensen, K., and Weaver, A. T.: Evaluation of the ECMWF ocean reanalysis system ORAS4, Q. J. Roy. Meteor. Soc., 139, 1132–1161, https://doi.org/10.1002/qj.2063, 2013. a, b
Blyth, C. R.: On Simpson's Paradox and the Sure-Thing Principle, J. Am. Stat. Assoc., 67, 364–366, https://doi.org/10.1080/01621459.1972.10482387, 1972. a
Branković, C. and Palmer, T. N.: Seasonal skill and predictability of ECMWF PROVOST ensembles, Q. J. Roy. Meteor. Soc., 126, 2035–2067, https://doi.org/10.1256/smsqj.56703, 2000. a
Bunzel, F., Notz, D., Baehr, J., Müller, W. A., and Fröhlich, K.: Seasonal climate forecasts significantly affected by observational uncertainty of Arctic sea ice concentration, Geophys. Res. Lett., 43, 852–859, https://doi.org/10.1002/2015GL066928, 2016. a
Dee, D. P., Uppala, S. M., Simmons, A. J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M. A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A. C. M., van de Berg, L., Bidlot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A. J., Haimberger, L., Healy, S. B., Hersbach, H., Hólm, E. V., Isaksen, L., Kållberg, P., Köhler, M., Matricardi, M., McNally, A. P., Monge-Sanz, B. M., Morcrette, J.-J., Park, B.-K., Peubey, C., de Rosnay, P., Tavolato, C., Thépaut, J.-N., and Vitart, F.: The ERA-Interim reanalysis: configuration and performance of the data assimilation system, Q. J. Roy. Meteor. Soc., 137, 553–597, https://doi.org/10.1002/qj.828, 2011. a
Enfield, D. B., Mestas-Nuñez, A. M., and Trimble, P. J.: The Atlantic Multidecadal Oscillation and its relation to rainfall and river flows in the continental U. S., Geophys. Res. Lett., 28, 2077–2080, https://doi.org/10.1029/2000GL012745, 2001. a
Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., and Taylor, K. E.: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization, Geosci. Model Dev., 9, 1937–1958, https://doi.org/10.5194/gmd-9-1937-2016, 2016. a
Ferranti, L., Corti, S., and Janousek, M.: Flow-dependent verification of the ECMWF ensemble over the Euro-Atlantic sector, Q. J. Roy. Meteor. Soc., 141, 916–924, https://doi.org/10.1002/qj.2411, 2015. a
Ferranti, L., Magnusson, L., Vitart, F., and Richardson, D. S.: How far in advance can we predict changes in large-scale flow leading to severe cold conditions over Europe?, Q. J. Roy. Meteor. Soc., 144, 1788–1802, https://doi.org/10.1002/qj.3341, 2018. a
Fetterer, F., Knowles, K., Meier, W. N., Savoie, M., and Windnagel, A. K.: Sea Ice Index, Version 3, National Snow and Ice Data Center [data set], https://doi.org/10.7265/N5K072F8, 2018. a
Frame, T. H. A., Methven, J., Gray, S. L., and Ambaum, M. H. P.: Flow-dependent predictability of the North Atlantic jet, Geophys. Res. Lett., 40, 2411–2416, 2013. a
Frías, M. D., Herrera, S., Cofiño, A. S., and Gutiérrez, J. M.: Assessing the skill of precipitation and temperature seasonal forecasts in Spain: Windows of opportunity related to ENSO events, J. Climate, 23, 209–220, https://doi.org/10.1175/2009JCLI2824.1, 2010. a
Giorgetta, M. A., Jungclaus, J., Reick, C. H., Legutke, S., Bader, J., Böttinger, M., Brovkin, V., Crueger, T., Esch, M., Fieg, K., Glushak, K., Gayler, V., Haak, H., Hollweg, H.-D., Ilyina, T., Kinne, S., Kornblueh, L., Matei, D., Mauritsen, T., Mikolajewicz, U., Mueller, W., Notz, D., Pithan, F., Raddatz, T., Rast, S., Redler, R., Roeckner, E., Schmidt, H., Schnur, R., Segschneider, J., Six, K. D., Stockhause, M., Timmreck, C., Wegner, J., Widmann, H., Wieners, K.-H., Claussen, M., Marotzke, J., and Stevens, B.: Climate and carbon cycle changes from 1850 to 2100 in MPI-ESM simulations for the Coupled Model Intercomparison Project phase 5, J. Adv. Model. Earth Sy., 5, 572–597, https://doi.org/10.1002/jame.20038, 2013. a
Goddard, L. and Dilley, M.: El Niño: Catastrophe or opportunity, J. Climate, 18, 651–665, https://doi.org/10.1175/JCLI-3277.1, 2005. a
Goeber, M., Wilson, C. A., Milton, S. F., and Stephenson, D. B.: Fairplay in the verification of operational quantitative precipitation forecasts, J. Hydrol., 288, 225–236, 2004. a
Grönås, S.: Systematic errors and forecast quality of ECMWF forecasts in different large-scale flow patterns, in: Seminar on Interpretation of Numerical Weather Prediction Products, 13–17 September 1982, Shinfield Park, Reading, ECMWF, 161–206, https://www.ecmwf.int/node/9654 (last access: 29 December 2024), 1982. a
Grönås, S.: A pilot study on the prediction of medium range forecast quality, ECMWF Technical Memoranda, p. 22, https://doi.org/10.21957/ostzejo17, 1985. a
Hamill, T. M. and Juras, J.: Measuring forecast skill: is it real skill or is it the varying climatology?, Q. J. Roy. Meteor. Soc., 132, 2905–2923, https://doi.org/10.1256/qj.06.25, 2006. a, b
Höschel, I., Illing, S., Grieger, J., Ulbrich, U., and Cubasch, U.: On skillful decadal predictions of the subpolar North Atlantic, Meteorol. Z., 28, 417–428, https://doi.org/10.1127/metz/2019/0957, 2019. a
Jones, C., Waliser, D. E., Lau, K. M., and Stern, W.: The Madden–Julian oscillation and its impact on northern hemisphere weather predictability, Mon. Weather Rev., 132, 1462–1471, 2004. a
Jones, C., Hazra, A., and Carvalho, L. M. V.: The Madden-Julian Oscillation and boreal winter forecast skill: An analysis of NCEP CFSv2 reforecasts, J. Climate, 28, 6297–6307, 2015. a
Kadow, C., Illing, S., Kunst, O., Rust, H. W., Pohlmann, H., Müller, W. A., and Cubasch, U.: Evaluation of forecasts by accuracy and spread in the MiKlip decadal climate prediction system, Meteorol. Z., 25, 631–643, https://doi.org/10.1127/metz/2015/0639, 2016. a
Kadow, C., Illing, S., Lucio-Eceiza, E., Bergemann, M., Ramadoss, M., Sommer, P., Kunst, O., Schartner, T., Pankatz, K., Grieger, J., Schuster, M., Richling, A., Thiemann, H., Kirchner, I., Rust, H., Ludwig, T., Cubasch, U., and Ulbrich, U.: Introduction to Freva – A Free Evaluation System Framework for Earth System Modeling, J. Open Res. Softw., 9, 13, https://doi.org/10.5334/jors.253, 2021. a, b, c
Kim, H.-M., Webster, P. J., and Curry, J. A.: Seasonal prediction skill of ECMWF System 4 and NCEP CFSv2 retrospective forecast for the Northern Hemisphere Winter, Clim. Dynam., 39, 2957–2973, https://doi.org/10.1007/s00382-012-1364-6, 2012. a
Kröger, J., Pohlmann, H., Sienz, F., Marotzke, J., Baehr, J., Köhl, A., Modali, K., Polkova, I., Stammer, D., Vamborg, F. S. E., and Müller, W. A.: Full-field initialized decadal predictions with the MPI earth system model: an initial shock in the North Atlantic, Clim. Dynam., 51, 2593–2608, https://doi.org/10.1007/s00382-017-4030-1, 2018. a
Kruschke, T., Rust, H. W., Kadow, C., Müller, W. A., Pohlmann, H., Leckebusch, G. C., and Ulbrich, U.: Probabilistic evaluation of decadal prediction skill regarding Northern Hemisphere winter storms, Meteorol. Z., 25, 721–738, https://doi.org/10.1127/metz/2015/0641, 2016. a, b
Manzanas, R., Frías, M. D., Cofiño, A. S., and Gutiérrez, J. M.: Validation of 40 year multimodel seasonal precipitation forecasts: The role of ENSO on the global skill, J. Geophys. Res.-Atmos., 119, 1708–1719, https://doi.org/10.1002/2013JD020680, 2014. a
Mariotti, A., Baggett, C., Barnes, E. A., Becker, E., Butler, A., Collins, D. C., Dirmeyer, P. A., Ferranti, L., Johnson, N. C., Jones, J., Kirtman, B. P., Lang, A. L., Molod, A., Newman, M., Robertson, A. W., Schubert, S., Waliser, D. E., and Albers, J.: Windows of Opportunity for Skillful Forecasts Subseasonal to Seasonal and Beyond, B. Am. Meteorol. Soc., 101, E608–E625, https://doi.org/10.1175/BAMS-D-18-0326.1, 2020. a, b
Marotzke, J., Müller, W. A., Vamborg, F. S. E., Becker, P., Cubasch, U., Feldmann, H., Kaspar, F., Kottmeier, C., Marini, C., Polkova, I., Prömmel, K., Rust, H. W., Stammer, D., Ulbrich, U., Kadow, C., Köhl, A., Kröger, J., Kruschke, T., Pinto, J. G., Pohlmann, H., Reyers, M., Schröder, M., Sienz, F., Timmreck, C., and Ziese, M.: MiKlip – a National Research Project on Decadal Climate Prediction, B. Am. Meteorol. Soc., 97, 2379–2394, https://doi.org/10.1175/BAMS-D-15-00184.1, 2016. a
Mauritsen, T., Bader, J., Becker, T., Behrens, J., Bittner, M., Brokopf, R., Brovkin, V., Claussen, M., Crueger, T., Esch, M., Fast, I., Fiedler, S., Fläschner, D., Gayler, V., Giorgetta, M., Goll, D. S., Haak, H., Hagemann, S., Hedemann, C., Hohenegger, C., Ilyina, T., Jahns, T., Jimenéz-de-la Cuesta, D., Jungclaus, J., Kleinen, T., Kloster, S., Kracher, D., Kinne, S., Kleberg, D., Lasslop, G., Kornblueh, L., Marotzke, J., Matei, D., Meraner, K., Mikolajewicz, U., Modali, K., Möbis, B., Müller, W. A., Nabel, J. E. M. S., Nam, C. C. W., Notz, D., Nyawira, S.-S., Paulsen, H., Peters, K., Pincus, R., Pohlmann, H., Pongratz, J., Popp, M., Raddatz, T. J., Rast, S., Redler, R., Reick, C. H., Rohrschneider, T., Schemann, V., Schmidt, H., Schnur, R., Schulzweida, U., Six, K. D., Stein, L., Stemmler, I., Stevens, B., von Storch, J.-S., Tian, F., Voigt, A., Vrese, P., Wieners, K.-H., Wilkenskjeld, S., Winkler, A., and Roeckner, E.: Developments in the MPI-M Earth System Model version 1.2 (MPI-ESM1.2) and Its Response to Increasing CO2, J. Adv. Model. Earth Sy., 11, 998–1038, https://doi.org/10.1029/2018MS001400, 2019. a
Miller, D. E. and Wang, Z.: Assessing seasonal predictability sources and windows of high predictability in the climate forecast system, version 2, J. Climate, 32, 1307–1326, https://doi.org/10.1175/JCLI-D-18-0389.1, 2019. a
Morice, C. P., Kennedy, J. J., Rayner, N. A., and Jones, P. D.: Quantifying uncertainties in global and regional temperature change using an ensemble of observational estimates: The HadCRUT4 data set, J. Geophys. Res.-Atmos., 117, D08101, https://doi.org/10.1029/2011JD017187, 2012. a, b
Murphy, A. H.: General Decompositions of MSE-Based Skill Scores: Measures of Some Basic Aspects of Forecast Quality, Mon. Weather Rev., 124, 2353–2369, https://doi.org/10.1175/1520-0493(1996)124<2353:GDOMBS>2.0.CO;2, 1996. a
Müller, W. A., Pohlmann, H., Sienz, F., and Smith, D.: Decadal climate predictions for the period 1901–2010 with a coupled climate model, Geophys. Res. Lett., 41, 2100–2107, https://doi.org/10.1002/2014GL059259, 2014. a, b
Müller, W. A., Jungclaus, J. H., Mauritsen, T., Baehr, J., Bittner, M., Budich, R., Bunzel, F., Esch, M., Ghosh, R., Haak, H., Ilyina, T., Kleine, T., Kornblueh, L., Li, H., Modali, K., Notz, D., Pohlmann, H., Roeckner, E., Stemmler, I., Tian, F., and Marotzke, J.: A Higher-resolution Version of the Max Planck Institute Earth System Model (MPI-ESM1.2-HR), J. Adv. Model. Earth Sy., 10, 1383–1413, https://doi.org/10.1029/2017MS001217, 2018. a, b
Pasternack, A., Bhend, J., Liniger, M. A., Rust, H. W., Müller, W. A., and Ulbrich, U.: Parametric decadal climate forecast recalibration (DeFoReSt 1.0), Geosci. Model Dev., 11, 351–368, https://doi.org/10.5194/gmd-11-351-2018, 2018. a
Pasternack, A., Grieger, J., Rust, H. W., and Ulbrich, U.: Recalibrating decadal climate predictions – what is an adequate model for the drift?, Geosci. Model Dev., 14, 4335–4355, https://doi.org/10.5194/gmd-14-4335-2021, 2021. a
Pearson, K., Lee, A., and Bramley-Moore, L.: VI. Mathematical contributions to the theory of evolution. – VI. Genetic (reproductive) selection: Inheritance of fertility in man, and of fecundity in thoroughbred racehorses, Philos. T. R. Soc. Lond., 192, 257–330, https://doi.org/10.1098/rsta.1899.0006, 1899. a
Peter, M., Rust, H. W., and Ulbrich, U.: Interannual variations in the seasonal cycle of extreme precipitation in Germany and the response to climate change, Nat. Hazards Earth Syst. Sci., 24, 1261–1285, https://doi.org/10.5194/nhess-24-1261-2024, 2024. a
Pohlmann, H., Müller, W. A., Kulkarni, K., Kameswarrao, M., Matei, D., Vamborg, F. S. E., Kadow, C., Illing, S., and Marotzke, J.: Improved forecast skill in the tropics in the new MiKlip decadal climate predictions, Geophys. Res. Lett., 40, 5798–5802, https://doi.org/10.1002/2013GL058051, 2013. a
Pohlmann, H., Müller, W. A., Bittner, M., Hettrich, S., Modali, K., Pankatz, K., and Marotzke, J.: Realistic Quasi-Biennial Oscillation Variability in Historical and Decadal Hindcast Simulations Using CMIP6 Forcing, Geophys. Res. Lett., 46, 14118–14125, https://doi.org/10.1029/2019GL084878, 2019. a, b
Polkova, I., Brune, S., Kadow, C., Romanova, V., Gollan, G., Baehr, J., Glowienka-Hense, R., Greatbatch, R. J., Hense, A., Illing, S., Köhl, A., Kröger, J., Müller, W. A., Pankatz, K., and Stammer, D.: Initialization and Ensemble Generation for Decadal Climate Predictions: A Comparison of Different Methods, J. Adv. Model. Earth Sy., 11, 149–172, https://doi.org/10.1029/2018MS001439, 2019. a
Qin, J. and Robinson, W. A.: The impact of tropical forcing on extratropical predictability in a simple global model, J. Atmos. Sci., 52, 3895–3910, https://doi.org/10.1175/1520-0469(1995)052<3895:TIOTFO>2.0.CO;2, 1995. a
R Core Team: R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, R Core Team [code], https://www.R-project.org/ (last access: 11 January 2024), 2021. a
Richardson, D., Black, A. S., Monselesan, D. P., Moore II, T. S., Risbey, J. S., Schepen, A., Squire, D. T., and Tozer, C. R.: Identifying Periods of Forecast Model Confidence for Improved Subseasonal Prediction of Precipitation, J. Hydrometeorol., 22, 371–385, https://doi.org/10.1175/JHM-D-20-0054.1, 2021. a
Richling, A., Grieger, J., Illing, S., Kadow, C., and Rust, H.: ProblEMS Plugin for Freva (1.6.3) – Probabilistic Ensemble verification for MiKlip using SpecsVerification (1.6.3), Zenodo [code], https://doi.org/10.5281/zenodo.10469658, 2024a. a
Richling, A., Grieger, J., and Rust, H.: Data and software from: Decomposition of skill scores for conditional verification – Impact of AMO phases on the predictability of decadal temperature forecasts, Zenodo [data set], https://doi.org/10.5281/zenodo.13122197, 2024b. a
Schulzweida, U.: CDO User Guide, Zenodo [code], https://doi.org/10.5281/zenodo.10020800, 2023. a
Simpson, E. H.: The Interpretation of Interaction in Contingency Tables, J. R. Stat. Soc. B, 13, 238–241, https://doi.org/10.1111/j.2517-6161.1951.tb00088.x, 1951. a
Tibaldi, S. and Molteni, F.: On the operational predictability of blocking, Tellus, 42A, 343–365, https://doi.org/10.1034/j.1600-0870.1990.t01-2-00003.x, 1990. a
Uppala, S. M., KÅllberg, P. W., Simmons, A. J., Andrae, U., Bechtold, V. D. C., Fiorino, M., Gibson, J. K., Haseler, J., Hernandez, A., Kelly, G. A., Li, X., Onogi, K., Saarinen, S., Sokka, N., Allan, R. P., Andersson, E., Arpe, K., Balmaseda, M. A., Beljaars, A. C. M., Berg, L. V. D., Bidlot, J., Bormann, N., Caires, S., Chevallier, F., Dethof, A., Dragosavac, M., Fisher, M., Fuentes, M., Hagemann, S., Hólm, E., Hoskins, B. J., Isaksen, L., Janssen, P. A. E. M., Jenne, R., Mcnally, A. P., Mahfouf, J.-F., Morcrette, J.-J., Rayner, N. A., Saunders, R. W., Simon, P., Sterl, A., Trenberth, K. E., Untch, A., Vasiljevic, D., Viterbo, P., and Woollen, J.: The ERA-40 re-analysis, Q. J. Roy. Meteor. Soc., 131, 2961–3012, https://doi.org/10.1256/qj.04.176, 2005. a
Yule, G. U.: Notes on the theory of association of attributes in statistics, Biometrika, 2, 121–134, https://doi.org/10.1093/biomet/2.2.121, 1903. a
Zhang, J. and Zhang, R.: On the evolution of Atlantic Meridional Overturning Circulation fingerprint and implications for decadal predictability in the North Atlantic, Geophys. Res. Lett., 42, 5419–5426, https://doi.org/10.1002/2015GL064596, 2015. a, b, c
Short summary
The performance of weather and climate prediction systems is variable in time and space. It is of interest how this performance varies in different situations. We provide a decomposition of a skill score (a measure of forecast performance) as a tool for detailed assessment of performance variability to support model development or forecast improvement. The framework is exemplified with decadal forecasts to assess the impact of different ocean states in the North Atlantic on temperature forecast.
The performance of weather and climate prediction systems is variable in time and space. It is...
