Articles | Volume 17, issue 11
https://doi.org/10.5194/gmd-17-4561-2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-17-4561-2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Methods for assessment of models
| 
10 Jun 2024
Methods for assessment of models |
| 10 Jun 2024
| 10 Jun 2024
EvalHyd v0.1.2: a polyglot tool for the evaluation of deterministic and probabilistic streamflow predictions
Thibault Hallouin
CORRESPONDING AUTHOR
Université Paris-Saclay, INRAE, HYCAR, Antony, France
now at: BRGM (French Geological Survey), Water Resources Unit, Orléans, France
François Bourgin
Université Paris-Saclay, INRAE, HYCAR, Antony, France
Charles Perrin
Université Paris-Saclay, INRAE, HYCAR, Antony, France
Maria-Helena Ramos
Université Paris-Saclay, INRAE, HYCAR, Antony, France
Vazken Andréassian
Université Paris-Saclay, INRAE, HYCAR, Antony, France
Related authors
Thibault Hallouin, Richard J. Ellis, Douglas B. Clark, Simon J. Dadson, Andrew G. Hughes, Bryan N. Lawrence, Grenville M. S. Lister, and Jan Polcher
Geosci. Model Dev., 15, 9177–9196, https://doi.org/10.5194/gmd-15-9177-2022, https://doi.org/10.5194/gmd-15-9177-2022, 2022
Short summary
Short summary
A new framework for modelling the water cycle in the land system has been implemented. It considers the hydrological cycle as three interconnected components, bringing flexibility in the choice of the physical processes and their spatio-temporal resolutions. It is designed to foster collaborations between land surface, hydrological, and groundwater modelling communities to develop the next-generation of land system models for integration in Earth system models.
Annie Y.-Y. Chang, Shaun Harrigan, Maria-Helena Ramos, Massimiliano Zappa, Christian M. Grams, Daniela I. V. Domeisen, and Konrad Bogner
EGUsphere, https://doi.org/10.5194/egusphere-2025-3411, https://doi.org/10.5194/egusphere-2025-3411, 2025
This preprint is open for discussion and under review for Hydrology and Earth System Sciences (HESS).
Short summary
Short summary
This study presents a machine learning-aided hybrid forecasting framework to improve early warnings of low flows in the European Alps. It combines weather regime information, streamflow observations, and model simulations (EFAS). Even using only weather regime data improves predictions over climatology, while integrating different data sources yields the best result, emphasizing the value of integrating diverse data sources.
Taha-Abderrahman El Ouahabi, François Bourgin, Charles Perrin, and Vazken Andréassian
EGUsphere, https://doi.org/10.5194/egusphere-2025-3586, https://doi.org/10.5194/egusphere-2025-3586, 2025
This preprint is open for discussion and under review for Hydrology and Earth System Sciences (HESS).
Short summary
Short summary
To improve hydrological uncertainty estimation, recent studies have explored machine learning (ML)-based post-processing approaches. Among these, quantile random forests (QRF) are increasingly used for their balance between interpretability and performance. We develop a hydrologically informed QRF trained in a multi-site setting. Our results show that the regional QRF approach is beneficial, particularly in catchments where local information is insufficient.
Eric Sauquet, Guillaume Evin, Sonia Siauve, Ryma Aissat, Patrick Arnaud, Maud Bérel, Jérémie Bonneau, Flora Branger, Yvan Caballero, François Colléoni, Agnès Ducharne, Joël Gailhard, Florence Habets, Frédéric Hendrickx, Louis Héraut, Benoît Hingray, Peng Huang, Tristan Jaouen, Alexis Jeantet, Sandra Lanini, Matthieu Le Lay, Claire Magand, Louise Mimeau, Céline Monteil, Simon Munier, Charles Perrin, Olivier Robelin, Fabienne Rousset, Jean-Michel Soubeyroux, Laurent Strohmenger, Guillaume Thirel, Flore Tocquer, Yves Tramblay, Jean-Pierre Vergnes, and Jean-Philippe Vidal
EGUsphere, https://doi.org/10.5194/egusphere-2025-1788, https://doi.org/10.5194/egusphere-2025-1788, 2025
This preprint is open for discussion and under review for Hydrology and Earth System Sciences (HESS).
Short summary
Short summary
The Explore2 project has provided an unprecedented set of hydrological projections in terms of the number of hydrological models used and the spatial and temporal resolution. The results have been made available through various media. Under the high-emission scenario, the hydrological models mostly agree on the decrease in seasonal flows in the south of France, confirming its hotspot status, and on the decrease in summer flows throughout France, with the exception of the northern part of France.
Olivier Delaigue, Guilherme Mendoza Guimarães, Pierre Brigode, Benoît Génot, Charles Perrin, Jean-Michel Soubeyroux, Bruno Janet, Nans Addor, and Vazken Andréassian
Earth Syst. Sci. Data, 17, 1461–1479, https://doi.org/10.5194/essd-17-1461-2025, https://doi.org/10.5194/essd-17-1461-2025, 2025
Short summary
Short summary
This dataset covers 654 rivers all flowing in France. The provided time series and catchment attributes will be of interest to those modelers wishing to analyze hydrological behavior and perform model assessments.
Vazken Andréassian, Guilherme Mendoza Guimarães, Alban de Lavenne, and Julien Lerat
EGUsphere, https://doi.org/10.5194/egusphere-2025-414, https://doi.org/10.5194/egusphere-2025-414, 2025
Short summary
Short summary
Using 4122 catchments from four continents, we investigate how annual streamflow depends on climate variables (rainfall and potential evaporation) and on the season when precipitation occurs, using and index representing the synchronicity between precipitation and potential evaporation. In all countries and under the main climates represented, synchronicity is, after precipitation, the second most important factor to explain annual streamflow variations.
Léonard Santos, Vazken Andréassian, Torben O. Sonnenborg, Göran Lindström, Alban de Lavenne, Charles Perrin, Lila Collet, and Guillaume Thirel
Hydrol. Earth Syst. Sci., 29, 683–700, https://doi.org/10.5194/hess-29-683-2025, https://doi.org/10.5194/hess-29-683-2025, 2025
Short summary
Short summary
This work investigates how hydrological models are transferred to a period in which climate conditions are different to the ones of the period in which they were set up. The robustness assessment test built to detect dependencies between model error and climatic drivers was applied to three hydrological models in 352 catchments in Denmark, France and Sweden. Potential issues are seen in a significant number of catchments for the models, even though the catchments differ for each model.
Guillaume Thirel, Léonard Santos, Olivier Delaigue, and Charles Perrin
Hydrol. Earth Syst. Sci., 28, 4837–4860, https://doi.org/10.5194/hess-28-4837-2024, https://doi.org/10.5194/hess-28-4837-2024, 2024
Short summary
Short summary
We discuss how mathematical transformations impact calibrated hydrological model simulations. We assess how 11 transformations behave over the complete range of streamflows. Extreme transformations lead to models that are specialized for extreme streamflows but show poor performance outside the range of targeted streamflows and are less robust. We show that no a priori assumption about transformations can be taken as warranted.
Ralph Bathelemy, Pierre Brigode, Vazken Andréassian, Charles Perrin, Vincent Moron, Cédric Gaucherel, Emmanuel Tric, and Dominique Boisson
Earth Syst. Sci. Data, 16, 2073–2098, https://doi.org/10.5194/essd-16-2073-2024, https://doi.org/10.5194/essd-16-2073-2024, 2024
Short summary
Short summary
The aim of this work is to provide the first hydroclimatic database for Haiti, a Caribbean country particularly vulnerable to meteorological and hydrological hazards. The resulting database, named Simbi, provides hydroclimatic time series for around 150 stations and 24 catchment areas.
Cyril Thébault, Charles Perrin, Vazken Andréassian, Guillaume Thirel, Sébastien Legrand, and Olivier Delaigue
Hydrol. Earth Syst. Sci., 28, 1539–1566, https://doi.org/10.5194/hess-28-1539-2024, https://doi.org/10.5194/hess-28-1539-2024, 2024
Short summary
Short summary
Streamflow forecasting is useful for many applications, ranging from population safety (e.g. floods) to water resource management (e.g. agriculture or hydropower). To this end, hydrological models must be optimized. However, a model is inherently wrong. This study aims to analyse the contribution of a multi-model approach within a variable spatial framework to improve streamflow simulations. The underlying idea is to take advantage of the strength of each modelling framework tested.
Laurent Strohmenger, Eric Sauquet, Claire Bernard, Jérémie Bonneau, Flora Branger, Amélie Bresson, Pierre Brigode, Rémy Buzier, Olivier Delaigue, Alexandre Devers, Guillaume Evin, Maïté Fournier, Shu-Chen Hsu, Sandra Lanini, Alban de Lavenne, Thibault Lemaitre-Basset, Claire Magand, Guilherme Mendoza Guimarães, Max Mentha, Simon Munier, Charles Perrin, Tristan Podechard, Léo Rouchy, Malak Sadki, Myriam Soutif-Bellenger, François Tilmant, Yves Tramblay, Anne-Lise Véron, Jean-Philippe Vidal, and Guillaume Thirel
Hydrol. Earth Syst. Sci., 27, 3375–3391, https://doi.org/10.5194/hess-27-3375-2023, https://doi.org/10.5194/hess-27-3375-2023, 2023
Short summary
Short summary
We present the results of a large visual inspection campaign of 674 streamflow time series in France. The objective was to detect non-natural records resulting from instrument failure or anthropogenic influences, such as hydroelectric power generation or reservoir management. We conclude that the identification of flaws in flow time series is highly dependent on the objectives and skills of individual evaluators, and we raise the need for better practices for data cleaning.
Maryse Charpentier-Noyer, Daniela Peredo, Axelle Fleury, Hugo Marchal, François Bouttier, Eric Gaume, Pierre Nicolle, Olivier Payrastre, and Maria-Helena Ramos
Nat. Hazards Earth Syst. Sci., 23, 2001–2029, https://doi.org/10.5194/nhess-23-2001-2023, https://doi.org/10.5194/nhess-23-2001-2023, 2023
Short summary
Short summary
This paper proposes a methodological framework designed for event-based evaluation in the context of an intense flash-flood event. The evaluation adopts the point of view of end users, with a focus on the anticipation of exceedances of discharge thresholds. With a study of rainfall forecasts, a discharge evaluation and a detailed look at the forecast hydrographs, the evaluation framework should help in drawing robust conclusions about the usefulness of new rainfall ensemble forecasts.
Thibault Hallouin, Richard J. Ellis, Douglas B. Clark, Simon J. Dadson, Andrew G. Hughes, Bryan N. Lawrence, Grenville M. S. Lister, and Jan Polcher
Geosci. Model Dev., 15, 9177–9196, https://doi.org/10.5194/gmd-15-9177-2022, https://doi.org/10.5194/gmd-15-9177-2022, 2022
Short summary
Short summary
A new framework for modelling the water cycle in the land system has been implemented. It considers the hydrological cycle as three interconnected components, bringing flexibility in the choice of the physical processes and their spatio-temporal resolutions. It is designed to foster collaborations between land surface, hydrological, and groundwater modelling communities to develop the next-generation of land system models for integration in Earth system models.
Alban de Lavenne, Vazken Andréassian, Louise Crochemore, Göran Lindström, and Berit Arheimer
Hydrol. Earth Syst. Sci., 26, 2715–2732, https://doi.org/10.5194/hess-26-2715-2022, https://doi.org/10.5194/hess-26-2715-2022, 2022
Short summary
Short summary
A watershed remembers the past to some extent, and this memory influences its behavior. This memory is defined by the ability to store past rainfall for several years. By releasing this water into the river or the atmosphere, it tends to forget. We describe how this memory fades over time in France and Sweden. A few watersheds show a multi-year memory. It increases with the influence of groundwater or dry conditions. After 3 or 4 years, they behave independently of the past.
Antoine Pelletier and Vazken Andréassian
Hydrol. Earth Syst. Sci., 26, 2733–2758, https://doi.org/10.5194/hess-26-2733-2022, https://doi.org/10.5194/hess-26-2733-2022, 2022
Short summary
Short summary
A large part of the water cycle takes place underground. In many places, the soil stores water during the wet periods and can release it all year long, which is particularly visible when the river level is low. Modelling tools that are used to simulate and forecast the behaviour of the river struggle to represent this. We improved an existing model to take underground water into account using measurements of the soil water content. Results allow us make recommendations for model users.
Emixi Sthefany Valdez, François Anctil, and Maria-Helena Ramos
Hydrol. Earth Syst. Sci., 26, 197–220, https://doi.org/10.5194/hess-26-197-2022, https://doi.org/10.5194/hess-26-197-2022, 2022
Short summary
Short summary
We investigated how a precipitation post-processor interacts with other tools for uncertainty quantification in a hydrometeorological forecasting chain. Four systems were implemented to generate 7 d ensemble streamflow forecasts, which vary from partial to total uncertainty estimation. Overall analysis showed that post-processing and initial condition estimation ensure the most skill improvements, in some cases even better than a system that considers all sources of uncertainty.
Paul Royer-Gaspard, Vazken Andréassian, and Guillaume Thirel
Hydrol. Earth Syst. Sci., 25, 5703–5716, https://doi.org/10.5194/hess-25-5703-2021, https://doi.org/10.5194/hess-25-5703-2021, 2021
Short summary
Short summary
Most evaluation studies based on the differential split-sample test (DSST) endorse the consensus that rainfall–runoff models lack climatic robustness. In this technical note, we propose a new performance metric to evaluate model robustness without applying the DSST and which can be used with a single hydrological model calibration. Our work makes it possible to evaluate the temporal transferability of any hydrological model, including uncalibrated models, at a very low computational cost.
Pierre Nicolle, Vazken Andréassian, Paul Royer-Gaspard, Charles Perrin, Guillaume Thirel, Laurent Coron, and Léonard Santos
Hydrol. Earth Syst. Sci., 25, 5013–5027, https://doi.org/10.5194/hess-25-5013-2021, https://doi.org/10.5194/hess-25-5013-2021, 2021
Short summary
Short summary
In this note, a new method (RAT) is proposed to assess the robustness of hydrological models. The RAT method is particularly interesting because it does not require multiple calibrations (it is therefore applicable to uncalibrated models), and it can be used to determine whether a hydrological model may be safely used for climate change impact studies. Success at the robustness assessment test is a necessary (but not sufficient) condition of model robustness.
Nabil Hocini, Olivier Payrastre, François Bourgin, Eric Gaume, Philippe Davy, Dimitri Lague, Lea Poinsignon, and Frederic Pons
Hydrol. Earth Syst. Sci., 25, 2979–2995, https://doi.org/10.5194/hess-25-2979-2021, https://doi.org/10.5194/hess-25-2979-2021, 2021
Short summary
Short summary
Efficient flood mapping methods are needed for large-scale, comprehensive identification of flash flood inundation hazards caused by small upstream rivers. An evaluation of three automated mapping approaches of increasing complexity, i.e., a digital terrain model (DTM) filling and two 1D–2D hydrodynamic approaches, is presented based on three major flash floods in southeastern France. The results illustrate some limits of the DTM filling method and the value of using a 2D hydrodynamic approach.
Manon Cassagnole, Maria-Helena Ramos, Ioanna Zalachori, Guillaume Thirel, Rémy Garçon, Joël Gailhard, and Thomas Ouillon
Hydrol. Earth Syst. Sci., 25, 1033–1052, https://doi.org/10.5194/hess-25-1033-2021, https://doi.org/10.5194/hess-25-1033-2021, 2021
Pierre Nicolle, François Besson, Olivier Delaigue, Pierre Etchevers, Didier François, Matthieu Le Lay, Charles Perrin, Fabienne Rousset, Dominique Thiéry, François Tilmant, Claire Magand, Timothée Leurent, and Élise Jacob
Proc. IAHS, 383, 381–389, https://doi.org/10.5194/piahs-383-381-2020, https://doi.org/10.5194/piahs-383-381-2020, 2020
Cited articles
Anctil, F. and Ramos, M.-H.: Verification Metrics for Hydrological Ensemble Forecasts, Springer Berlin Heidelberg, Berlin, Heidelberg, 1–30, ISBN 978-3-642-40457-3, https://doi.org/10.1007/978-3-642-40457-3_3-1, 2017. a, b
Barnston, A. G.: Correspondence among the correlation, RMSE, and Heidke forecast verification measures; refinement of the Heidke score, Weather Forecast., 7, 699–709, https://doi.org/10.1175/1520-0434(1992)007<0699:CATCRA>2.0.CO;2, 1992. a
Bellier, J., Zin, I., and Bontron, G.: Sample Stratification in Verification of Ensemble Forecasts of Continuous Scalar Variables: Potential Benefits and Pitfalls, Mon. Weather Rev., 145, 3529–3544, https://doi.org/10.1175/MWR-D-16-0487.1, 2017. a, b
Beven, K. and Young, P.: A guide to good practice in modeling semantics for authors and referees, Water Resour. Res., 49, 5092–5098, https://doi.org/10.1002/wrcr.20393, 2013. a, b
Bourgin, F., Andréassian, V., Perrin, C., and Oudin, L.: Transferring global uncertainty estimates from gauged to ungauged catchments, Hydrol. Earth Syst. Sci., 19, 2535–2546, https://doi.org/10.5194/hess-19-2535-2015, 2015. a
Brier, G. W.: Verification of forecasts expressed in terms of probability, Mon. Weather Rev., 78, 1–3, https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2, 1950. a, b
Brown, J. D., Demargne, J., Seo, D.-J., and Liu, Y.: The Ensemble Verification System (EVS): A software tool for verifying ensemble forecasts of hydrometeorological and hydrologic variables at discrete locations, Environ. Model. Softw., 25, 854–872, https://doi.org/10.1016/j.envsoft.2010.01.009, 2010. a, b
Candille, G. and Talagrand, O.: Evaluation of probabilistic prediction systems for a scalar variable, Q. J. Roy. Meteor. Soc., 131, 2131–2150, https://doi.org/10.1256/qj.04.71, 2005. a
Casati, B.: Comment on egusphere-2023-1424, https://doi.org/10.5194/egusphere-2023-1424-RC1, 2023. a, b
Casati, B., Wilson, L. J., Stephenson, D. B., Nurmi, P., Ghelli, A., Pocernich, M., Damrath, U., Ebert, E. E., Brown, B. G., and Mason, S.: Forecast verification: current status and future directions, Meteorol. Appl., 15, 3–18, https://doi.org/10.1002/met.52, 2008. a
Casati, B., Dorninger, M., Coelho, C. A. S., Ebert, E. E., Marsigli, C., Mittermaier, M. P., and Gilleland, E.: The 2020 International Verification Methods Workshop Online: Major Outcomes and Way Forward, B. Am. Meteorol. Soc., 103, E899–E910, https://doi.org/10.1175/BAMS-D-21-0126.1, 2022. a
Clark, M. P., Vogel, R. M., Lamontagne, J. R., Mizukami, N., Knoben, W. J. M., Tang, G., Gharari, S., Freer, J. E., Whitfield, P. H., Shook, K. R., and Papalexiou, S. M.: The Abuse of Popular Performance Metrics in Hydrologic Modeling, Water Resour. Res., 57, e2020WR029001, https://doi.org/10.1029/2020WR029001, 2021. a
Crochemore, L., Perrin, C., Andréassian, V., Ehret, U., Seibert, S. P., Grimaldi, S., Gupta, H., and Paturel, J.-E.: Comparing expert judgement and numerical criteria for hydrograph evaluation, Hydrol. Sci. J., 60, 402–423, https://doi.org/10.1080/02626667.2014.903331, 2015. a
Gao, Y., Merz, C., Lischeid, G., and Schneider, M.: A review on missing hydrological data processing, Environ. Earth Sci., 77, 47, https://doi.org/10.1007/s12665-018-7228-6, 2018. a
Garcia, F., Folton, N., and Oudin, L.: Which objective function to calibrate rainfall–runoff models for low-flow index simulations?, Hydrol. Sci. J., 62, 1149–1166, https://doi.org/10.1080/02626667.2017.1308511, 2017. a
Gneiting, T. and Raftery, A. E.: Strictly Proper Scoring Rules, Prediction, and Estimation, J. Am. Stat. A., 102, 359–378, https://doi.org/10.1198/016214506000001437, 2007. a, b
Gneiting, T. and Ranjan, R.: Comparing Density Forecasts Using Threshold-and Quantile-Weighted Scoring Rules, J. Bus. Econ. Stat., 29, 411–422, 2011. a
Gneiting, T., Stanberry, L., Grimit, E., Held, L., and Johnson, N.: Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds, TEST, 17, 211–235, https://doi.org/10.1007/s11749-008-0114-x, 2008. a, b
Gupta, H. V., Kling, H., Yilmaz, K. K., and Martinez, G. F.: Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling, J. Hydrol., 377, 80–91, https://doi.org/10.1016/j.jhydrol.2009.08.003, 2009. a, b, c, d
Gupta, H. V., Perrin, C., Blöschl, G., Montanari, A., Kumar, R., Clark, M., and Andréassian, V.: Large-sample hydrology: a need to balance depth with breadth, Hydrol. Earth Syst. Sci., 18, 463–477, https://doi.org/10.5194/hess-18-463-2014, 2014. a
Hall, C. A., Saia, S. M., Popp, A. L., Dogulu, N., Schymanski, S. J., Drost, N., van Emmerik, T., and Hut, R.: A hydrologist's guide to open science, Hydrol. Earth Syst. Sci., 26, 647–664, https://doi.org/10.5194/hess-26-647-2022, 2022. a
Hallouin, T.: Data and code to reproduce figures in the article on evalhyd, Zenodo [code and data], https://doi.org/10.5281/zenodo.11059148, 2024. a
Hallouin, T. and Bourgin, F.: evalhyd: A polyglot tool for the evaluation of deterministic and probabilistic streamflow predictions, HAL INRAE [code], https://hal.inrae.fr/hal-04088473, 2024a. a
Hallouin, T. and Bourgin, F.: evalhyd HTML documentation, Software Heritage [data set], https://archive.softwareheritage.org/swh:1:snp:06bf77ee55040ed205e757b513a0807c9209770f;origin=https://github.com/hydroGR/evalhyd (last access: 25 April 2024), 2024b. a
Hamill, T. M. and Juras, J.: Measuring forecast skill: is it real skill or is it the varying climatology?, Q. J. Roy. Meteor. Soc., 132, 2905–2923, https://doi.org/10.1256/qj.06.25, 2006. a
Harrigan, S., Zsoter, E., Barnard, C., Wetterhall, F., Ferrario, I., Mazzetti, C., Alfieri, L., Salamon, P., and Prudhomme, C.: River discharge and related historical data from the Global Flood Awareness System. v2.1, Copernicus Climate Change Service (C3S) Climate Data Store (CDS) [data set], https://doi.org/10.24381/cds.a4fdd6b9, 2021. a, b
Hersbach, H.: Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems, Weather Forecast., 15, 559–570, https://doi.org/10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2, 2000. a, b
Huang, Z. and Zhao, T.: Predictive performance of ensemble hydroclimatic forecasts: Verification metrics, diagnostic plots and forecast attributes, Wiley Interdisciplinary Reviews-Water, 9, e1580, https://doi.org/10.1002/wat2.1580, 2022. a, b
Hutton, C., Wagener, T., Freer, J., Han, D., Duffy, C., and Arheimer, B.: Most computational hydrology is not reproducible, so is it really science?, Water Resour. Research, 52, 7548–7555, https://doi.org/10.1002/2016WR019285, 2016. a, b
Kling, H., Fuchs, M., and Paulin, M.: Runoff conditions in the upper Danube basin under an ensemble of climate change scenarios, J. Hydrol., 424–425, 264–277, https://doi.org/10.1016/j.jhydrol.2012.01.011, 2012. a, b
Knoben, W. J. M., Freer, J. E., and Woods, R. A.: Technical note: Inherent benchmark or not? Comparing Nash–Sutcliffe and Kling–Gupta efficiency scores, Hydrol. Earth Syst. Sci., 23, 4323–4331, https://doi.org/10.5194/hess-23-4323-2019, 2019. a
Knoben, W. J. M., Clark, M. P., Bales, J., Bennett, A., Gharari, S., Marsh, C. B., Nijssen, B., Pietroniro, A., Spiteri, R. J., Tang, G., Tarboton, D. G., and Wood, A. W.: Community Workflows to Advance Reproducibility in Hydrologic Modeling: Separating Model-Agnostic and Model-Specific Configuration Steps in Applications of Large-Domain Hydrologic Models, Water Resour. Res., 58, e2021WR031753, https://doi.org/10.1029/2021WR031753, 2022. a
Krause, P., Boyle, D. P., and Bäse, F.: Comparison of different efficiency criteria for hydrological model assessment, Adv. Geosci., 5, 89–97, https://doi.org/10.5194/adgeo-5-89-2005, 2005. a
Michie, D.: “Memo” Functions and Machine Learning, Nature, 218, 19–22, https://doi.org/10.1038/218019a0, 1968. a
Moriasi, D. N., Arnold, J. G., Van Liew, M. W., Bingner, R. L., Harmel, R. D., and Veith, T. L.: Model evaluation guidelines for systematic quantification of accuracy in watershed simulations, Transactions of the ASABE, 50, 885–900, https://doi.org/10.13031/2013.23153, 2007. a, b, c
Nash, J. and Sutcliffe, J.: River flow forecasting through conceptual models part I – A discussion of principles, J. Hydrol., 10, 282–290, https://doi.org/10.1016/0022-1694(70)90255-6, 1970. a, b, c
Oudin, L., Andréassian, V., Mathevet, T., Perrin, C., and Michel, C.: Dynamic averaging of rainfall-runoff model simulations from complementary model parameterizations, Water Resour. Res., 42, W07410, https://doi.org/10.1029/2005WR004636, 2006. a
Pechlivanidis, I. G., Jackson, B., McMillan, H., and Gupta, H.: Use of an entropy-based metric in multiobjective calibration to improve model performance, Water Resour. Res., 50, 8066–8083, https://doi.org/10.1002/2013WR014537, 2014. a
Pushpalatha, R., Perrin, C., Moine, N. L., and Andréassian, V.: A review of efficiency criteria suitable for evaluating low-flow simulations, J. Hydrol., 420–421, 171–182, https://doi.org/10.1016/j.jhydrol.2011.11.055, 2012. a, b
Renard, B., Kavetski, D., Kuczera, G., Thyer, M., and Franks, S. W.: Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors, Water Resour. Res., 46, W05521, https://doi.org/10.1029/2009WR008328, 2010. a
Santos, L., Thirel, G., and Perrin, C.: Technical note: Pitfalls in using log-transformed flows within the KGE criterion, Hydrol. Earth Syst. Sci., 22, 4583–4591, https://doi.org/10.5194/hess-22-4583-2018, 2018. a
Schaake, J. C., Hamill, T. M., Buizza, R., and Clark, M.: HEPEX: The Hydrological Ensemble Prediction Experiment, B. Am. Meteorol. Soc., 88, 1541–1548, https://doi.org/10.1175/BAMS-88-10-1541, 2007. a
Slater, L. J., Thirel, G., Harrigan, S., Delaigue, O., Hurley, A., Khouakhi, A., Prosdocimi, I., Vitolo, C., and Smith, K.: Using R in hydrology: a review of recent developments and future directions, Hydrol. Earth Syst. Sci., 23, 2939–2963, https://doi.org/10.5194/hess-23-2939-2019, 2019. a
Stagge, J. H., Rosenberg, D. E., Abdallah, A. M., Akbar, H., Attallah, N. A., and James, R.: Assessing data availability and research reproducibility in hydrology and water resources, Sci. Data, 6, 1–12, 2019. a
Talagrand, O., Vautard, R., and Strauss, B.: Evaluation of probabilistic prediction systems, 1–26, ECMWF, Shinfield Park, Reading, 1997. a
Thébault, C.: Quels apports d'une approche multi-modèle semi-distribuée pour la prévision des débits?, Theses, Sorbonne Université, https://theses.hal.science/tel-04519745 (last access: 25 April 2024), 2023. a
Weisheimer, A. and Palmer, T. N.: On the reliability of seasonal climate forecasts, J. R. Soc. Interface, 11, 20131162, https://doi.org/10.1098/rsif.2013.1162, 2014. a
Willmott, C. J. and Matsuura, K.: Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance, Climate Res., 30, 79–82, https://doi.org/10.3354/cr030079, 2005. a, b
Winkler, R. L. and Murphy, A. H.: Use of probabilities in forecasts of maximum and minimum temperatures, Meteorological Magazine, 108, 317–329, 1979. a
Yuan, X. and Wood, E. F.: On the clustering of climate models in ensemble seasonal forecasting, Geophys. Res. Lett., 39, L18701, https://doi.org/10.1029/2012GL052735, 2012. a
Zsoter, E., Harrigan, S., Barnard, C., Blick, M., Ferrario, I., Wetterhall, F., and Prudhomme, C.: Reforecasts of river discharge and related data by the Global Flood Awareness System. v2.2, Copernicus Climate Change Service (C3S) Climate Data Store (CDS) [data set], https://doi.org/10.24381/cds.2d78664e, 2020. a, b
Short summary
The evaluation of the quality of hydrological model outputs against streamflow observations is widespread in the hydrological literature. In order to improve on the reproducibility of published studies, a new evaluation tool dedicated to hydrological applications is presented. It is open source and usable in a variety of programming languages to make it as accessible as possible to the community. Thus, authors and readers alike can use the same tool to produce and reproduce the results.
The evaluation of the quality of hydrological model outputs against streamflow observations is...
