Global Sensitivity Analysis of the distributed hydrologic model ParFlow-CLM (V3.6.0)
Abstract. The integrated distributed hydrological model ParFlow-CLM was used to predict water and energy transport between subsurface, land surface, and atmosphere for the Stettbach headwater catchment, Germany. Based on this model, a global sensitivity analysis was performed using the Latin-Hypercube (LH) sampling strategy followed by the One-factor-At-a-Time (OAT) method to identify the most influential and interactive parameters affecting the main hydrologic processes. In total 12 parameters were evaluated including soil hydraulic properties, storage, Manning coefficient, leaf area index, stem area index, and aerodynamic resistance that characterize water and energy fluxes in soil and vegetation. In addition, the sensitivity analysis was also carried out for different slopes and meteorological conditions to test the transferability of the results to regions with other topographies and climates. Our results show that the simulated energy fluxes, i.e. latent heat flux and sensible heat flux are sensitive to the parameters such as wilting point, leaf area index, and stem area index, especially for steep slope and subarctic climate conditions. The simulated soil evaporation, plant transpiration, infiltration, and runoff, are most sensitive to soil porosity, the van Genuchten parameter n representing the soil pore size distribution, soil wilting point, and leaf area index. The subsurface soil water storage and groundwater storage are most sensitive to soil porosity, while the surface water storage was most sensitive to the soil roughness parameter. For the different slope and climate conditions, the rank order of input parameter sensitivity is consistent, but the magnitude of parameter sensitivity is very different. The strongest deviation in parameter sensitivity occurred for sensible heat flux under the different slope conditions as well as for transpiration under different climate conditions. Overall, this study provides an insight into the most important input parameters that control hydrological fluxes and how the simulated variables vary with the change in parameter values, which can improve our understanding of the key processes in the model and help us to reduce the computational demands of completing multiple simulations of expensive domains.
This preprint has been withdrawn.
Wei Qu et al.
Wei Qu et al.
SA analysis data and input files https://doi.org/10.5281/zenodo.6553492
Model code and software
ParFlow-CLM model https://doi.org/10.5281/zenodo.4639761
Wei Qu et al.
Viewed (geographical distribution)
In the manuscript by Wei et al, the authors apply global sensitivity analysis (GSA) using Latin-Hypercube One-factor-at-A-Time method to the distributed coupled model ParFlow-CLM. In general, I think that applications of GSA to complex and spatially distributed model are welcome contributions, given the challenges related to the long model run time and the high dimension of the parameter space.
However, I would recommend to reject the manuscript, as 1) I believe that it does not align with the aim and scope of GMD, 2) the objectives and novelty of the manuscript need clarification, 3) the implementation of the GSA has flaws and the interpretation of the sensitivity indices (based on provided equation) is erroneous, 4) the model calibration need clarification 5) the manuscript lacks a discussion of the results in light of the very large existing literature on sensitivity analysis and 6) the manuscript is unprecise at many locations. Please find below detailed comments.
1) The manuscript does not match the aim and scope of GMD, in that it does not describe a new model or new model developments, and it does not introduce a new method for assessment of models. Specifically, the GSA method used is published elsewhere (van Griensven et al., 2006) and the manuscript does not describe a novel framework for application of GSA to a complex and spatially distributed model. It actually does not describe the strategy used to apply the GSA method. The text p10 L236 suggests that the parameters were considered as being spatially homogeneous, but this would need to be better explained in the manuscript as this seems to be a very strong assumption. Previous works have examined the application of GSA to spatially distributed model and contributed to address this challenge (a few examples are: Herman et al, 2013; Rouzies et al., 2021; Smith et al., 2022; van Werkhoven et al., 2008). It is not clear how the manuscript may relate to such previous works.
2) The objective and the novelty of the manuscript need clarification.
- Sensitivity analysis was previously applied to the ParFlow-CLM model, and in this respect, the authors cite the studies of Jefferson (2015, 2017) and Srivastava et al., (2014). It is not clear how the manuscript relates to these previous studies.
- In addition, the authors mention as an objective “to test the transferability of the results to regions with other topographies and climates” (p3 P80-81). It is well known from many past studies that sensitivity indices can vary tremendously across places (an extensive review is provided in Wagener and Pianosi, 2019). Therefore, we know a priori that it is very risky to extrapolate sensitivity analysis results beyond the study location. I also think that the analysis performed by the authors in this respect lacks breadth, because it is only based on three study sites.
3) The implementation of the GSA has flaws and the interpretation of the sensitivity indices is erroneous.
- The sensitivity index is defined in Eq. 6-7. The index is actually different to the one introduced in the original method (Eq. 7 in van Griensven et al., 2006). In van Griensven et al., the index corresponds to the average of the absolute value of the partial effect, while in the manuscript it is directly the average of the partial effects. The problem of not using absolute values is that compensations between the partial effects can occur when the model response is non-monotonic. This has been discussed for instance in Campolongo et al. (2007) with respect to the Morris method and it is common practice to use the absolute value of the partial (elementary) ieffect to avoid this compensation problem (e.g. Eq. 1.49 in Saltelli et al., 2008).
- The interpretation of the sensitivity index (e.g. p5 L128, p11 L247, p11 L254-255) is not correct. The sign of the sensitivity index of Eq. 7 does not provide reliable information on the direction of change (as I mentioned earlier, the relationship may be non-monotonic) and a sensitivity index equal to zero does not mean that the parameter is not influential (as there can be compensation effects when absolute values of the partial effects are not computed).
- The authors only use a subset of parameters for GSA (12 parameters), while the number of parameters is much larger in such a complex model. In particular, it has been shown that land surface models can include a large number of hard coded parameters with an empirical basis (that are therefore largely uncertain) and many of which can have a large impact on the model output. This is documented for instance for the NOAH-MP land surface model in Cuntz et al. (2016) and Mendoza et al. (2015). Therefore, the authors would need at least to discuss this issue in the manuscript.
- Information on the setup of the GSA is missing. In particular, it would be crucial to know the sample size used and whether this sample size is large enough to obtain robust sensitivity indices estimates (as explained in Sarrazin et al., 2016). An indication on the sample size is also important from a methodological point view to understand which resources are required to run the analyses.
- The GSA only covers about two years of simulations, which may be too short to obtain reliable and period-independent sensitivity indices estimates. I refer for instance to Shin et al. (2013) when the impact of the length of the simulation period is discussed). The sensitivity indices may take different values over another simulation period. I also did not understand what the time period selected is. In Sect. 2.5.2 the period is “from 2016-07-01 to 2018-31-12” is reported, while in Figures 4-5 “from 01.01.2021 to 31.01.2022” is reported.
4) The model calibration criteria are unclear and I think that the interpretation of the calibration results are not complete.
- Based on which criteria was the “best simulation” (p9 L223) identified?
- “This indicates that there are still inaccuracies in the model parameters” (p10 L235): Mismatch in the simulated water balance could come from both issues in the parameter values (due for instance in part to the fact that the bias may not be considered as a criterion to select the best simulation), but also the model structures.
- The authors would need to better explain the objectives of calibrating the model before performing GSA.
5) The manuscript lacks a discuss section that would discuss the results in light of the very large existing literature on sensitivity analysis.
6) At numerous locations, the text is unclear to me and I provide examples below:
- p1 L25-26 “help us to reduce the computational demands of completing multiple simulations of expensive domains”: this needs clarification
- p2 L47 “Thus”: I did not get this logical link between the two sentences.
- p2 L51-52 “In addition, the identification of sensitive parameters should also help to reduce the danger of non-unique solutions, i.e. equifinality”: to me the issue of equifinality arises because there are limited data/information available to constrain the model structures and parameters.
- p4 L122 “This is more efficient than LH”: I did not get why and what is meant by LH. The GSA method could directly be applied using LH without building a tailored sample?
- p16 L329-332: The two last sentences of the manuscript are fuzzy. Therefore, the conclusions of the manuscript are not clear.
Campolongo, F., Cariboni, J., Saltelli, A.: An effective screening design for sensitivity analysis of large models. Environ. Model. Softw. 22 (10), 1509-1518.http://dx.doi.org/10.1016/j.envsoft.2006.10.004, 2007.
Cuntz, M., Mai, J., Samaniego, L., Clark, M., Wulfmeyer, V., Branch, O., Attinger, S. and Thober, S.: The impact of standard and hard-coded parameters on the hydrologic fluxes in the Noah-MP land surface model, J. Geophys. Res., 121(18), 10,676-10,700, doi:10.1002/2016JD025097, 2016.
Herman, J. D., Kollat, J. B., Reed, P. M., and Wagener, T.: From maps to movies: high-resolution time-varying sensitivity analysis for spatially distributed watershed models, Hydrol. Earth Syst. Sci., 17, 5109–5125, https://doi.org/10.5194/hess-17-5109-2013, 2013.
Jefferson, J.L., Gilbert, J.M., et al. 2015. Active subspaces for sensitivity analysis and dimension reduction of an integrated hydrologic model. Comput Geosciences 83, 127-138.
Jefferson, J.L., Maxwell, R.M., et al. 2017. Exploring the Sensitivity of Photosynthesis and Stomatal Resistance Parameters in a Land Surface Model. J Hydrometeorol 18(3), 897-915.
Mendoza, P. A., Clark, M. P., Barlage, M., Rajagopalan, B., Samaniego, L., Abramowitz, G. and Gupta, H.: Are we unnecessarily constraining the agility of complex processâbased models?, Water Resour. Res., 51(1), 716–728, doi:10.1002/2014WR015820, 2015.
Rouzies, E., Lauvernet, C., Sudret, B., and Vidard, A.: How to perform global sensitivity analysis of a catchment-scale, distributed pesticide transfer model? Application to the PESHMELBA model, Geosci. Model Dev. Discuss. [preprint], https://doi.org/10.5194/gmd-2021-425, in review, 2021.
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., Tarantola, S.: Global Sensitivity Analysis, The Primer, Wiley, 2008.
Sarrazin, F., Pianosi, F., & Wagener, T.: Global sensitivity analysis of environmental models: Convergence and validation. Environmental Modelling & Software, 79, 135–152. https://doi.org/10.1016/j.envsoft.2016.02.005, 2016.
Shin, M., Guillaume, J.H.A., Croke, B.F.W., Jakeman, A.J.: Addressing ten questions about conceptual rainfall-runoff models with global sensitivity analyses in R. J. Hydrol. 503 http://dx.doi.org/10.1016/j.jhydrol.2013.08.047, 135-125, 2013.
Smith, J. D., Lin, L., Quinn, J. D., and Band, L. E.: Guidance on evaluating parametric model uncertainty at decision-relevant scales, Hydrol. Earth Syst. Sci., 26, 2519–2539, https://doi.org/10.5194/hess-26-2519-2022, 2022.
Srivastava, V., Graham, W., et al. 2014. Insights on geologic and vegetative controls over hydrologic behavior of a large complex basin - Global Sensitivity Analysis of an integrated parallel hydrologic model. J Hydrol 519, 2238-2257.
van Griensven, A., Meixner, T., Grunwald, S., Bishop, T., Diluzio, M., and Srinivasan, R.: A global sensitivity analysis tool for the parameters of multi-variable catchment models, Journal of Hydrology, 324, 10 – 23, https://doi.org/10.1016/j.jhydrol.2005.09.008, 2006.
van Werkhoven, K., Wagener, T., Reed, P., and Tang, Y.: Rainfall characteristics define the value of streamflow observations for distributed watershed model identification, Geophys. Res. Lett., 35, L11403, https://doi.org/10.1029/2008GL034162, 2008.
Wagener, T, Pianosi, F: What has Global Sensitivity Analysis ever done for us? A systematic review to support scientific advancement and to inform policy-making in earth system modelling, Earth-Science Reviews, 194, 1-18, https://doi.org/10.1016/j.earscirev.2019.04.006, 2019.