Dear Reviewer,

We welcome your comment.

In response to this comment, it is true that a multi-Gaussian assumption is made with respect to measurement noise. Furthermore, a multi-Gaussian assumption underpins the development of the equations on which the DSI methodology is based. However, as discussed in our paper and in papers we cite in our paper, it is less involved in the implementation of these equations than in their derivation, because the surrogate model embodying the measurement-to-prediction covariance is historically fitted before prediction uncertainties are evaluated. Deviations from measurement-to-prediction linearity/Gaussianity are thus at least partially accounted for.

We note that all methods of uncertainty analysis involve assumptions. The assumption of a Gaussian likelihood function is very common. However, if the prediction uncertainty is dominated by non-uniqueness rather than measurement noise (as is the case in our example), then the detrimental effects of this assumption are small.

We also note that while uncertainty analysis methods such as rejection sampling and Markov chain Monte Carlo do not require a Gaussian prior or Gaussian likelihood function, these methods are impractical in contexts where the number of parameters is high and model run times are long. High model run times and parameterisation complexity characterise many groundwater modelling contexts, including the one exemplified in our paper. Iterative ensemble smoothers (with which we compare DSI) are affected by the multi-Gaussian assumption. We note, however, that the realisations that compose the initial ensemble from which the IES inversion process is started do not have to be samples of a multi-Gaussian distribution; however, they are adjusted as if they are.

A strength of the DSI method is that the prior does not have to be multi-Gaussian. There is nothing in its theory that requires this. Furthermore, the prior samples used to start the DSI process can accommodate both aleatory and epistemic uncertainties. Parameter fields may be categorical in nature, and may in fact sample entirely different prior probability distributions. The multi-Gaussian assumption used to link measurements of past system behaviour to predictions of future system behaviour is independent of any assumptions about the prior. Furthermore, because a direct link is made between measurements and predictions (thereby bypassing parameters), this multi-Gaussian assumption is likely to have a weaker effect on the results of the predictive uncertainty analysis process than highly parameterised methods that rely implicitly or explicitly on parameter adjustment (such as linear Bayesian methods, randomised maximum likelihood methods and iterative ensemble smoother methods).

The reviewer is right in stating that the prior parameter probability distribution we use in our example is "correct" in the sense that it includes the associated permeability arising from alluvial channels, and that the stochastic characterisation of these channels is the same for our "reality field" as it is for our prior. However, as pointed out in our paper, this allowed us to assess the quality of the uncertainties calculated using DSI. Furthermore, it allowed us to compare the performance of DSI with that of IES - a method that is challenged by the lack of Gaussianity for the reasons given above. While we agree that synthetic cases are artificial by design, they allow a numerical experiment to be carried out under "controlled conditions" such as those we have created and documented in our paper.

We agree that a real-world example would have been nice. However it would have lengthened our paper, while eroding our ability to verify DSI performance to the extent that we did. Furthermore, authors that were cited in our text did demonstrate their version of DSI on real-world cases. For example, we can refer to He et al. (2018) – a paper that we cite on line 65 of the original manuscript. They have demonstrated the EVA method for data worth assessment. This method is based on a theory identical to that of DSI, and their paper includes a real-world example. Meanwhile, our paper focused on providing examples of innovative use of the DSI surrogate model in uncertainty and data worth assessment.

As a final comment, we agree with the reviewer and would like to emphasise that this is exactly what we mentioned in the text, pointing out that some fast travel times exceeded those calculated using the previous parameter fields (i.e. slowest travel times). Lines 534 have been changed to clarify this.

Reference

He, J., Sarma, P., Bhark, E., Tanaka, S., Chen, B., Wen, X-H and Kamath, J., 2018. Quantifying expected uncertainty reduction and value of information using ensemble-variance analysis. SPE Journal. 23. 10.2118/182609-PA.

Dear reviewer,

Thank you for taking the time to review our paper. A follow-up to your comments can be found in the supplement to this response.

Best,

Delottier, P. Brunner and J. Doherty

Dear Editor,

Thank you for considering our paper. We have added the source codes for PEST and especially DSI in a Zenodo repository (DOI: 10.5281/zenodo.7913402) with a license file in it. Modification will be made accordingly on the Code availability section.

Concerning the reviews, we note that no technical of methodological concerns are raised. One reviewer suggests to employ a more complex model, but if we do this we loose the capacity to compare our results with a known “truth”. Moving the assessment of the approach to a supplementary section does not necessarily corresponding to scientific practice as the literature on uncertainty analysis and related topics (particularly those which demonstrate new methodologies) is replete with papers that restrict their analyses to synthetic cases that render their methodologies amenable to verification). Hence we do not see that repeating the description of our work flow for a real case would either improve the description of this workflow, or improve verification of the methodology.

The other reviewer suggests that we compare our methodology with that of MCMC based on a simpler model, but also highlights that our synthetic case is well chosen. It is well chosen because it demonstrates uncertainty and data worth analysis in hydrogeological contexts in which parameter numbers are high, and in which prior parameter probability distributions are not amenable to analytical description (we are happy to add a paragraph to our paper to make this clear, if desired). MCMC does not work well in these contexts. Hence we have compared our methodology to another, well known and well understood methodology which does work well in this context. We cannot see how the development of a simple model that would allow DSI to be compared with MCMC would serve our primary purposes of demonstration of the DSI methodology to our readers, and verification of novel analysis types in complex hydrogeological environments with the DSI-produced surrogate model. In contrast, we see that it would add substantially to the length of the paper, while adding little to a reader’s ability to assess whether the methods and workflows that we present are suitable for his/her case.

The industry is in urgent demand for more efficient approaches in this field. We strongly hope that the publication is not delayed.

Best,

Delottier, P. Brunner and J. Doherty