Paper summary

This paper summarizes the outcomes of a large effort to create graphical representations of the inner workings of the 16 models that participated in ISIMIP2b. The result is a pair of diagrams, where the first shows in a 3-dimensional way (A) how the model discretizes the vertical domain (e.g. snow, canopy, soil, groundwater), (B) which lateral (surface) components it includes (e.g. lakes, reservoirs, wetlands), and (C) which human water use sectors are included (e.g. agriculture, livestock, industry). If a model doesn’t include a given component, that component is greyed out. The paper briefly describes the process that led to the creation of these diagrams, possible ways in which the diagrams could be used, and some thoughts about creating these diagrams for models not included in ISIMIP2b.

Review summary

I think this paper is an interesting and timely contribution, and I expect this process was anything but easy. Better understanding which models to use when, where and for what purpose is critical for actionable decision making, and these diagrams might be helpful in both conversations between model developers and model users, as well as outline experiments that would lead to such better understanding. The graphics are clean and the paper is generally easy to read. However, I think some important information is missing and I think the paper needs to be revised before it can be published.

Major comments

I have several major comments, based on my reading of the paper and multiple line-by-line comments that can be found in the attached .pdf:

1. The main methodology used in this paper seems to be that everyone involved went through a long process of deliberations about how the final diagrams should look. I'm not very familiar with how such social processes are typically documented and described in journal articles, but the current description of it in the paper is very short: there is almost nothing about the process beyond its outcomes. However, these discussions lie at the heart of the resulting diagrams and I think more description of how they were organized, which stakeholders and backgrounds were present, how different points of views and needs were balanced etc. is needed. I think the paper in particular needs more information about how the main trade-offs between accuracy/detail and aesthetics/clarity were made, and why the resulting two diagrams are seen as the right balance between these different things. Are there transcripts of the conversations that were had?

2. I have some concerns about some of the phrasing in this paper, and how that relates to the wider context of modeling capabilities as well as extensions of these diagrams. The paper is quite clear about the fact that the term for the complete diagram (i.e., the “‘ideal’ model”) is not meant to be seen as a statement that describes how the ideal Earth system model looks. I believe that if this is so, then simply using a different term is more appropriate. I’ve suggested “ISIMIP-complete” in the comments but I think anything that avoids the implicit message that this is how an ideal model looks is better than what is currently used.

This becomes particularly important in the discussion section of the paper. Here the authors discuss these ISIMIP models and the resulting “ideal” model diagram as what is currently feasible within the scientific community. The word “ideal”, in my opinion, implies much more than is justified here. The community as a whole has larger modeling capabilities than what is shown by this specific subset of models, and what can feasibly be done by the community extends beyond what this “ideal” model diagram shows. I think it is important to be honest about limitations in our models (and these diagrams do a good job of giving high-level overviews of what specific models can and cannot do), but I think it is equally important to not undersell what is currently feasible if the community were to integrate all the separate bits of expertise in a coherent way. I don’t think this is necessarily a discussion that needs to be had in this paper, but I do think it is important to acknowledge that the word “ideal” implies certain things, no matter how often the paper says that that is not the way the reader should interpret the word - particularly if these diagrams are partially meant to facilitate discussion with stakeholders who possibly don’t have much personal modeling experience or clear overviews of the current-state-of-the-art of environmental modeling. Using a different term than “ideal” completely avoids all of this.

Finally, I think the phrase “ideal” model limits the ability to extend this diagram beyond what it currently includes. The authors list multiple aspects of environmental modeling that are not included by any of the models in ISIMIP2b and thus are not included in the “ideal” diagram. What happens when a model is included that introduces a new capability? Will this lead to the “ideal v2” or “slightly more ideal” diagram? A more version-y phrase would be more extensible in such future scenarios. This would also be more in line with the GMD requirement to include specific version numbers in the titles of a number of manuscript types (I’m aware that this is not required for review and perspective papers, but that doesn’t make it a bad idea in general).

3. I think the paper could also use a bit more text on some of the more practical concern about modeling capabilities that go beyond what’s currently in the diagrams. How easy will it be to adapt the JSON tool and the diagrams themselves with new fluxes or other relevant information? Will there have to be a new design process to avoid cluttering what is currently there? Will new trade-offs between accuracy and aesthetics need to be made?

4. Something I have missed in the paper is a description about energy balance calculations. The model inputs suggest that at least some of these models try to explicitly account for energy balance components but neither the diagrams nor the text provide any information about this. Were energy-related state variables simplified away in favour of readability? If so, I think this needs to be discussed in the paper.

Minor comments

Please see the attached .pdf. There is some overlap between those comments and the major ones I outlined above.

The paper presents a graphical representation of global water models that aims to simplify model communication, both for a non-technical audience as well as for technical model users. The paper focuses on ISIMIP 2b models, creating a "reference" model that is a superset of all the ISIMIP 2b models. The authors briefly describe the process that they followed to arrive to this graphical representation, they describe and explain how to use it. They also mention an open-source tool to create versions of this image.

The paper presents an interesting "tool" to communicate the main characteristics of global water models. A graphical representation may simplify the process of comparing different models, so making it easier to grasp where differences may appear and what experiments may be required to fully understand modeling choices. It may also be interesting to a broader audience, that may find a hint of which effects may be at play when interpreting the results of one of these models. In this regard, the paper is very interesting. I believe that it may deserve publication in GMD.

However, I have some comments that I would like to see answered -or implemented when possible- before publication.

1) I wonder how general the framework is and if it could not be applied to "almost" any water or hydrological model. I understand that ISIMIP 2b models were the reference, but the authors mention that adaptations may be required for ISIMIP 3 models, and also, many hydrological models could fit into the framework. A more clear discussion on this topic would be appreciated, since I believe the framework may be more general, but I may be missing some limitations.

   The framework being fully general may be an argument for having the paper published in GMD. If its generality is more limited, it may require a more clear statement of the limitations. In a sense, what I may be missing is a clear characterization of the framework, testing it with models outside of the ISIMIP 2b experiment: what else would be necessary to capture other models?

2) Related to the previous point, I would have appreciated a more detailed methods section. This paper's may have been a "naive" approximation to solving the problem, since the authors mentioned that other professionals should have been included in the effort earlier in the process. However, I believe the methods section does not present a structured procedure to follow. I am thinking about other communities of modelers that may be interested in doing something similar. They would not be able to learn much from the current description of the methodology in the paper.

   Here, I believe that the best approach would be to describe the methodology estimated to be the best after the fact, and then clearly indicate where the actual methodology differed from the suggested one. Such a description may also serve for scientists in other fields (not necessarily technical) to suggest improvements or modifications that may expand the methodology further.

   3) I was surprised that the code that the authors made available to generate this graphical representations receives such little attention in the manuscript. Obviously, if the framework is less general than what I think, it may make sense to just present the results for the models of ISIMIP 2b, but if it is more general that that, explaining how to use it would be a nice inclusion on the paper or at least in the GitHub repo. As it stands now, the repo is not very user-friendly.

   4) I am not sure that including all the images as an appendix makes sense. As supplementary information it may. I believe that presenting the framework and a couple of examples should be more that enough for the paper.

   Some minor comments follow:

   1. The first sentence of the introduction is a bit complicated (I am not a native English speaker). I would say that rephrasing it would help the readers get into the paper.

   2. Line 124: I would write greying out together, before the parenthesis. As it currently stands, it is difficult to know why the out outside fo the parenthesis is there.

   3. Line 208: The reference to "Telteu et al. (2021)" should be better included in the text or included in parenthesis.

   4. Line 253: Same comment as before.