General Comments

This paper has two purposes – to inform the reader about activities of the CSDMS and to describe the tools and standards developed for model interoperability.

1. On the usage of FAIR. FAIR standards apply to data not software.  Several papers have discussed why software is not data which motived the development of FAIR4RS.  While FAIR is used appropriately in some contexts, it is not used appropriately in others.
2. Verification and Validation. The paper begins a discussion on code testing but does not address this point further for codes in the CSDMS repository. Assuming unit testing is done, and codes are validated and verified, it would be nice to have a discussion on what this means in the context of the ecosystem created especially in coupling models/components and error propagation.
3. Computational Overhead. CSDMS has created a rich ecosystem of tools and standards. Has any performance benchmarking been done to understand the computational overhead?

Specific Comments

Line 94:  I am unclear what the analogy the quote is trying to draw.

Line 143: The reviews for JOSS journals are objective reviews set against a checklist of items.  Reviewers check for what the authors says it does.  Do they inspect the software code itself, that is “evaluate the software directly”?

Figure 2: Include a descriptive figure caption.

Line 241: Strictly speaking, reproducing needs an executable, not necessarily the source code.

Line 244: FAIR data need a persistent identifier such as a DOI.

Figure 3. Include a descriptive figure caption.

Line 347: What happens in the interpolation when there is a first order discontinuity?

Line 360: What happens when the timesteps are not multiples of one another?

Figure 13: Clarify that c. and d. are with the “with landsliding” model

Line 580: Sentences is awkward. Librarization of tools to access data sets not librarization of the data sets themselves.

Technical Corrections

Review usage of title case in section and subsection headings for consistency

For inline text, use Fig. or Figure consistently. Similarly for figure referenced in parens, ()

Figure 1: Many of the labels on the figures are hard to read at 100% magnification

Line 154:

For citing PETSc, see:

            ftp://ftp.mcs.anl.gov/pub/petsc/nightlylogs/xsdk/xsdk-configuration-tester/packages/petsc/src/docs/website/documentation/referencing.html

deal.II (not capitalized)

For citing deal.II, see:

            https://www.dealii.org/publications.html

Line 279, 359: Are contractions allowed?

Line 295: References Fig. 6 before referencing Fig. 5 (and not referenced elsewhere)

Table 3. Change Table caption to:  “… Python Modeling Toolkit pymt.”

All models are capitalized here but may be used as lowercase in text. Theses should be consistent.

Line 384: (Fig. 2).  Should this be deleted?

Figure 1 and Figure 13: Descriptions use a syntax of referring to the different panels e.g,. a. vs (a)

Figure 13: Sediment thickness is plotted as soil depth as a yellow and not an orange line according to the legend.

Figure 14: Technically there is no “(b)” so should there be an “(a)”?

Figure 15: Define EKT

Line 614: Define ESPIN

Line 618: CoMSES Net

Line 619: pyOpenSci

This manuscript outlines the challenges, opportunities, and contributions of the Community Surface Dynamics Modeling System (CSDMS) community towards advancing integrated, extensible, and reproducible modeling of Earth surface processes. The authors provide some conceptual context for the problem and issues that they are addressing, which is the increasing importance of models in Earth surface processes, in conjuction with some of the challenges that this growing and evolving facet of Earth science is experiencing. The authors outline an interesting set of model operations – drawing a comparison with Bloom's Taxonomy – of increasing complexity that  are required of models. They then review a suite of tools developed in support of the CSDMS effort that address varying aspects of this taxonomy. 

The manuscript is very well written and organized. I am particularly fond of their proposed taxonomy and believe that could be the basis of further expanding how we not only think of models, but also how we thinking about educating scientists about models. I presents relevant examples using tools within the CSDMS ecosystem, as well as an overview of the ways that CSDMS seeks to promote development and acquisition of modeling skills and habits of mind in the community. 

In all honesty, I cannot find any significant errors or issues within this manuscript. My only very minor quibble is that I believe that the authors could perhaps elaborate more on some of the more future-oriented challenges, particularly as they relate to educating the next generation of scientists. The authors, in the Discussion, allude to a set of skills that CSDMS has observed are important for modeling and I wonder if those could be posed as learning outcomes that might be necessary for students. Could, for example, figure 2 be replicated along with some key skills or learning outcomes that are appropriate to each of these operations. To be clear, I don't think the authors need to address this suggestion, as it may be beyond the scope of this manuscript, but that is what immediately came to mind when I read the article. 

I believe this manuscript makes an important contribution to the Earth surface modeling community and suggest that it be published in its present form.