Unsollicited review by Tijn Berends



I am glad to see a manuscript like this. With the ever-increasing political and societal demand for new, more accurate scientific knowledge about the Earth system, and particularly its future state, the complexity of computational models has grown exponentially over the past few decades. The need for software engineering skills, on top of the knowledge of the scientific domain, and the wide and varying set of skills required of an active research scientist, is now an undeniable reality. New literature investigating just exactly what “software engineering skills” entails in the context of research software is therefore a valuable addition.

Having the luxury of being an uninvited reviewer, I can constrain myself to only pointing out the bits that really strike me, and leave the detailed feedback to the invited reviewers. Two points stand out to me in this manuscript that I think could do with some improvement.

Firstly, there is the concept of “self-explanatory code” mentioned in lines 406-413. While I appreciate that the authors are merely citing another group’s description of that group’s own work, I believe this statement needs a disclaimer. Depending on how you define “self-explanatory”, either all code qualifies as such, or none. If, for example, we define code as “self-explanatory” when one can eventually arrive at an understanding of its functionality without consulting the original author, then all code is self-explanatory – with, of course, the caveat that “eventually” can, in many cases, be prohibitively far into the future. On the other hand, if we define code as “self-explanatory” when we require no other resources to (again, eventually) understand its functionality, then probably no code ever meets this definition, at least in the context of research software, which always requires a substantial level of background knowledge on the part of the developer. E.g., is the code that calculates the sea-level equivalent volume of an ice sheet truly self-explanatory if it does not explain the concept of sea-level equivalent volume? In my view, these considerations illustrate that the phrase “self-explanatory code” is so difficult to define as to be practically meaningless. In my experience, it is used mainly by people who inherited code from their supervisor that is not as well-commented as they’d like it to be, but cannot say so out loud for fear of their career prospects. I’m sure the authors can add these considerations, possibly in a rephrased manner, to their revised manuscript.

Secondly, there is the first of the authors’ recommended best practices, in lines 470-473, where they support the use of Agile as a project management framework. Having briefly worked at a company that applied this framework (and much longer as a researcher building my own numerical models), I have some small amount of experience with it, and I must say I do not immediately see its value in a research setting. The highly individualistic nature of scientific research(ers), the very poorly-defined goals, scope, and expected duration of research projects, as well as the high degree of overlap between developers, managers, stakeholders, and users, make the use of Agile difficult in a research context. Additionally, while I see how Agile working can improve the speed with which a large group of people produce output within a certain project, I do not immediately see how Agile affects the quality of that output – which is the subject of this manuscript. I.e., agile scientists might produce science faster, but not necessarily better. If the authors can provide arguments to the contrary, I’d happily read them, but right now nothing of the sort is written in the manuscript – in fact, the concept of Agile is only mentioned briefly at one point in the introduction, and then does not appear again until it is listed as the first “recommended best practice”. I hope the authors can remedy this lack of evidence for their claim of Agile being a “best practice” in the revised version of their manuscript.