Review of the development and application of an Arctic Sea Ice Emulator v.1

This paper describes a framework or emulator which, using global temperature anomaly input, produces Arctic yearly mean temperature, Arctic monthly temperatures and finally seasonal arctic sea ice extent. The modelling is done using a few parametrized equations per step, each calibrated to fit CMIP6 modelling output (per model) and optionally bias corrected to fit observational data. Applications shown include calculation of remaining carbon budget (or temperature change needed to limit by) to avoid accelerated arctic sea ice loss due to ice detachment from land, and identification of modelling similarities, differences and fitness to observations of Arctic temperatures and sea ice extent.

This is a useful endeavor towards the understanding of Arctic Sea Ice modelling, with applications to related climate tipping points and model evaluation, and the model description is fairly clear, though I believe it would benefit from some improvements as detailed below. I also believe the model evaluation and applications could do with some further improvements as some general questions remain. The model evaluation is done looking at extensions of the datasets out to 2300, from a calibration of the scenarios up to 2100. To me, a more useful model evaluation would consist of applying the framework to other scenarios run by the same model. Most models presented here will have run one or more additional scenarios such as SSP-3.70 or SSP-1.19. In addition, many of them have run multiple ensemble members. Considering how the fits would do in that context to explore variability would make a lot of sense to me, at least producing some plots to see how the parametrization fits in the range of the climate models internal variability. Another point of particular interest to the applications for this tool which I am missing some discussion of, is overshoot. Can this emulation deal with that, or do you believe it is beyond the validity of the parametrizations? Is it only beyond that if accelerated tipping points occur before the maximal warming is achieved, or will it also be true otherwise. I think some discussion of the applicability of this tool to overshoot scenarios is necessary here.

One more slightly general note before we get into the details. I would like a bit of reflection on the framing of this paper and what it is. When I come into a model description paper, what I hope to see is a description a model that can be reused also by others outside the group of the authors, where the code is nicely reusable, with interesting scientific applications and somewhat of a finished product. In this case (and a lot of other cases), what I see is rather the description of a really interesting parametrization framework for a Arctic temperatures and sea ice, calibrated to model output, but with functional forms and based on empirical and mechanistic expert judgement. I also see a workflow in steps, but it is less clear how the code would allow me to apply that workflow to new datasets. Instead the code looks more like supporting material that would allow the reproduction of the current results, rather than something that is meant for further use (at least by outsiders). (An added problem here is the lack of a proper README-file for the code, and the choice of programming language, which requires licensed software (Matlab) possibly with an unknown number of add-on libraries with their own libraries to be run). All of this takes nothing away from the work presented, it is very nice and merits publication, but I think this is less the description of “an emulator” and more the description of an emulation framework (personally I’m not sure the word emulation is even the best here, as currently it seems to be used to mean practically everything, hence the meaning of it gets sucked out of the term). I will give more advice in detail below, on what would help in making this a better description paper regardless of whether we are describing an emulator or a framework, however, I’d like the authors to consider what they think this article is describing in the way that they write it, as that might also add to the clarity of the presentation.

Below, I will go through the paper with more detailed comments.

Starting with the title, I believe that one requirement of GMD model description papers is for model name and version to be included in the title. Is the model called “Arctic Sea Ice Emulator v.1”? If so, prefacing the name with the article “an” seems strange. I note that the github code repository for the code has the descriptive yet somewhat unusable name “Development-and-Application-of-an-Arctic-Sea-Ice-Emulator-Journal-Article-2024”, if the name is in fact “Arctic Sea Ice Emulator”, that would be a better name even in the repository. For a model description paper, I expect a description of a model that can be used more generally, so the model shouldn’t be named and coupled this closely to the paper, but this also feeds into my slightly more general point on what you mean for this article to be about.

Abstract:

Just a couple of what I assume are typos here:

Line 3: “we developement” -> “we develop”

Line 5: "emitted to prevent” -> “emitted while preventing” (or something like that. I think nobody is emitting CO2 to prevent seasonally ice free conditions in the Arctic, and if they are that would be a very bad idea…)

Introduction

Line 39: Not sure “ascertain” is the right word here, I think I would prefer something like “find”

Line 49: “we aim assess” -> “we aim to assess”

Arctic Sea Ice Emulator Setup

A couple of general comments here. I would maybe not have a subsection headline for the overview but prefer to have that just immediately, but no big deal. When going through the overview I would also like references to the sections in which the various steps are described. I would also have preferred to have a separate section on the data used as this is quite instrumental to the workings of the methodology. Currently, the description of what is used for the CMIP6 models is described in the overview in brief, whereas the observational data is described in the data availability section. Ideally, I think both should be described with some discussion of choices made in a separate subsection here, while how to get those datasets should be described for both data types in the data availability section. Tables of involved models and ensemble members used would be useful too. One option if you think this fits poorly in the main text is putting such a section including model and observation data tables in the supplement.

Overall, to be a functioning model description paper I think each subsection should also mention the code/ scripts that implement the procedure described in the section.

The workflow figure is very nice, but I’d like some more information explaining it in the caption, including information on what sections to refer to for more information on the various steps and workflows involved.

I am, however, missing a good connection between the text and the code base. How do I run it? How does it work? Given the right data, can I plug and play the whole workflow, or do I need to do each step separately. Can I run it for a different (set of) models or experiments? All of this information doesn’t have to be given in detail in the overview, (especially details on how I run it or how it works), but since I also can’t find it in the README in the repository, I don’t really know where to look, and some of these questions should be answered in the overview, so I know whether I can expect this to be a runnable tool, or a description of a framework that I could possibly reimplement from the description (that is also fine, but again that speaks to what this is and isn’t.)

Another general comment is that there are a lot of parameters and constants defined here, and it seems that the same letters are reused for different things here (though it is not always entirely clear).For instance, is the a in 3c and 3d, the same as the a in 3b? I would much prefer if no letters were reused. Referring early to the parameter tables in the supplement and stating the number of free parameters would be helpful, but also having more subscripts on the parameters to explain what they do, for instance a in 3b could be a_linSIAloss or something like that, that would help in following the equations. I think only a relatively simple going over to clarify this more would help the readability of the equations a lot.

Line 86: The equation for tas(AAAB) given here deserves an equation number and a separate line.

Line 93: “for the first ensemble member of each”, this turn of phrase confuses me here. My understanding was that you are only using the first ensemble member, hence the procedure her is that you find tas(AREF) for whatever model data  you are trying to replicate, so in principle, I could choose some other member and do the same?

Line 102: Equation 1, how is this tied to the equation for tas(AAB), is p matched up to fit the linear regression? There are so many parameters and functions floating around, please make it as clear  as possible how they fit together.

Line 126: I know this might be outside your control, but it would be very helpful if this equation came on the next page alongside the next equation and its description

Line 129: “where tas(AAAB)”, were you meaning to introduce tas(AMAB) here? That’s the new term in the equations? The way it stands now, I got quite confused and had to reread several times…

Figure 4, caption, second to last line: Duplicate “are” here, “2100 are are displayed”

Line 161: “which is we define” -> “which we define”

Line 198, equation 3b: As far as I can see, the term (1 + exp(tas(t=0 -b ))) is a constant (though model and parameter dependent) term. I would consider giving it it’s own name and defining it in its own equation, at it makes it easier to read the functional form and evolution of the sea ice area from the equation.

Line 205: “on the denominator” -> “in the denominator”

Results

Line 246-254: A little more detail on this evalution would be nice. Did you have data out to 2300 for all of the models and experiments considered? Was this part of your model selection criterion? This information could easily have fit into a dataset section in Methods, but if it isn’t there, we need that info here (and repetition here might be useful anyway).

All of 3.1: I’d really like a discussion/understanding of why you don’t test using other experiments for the same models/ensemble members also. Is it because you would need separate calibration for each experiment? If so that again would make this much less useful as an emulator (but just as useful as a parametrization tool to understand Arctic Sea Ice loss, so no shame in that). Anyway, you should discuss this, discuss whether there could be some extension to other experiments, I’d be particularly interested in overshoot scenarios. Even if you need the scenario itself to get a new parameterization, checking whether the scheme works for overshoot would still be interesting, and I’d be very interested in such an application, or at least hear the arguments why you would think the parametrization might not work for that to understand the limitations of the scheme.

Line 311: From the previous text it looks to me like this should be -14Gt rather than 14Gt

Line 332-337: That is the t in these formulae? tonne of CO2?

Discussion

Line 367: “who use five” -> “who used five”

Code availability

In the code availability section, I’d like there to be some more details. I think linking to the github repository (so one can easily see it exists) is useful, and some explanations on what requirements to run are (you could also solve that in the code README, at least detailing which Matlab extra libraries are needed), but I think here or somewhere you should let the reader know that your code is in Matlab so they know they might not be able to run it (this is less important if you mainly consider this to be a parametrization scheme see previous comments on this). Regardless, the README should include code flow and how to run to reproduce the various parts of the paper. I have also noted that the various parts of the code, comments on structure refers to sections in a “thesis”, does this mean parts of the article? Or an actual other thesis (if so you need to refer to this thesis in this paper, even if it is not published yet)? If you are referring to this article in the code, then please make that clearer in the code.

Data availability

I’ve commented on this further up, but just noting again that details on the CMIP6 data used and the availability of it should also be listed here.

Supplement

Table S1 and table S2, the references in the last sentences should not be “bottom of the table”. I assume you are referring to tables S3 and S4 respectively, if so, do that.

Some tables with error scores per model would be good to see.

S2.2.: This section is strange. Are you just defining the IPCC terms here? If so that doesn’t fit the headline very well. To me it looks like this should be a part of Section S2.3, i.e. definitions needed for that.

General comments

This paper describes a newly developed Arctic sea ice emulator for use with a more comprehensive climate and carbon cycle emulator such as MAGICC.  This sea ice emulator can integrate insights from complex multi-model climate projections combined with observational constraints regarding the Arctic response to changes in global surface temperature. Elaborated parameterizations dealing with key features of Arctic warming patterns and sea ice behaviors result in the successful emulation of complex models' response of the sea ice area to a wide range of warming level pathways over the 21st century and beyond. Emulators can provide a probabilistic approach to the scientific basis for climate change mitigation, and this paper includes such an application study as well. Thus, this paper is suitable for publication in the GMD. However, the current manuscript may require revisions for clarification and consistency within the scope. The following are my concerns and suggestions for consideration.

L53

The acronym SSP has not been described yet. However, it is not necessary to limit the range of scenarios to a specific set of scenarios in this context.

Subsection 2.2

Are there any drifting errors in the preindustrial control runs for the CMIP6 model variables used in this study? Are these erroneous trends removed?

Subsections 2.4 and 2.5

Given that air temperatures over the sea surface greatly depend on whether the surface is covered by sea ice, decreases in the amplitude of the seasonal cycle, at least partly, reflect decreases in the sea ice area. Is it appropriate to process Steps ii and iii independently?

Likewise, is it not necessary to address the temperature biases described in Section 2.4.1, regarding possible sea-ice biases?

Equations (2a), (2b), and (2d)

It is necessary to clarify the dimensions of the parameters. Although Equations (2a) and (2b) imply that P, f, and h have the dimension of temperature, if so, Equation (2d) makes h dimensionless, which is a contradiction.

L250-251 and Figure 6

The acronym RMSE is referred to inappropriately regarding its standard definition, that is, the root mean squared error. The goodness-of-fit statistics are confusing because they are indicated by different indexes: RMSE correlation, correlation coefficient, and RMSE fit.

Figure 6 shows relatively large emulation errors for IPSL-CM6A-LR in May. Is there any need to mention anything about this point?

Subsections 3.3 to 3.5 and the second paragraph of Section 4

Given that this paper is submitted as a model description paper, the results and discussion in these parts, while providing an interesting application study, give the impression of being out of the scope. The introduction should provide the context of the application use presented in this paper. The related results and discussion should focus on methodological matters rather than mitigation issues, such as the remaining carbon budget, to be consistent with a model description paper.

L289-291

'slightly larger' is valid for sensitivity to temperature, but not for sensitivity to CO2 emissions. The right panel in Figure 7 shows 'slightly smaller' for the latter.

Code availability

The README document in the code archive should include a description of the programming environment and where to obtain the input data necessary for execution.

Data availability

The current manuscript describes what data were used but does not describe how they can be obtained. The observational and reference data used in this study are described in the main text.

Figure labels and markers

The labels in the figures and the median markers in the box plots are too small.

Figure 1

The figure caption would be more suitable with 'calibration and constraining processes' than with 'conceptual model.’ Font size should be increased for readability.

Figure 2

Is the description 'Ranges indicate the 17th-83rd percentile of the scenario range' correct?

I assume that it is a model-structure uncertainty range rather than a scenario range.

Figure 3

The figure caption should should include a description of each panel.

What is the histogram at the right end of panel (a)?

Panel (a) shows that the values for CMIP6 models fall to zero from place to place. What does this mean?

Figure 7

If the quantity of CO2 emissions is cumulative, it should be clearly indicated from which point in time it has accumulated.

The unit of weight should be clearly stated as tCO2 or tC.

Minor comments and technical corrections

L18

The acronym SIA is not explicitly defined.

L126

In Equation (2a), the outermost and innermost parentheses are redundant.

L129

Considering the difference in the number of days in each month, the equally spaced m values do not exactly represent each month.

L136

To avoid 'moves ... vertically' that implicitly assumes the orientation of a graph plot, how about rewriting 'a non-optimised parameter that moves the temperature curve vertically to ensure ...' to 'a non-optimised offset value to ensure ...'?

Likewise, 'vertically shifted' on L207 should be rewritten appropriately.

L180

The description of the seasonal asymmetry of SIA changes may need some literature.

L285

Probably, 'than' is an error for 'that.’

L289

Delete 'm' just after '-2.5'

L496

In this journal, references may be provided with full author lists. I notice that the author list of Nicholls et al. (2020) is incomplete.

Figure 4

The caption includes duplicated 'are'

Figures 4 and 5

Although the selected CMIP6 models are arbitrary examples, it appears preferable to show the same set of models in these figures.

Supplementary tables

The units of the values are not stated.