Comments on the manuscript entitled “Design, evaluation and future projections of the NARCliM2.0 CORDEX-CMIP6 Australasia regional climate ensemble” by Virgilio et al. submitted to GMD

General comments:

The authors have compared the experimental designs and results across three generations of NARCliM RCMs. The latest iteration, NARCliM 2.0, features enhanced spatial resolution and utilizes CMIP6 experiment outputs as large-scale forcing data, representing advancements over earlier phases. The ensemble simulations of NARCliM 2.0 were conducted after a rigorous evaluation and selection process involving CMIP6 models and various physics configurations of the WRF model. This approach has the potential to provide more robust projections of regional climate over Australia. The ensemble simulations, incorporating diverse GCM-RCM combinations, make significant contributions to CORDEX. Therefore, I recommend acceptance pending minor revisions, including clarifications, correction, and reorganization in certain sections. Specific comments are outlined below:

Specific comments:

L108: Please replace "NARCliM2.0" with "NARCliM 2.0 (NARCliM 1.5)".

Section 3.2.1: It is unclear which variables were evaluated to assess CMIP6 GCM performance. Note that precipitation, daily maximum and minimum surface air temperatures do not serve as boundary conditions for driving the RCM. It would be preferable to evaluate U, V, T, Q, Z, SST, PSL for dynamical downscaling purposes. This issue should be properly addressed or discussed.

Table 2: Please clarify how many GCM-RCM runs were conducted for CORDEX-CMIP6 NARCliM 2.0. Specify the combinations used. Were all five GCMs downscaled by seven RCMs each? Presenting this information in a table format would aid readers in quickly accessing these details.

L423-424: The authors employed a cold restart for the SSP experiments. Did the authors examined the duration required for deep soil spin-up? Why not use soil moisture from a historical RCM run in 2014 or ERA5 reanalysis as initial conditions for the SSP experiments?

Section 4 Evaluation methods: these evaluation methods were already used in previous sections. It would improve clarity to present this section earlier in the manuscript.

L453-456: RMSE and PSS are typically used to assess model performance in simulating individual variables. However, it remains unclear how overall RCM performance in simulating multiple variables is determined. Did the authors normalized the biases/RMSEs when sum them together? Otherwise, the biases/RMSEs are in different order of magnitude. The authors may consider employing the Model Climate Performance Index (Gleckler et al., 2008) or multivariable integrated skill score (Zhang et al., 2021) for a comprehensive assessment in terms of the model performance in simulating multiple variables.

L699: Please replace "CMPI6" with "CMIP6".

L707-712: Could you explain why projected changes in TAS exhibit distinct spatial patterns between NARCliM 2.0 and NARCliM 1.5/1.0?

Fig.15: The quality of this figure appears low. Why do the stippling areas form very regular circles in the many subpanels, e.g., b, c, e, n, p, t, u, v? Consider presenting these figures as supplementary material and summarizing the statistics using a Taylor diagram.

L804-816: These discussions are somewhat tangential to the study's main focus and could be shortened or omitted. Instead, further investigate/discuss the differences in projected changes in the surface air temperature and precipitation among the three generations of NARCliM. For example, explore why widespread wet biases observed in NARCliM 1.x are substantially reduced in NARCliM 2. Are these biases attributable to GCMs, RCMs, or both?

References

Gleckler et al. 2008: Performance metrics for climate models, J. Geophys. Res., 113, D06104, https://doi.org/10.1029/2007JD008972

Xu Z. & Han Y., 2019: Comments on ‘DISO: A rethink of Taylor diagram’. International Journal of Climatology.40, 2506-2510

Zhang et al. 2021: An improved multivariable integrated evaluation method and tool (MVIETool) v1.0 for multimodel intercomparison, Geosci. Model Dev. https://doi.org/10.5194/gmd-14-3079-2021

General comments:

The authors perform extensive testing of WRF physics schemes for future regional climate projections over SE Australia. Impressively, the model is run at 4km convective permitting resolution. After choosing operational configurations, the authors document the historical biases and future projections. While the analysis is rather simple, it is very helpful that comparisons are made against previous generations of NARCLIM. I think this will form a very important foundational paper. I suggest major revisions based on my comments below, which mostly relate to clarifying important points and improving the presentation and interpretation of results.

Specific comments

-The authors highlight that NarCLIM2 has large improvements in tasmax biases, with small absolute biases of ~0.5K over many regions. Are these biases also evident when downscaling all individual GCMs, or simply in the ensemble mean? This relates to the order of operations of where the bias is computed (i.e. before or after the multi-model mean is computed). My concern is that there may be cancelling of biases (e.g. if one downscaled model has a warm bias and the other a cold bias). Can the authors confirm that this is not simply cancelling of biases? Related to this, showing biases for each downscaled model (perhaps in Supplementary material) would help to confirm this.

-Some discussion of observational uncertainty seems warranted, especially if model biases are truly approaching 0.5K.

-The text and figures swap between K and Celsius units, best to choose one.

-Obviously a large effort has gone into producing the convection-permitting resolution model output. However, the improvements are mostly seen in temperature and not in precipitation. Perhaps this is because the focus here is on evaluating mean precipitation and not extremes? Can the authors comment further on this? Referring and discussing other international literature here would be useful also.

-On statistical significance. My personal view is that statistical significance is generally misunderstood and misinterpreted in climate science. However, I do think using significance in terms of model agreement is much more defensible (as you have done on top of this). If statistical significance is used, the authors also need to account for multiple testing (e.g. via the false discovery rate), which does not appear to be done:

https://journals.ametsoc.org/view/journals/bams/97/12/bams-d-15-00267.1.xml?tab_body=abstract-display

-In Figure 15, is there an understanding of why the projections for ACCESS-ESM1-5 projections are so dry? Presumably this is in the GCM also? Do we know why that is from the physical perspective?

-Table 1 is very helpful. Can an extra row on computational resources (core hours) be added? This would help emphasise how much more of an effort going to 4km resolution is.

-Figure 4, for precip, are the units mm/day?

-Figure 9 (and others), I found it difficult to see the stippling/hatching. The resolution of the file was low (not sure if this was an issue with the pdf preprint?) but please ensure that high resolution figures are used and that the journal isn’t compressing these in the final version. The resolution is particularly low for Figure 15 and very difficult to read.

-I think in some figures there is a lot more repetitive text than there needs to be. Rethinking the layout headers of certain figures would help. For example, Figure 9 and 12, (Annual, DJF, JJA) could simply be headers at the top of each page, and the different versions of Narclim could be along the LHS of page. The text is often also too small to read. E.g. the colorbar caption in Figure 15 is excessively long and this information could simply be in the caption.

Review of Preprint gmd-2024-87: “Design, evaluation and future projections of the NARCliM2.0 CORDEX-CMIP6 Australasia regional climate ensemble” by Virgilio et al.

General comments

The authors present the regional climate model NARCliM2.0 and evaluate it using various GCM and RCM ensembles, as well as its precursor versions 1.0 and 1.x. The research topic is highly interesting, and the research work has been conducted meticulously and comprehensively, making it very valuable for regional climate model evaluation and future climate projections in Australia. The research framework is also inspiring for regional climate science, particularly for other regions with large populations. The manuscript is well-written and well-structured. In conclusion, I recommend publication in GMD after the specific comments listed below have been addressed.

Specific comments

Line 81: “and 3) summarise the climate projections produced by CMIP6-NARCliM2.0 and how these” to “3) summarise the climate projections produced by CMIP6-NARCliM2.0 and how these”.

Line 83-88: “section x.” to “Section x”. Please check all “section x.x” and “sect. x.x” in the manuscript.

Line 108-109: “NARCliM2.0 RCMs have a 20 km resolution CORDEX-Australasia domain (versus 50 km) and 4 km (versus 10 km) domain over southeast Australia and use 45 (versus 30) vertical levels”. The horizontal resolution in NARCLiM2.0 has more than doubled resolutions, yet the vertical resolution is from 30 to 45 vertical levels. What do authors think of the choice of 45 levels instead 60 or even more?

Line 142: “manuscripts describe elements shown in Figure 2, and which are therefore only summarised briefly in”, remove “and”.

Line 164-167: “The performances of the different test RCM configurations are evaluated, ultimately selecting a subset of seven RCMs for subsequent downscaling of ERA5 reanalysis and comprising the CORDEX evaluation experiment.” To "The performance of the different test RCM configurations is evaluated, ultimately leading to the selection of a subset of seven RCMs for subsequent downscaling of ERA5 reanalysis as part of the CORDEX evaluation experiment”.

Line 170: ‘production’ should be “production”. Please check all ‘something’ in the manuscript.

Line 190-191: “Non-normally distributed variables (e.g. snow depth and precipitation) are checked for global minima and maxima only.” To "Non-normally distributed variables (e.g., snow depth and precipitation) are checked only for global minima and maxima." Please check all “e.g.” in the manuscript.

Line 201: “Check that changes over time are within realistic ranges (i.e. assess temporal gradients).” To “Check that changes over time are within realistic ranges (i.e., assess temporal gradients).” Please check all “i.e.” in the manuscript.

Line 354-355: “Some studies have shown using this option improves modelling of soil moisture (e.g. Zhuo et al., 2019).” to “Some studies have shown that using this option improves the modeling of soil moisture (e.g., Zhuo et al., 2019).”

Table 9: I am confused about how exactly the “R1-R7” RCMs are shortlisted. It said in Line 609 that “RCMs are shortlisted from the set of 20 if they rank highly for both performance and independence”, but it is not clear how the RCMs are ranked from “R1” to “R7”. Please explain it in more detail.

Figure 15: there are many subfigures and their titles are not easy to read. Please consider improve the visualization.

Line 777: “with 4 RCMs using BMJ, 2 RCMs using Tiedtke, and 1 using Kain-Fritsch.” Please give the references to the cumulus parameterisations.