Manuscript title: Evaluation of CORDEX ERA5-forced ‘NARCliM2.0’ regional climate models over Australia using the Weather Research and Forecasting (WRF) model version 4.1.2

Authors: Di Virgilio et al

Reviewed by: Sugata Narsey (BoM)

Recommendation: Minor revisions

This manuscript documents the evaluation of the NARCliM2.0 regional climate model (RCM) driven using ECMWF Reanalysis v5 (ERA5). The manuscript is well-written, and systematically addresses key aspects of the evaluation of their model. They go further than a basic evaluation, providing useful insights into the regional impacts of multiple parameterisation configurations of the model. They find that changing the physics choices in their model can have quite dramatic effects on regional climate biases for Australia. A nice addition to this study is their analysis of the relative sources of bias estimated by interchanging their RCM with the previous version of the NARCliM RCM, and also interchanging the driving ERA5 reanalysis with the ERA Interim reanalysis data previously used. By doing so they find that the RCM set-up appears to be a stronger influence on the mean state bias in their regional climate simulations compared to the choice of driving reanalysis data. This manuscript forms an important scientific basis for the production of a nationally significant projections dataset and is an important contribution to regional modelling for the Southern Hemisphere.

I have some comments around evaluation choices, and around specific wording especially with regards to claims of improvement for precipitation since the biases over northern Australia appear significant. However overall, this manuscript is appropriate for publication, and my recommendation is for minor revisions.

Main comments:

1. The distribution plots show nationally aggregated data, however I find this problematic since the map plots show that Tmax and precipitation in particular have opposing biases in the northern and southern regions (Fig 3 and Fig 7). Additionally, the bimodal distribution of Tmin in Fig 4 might be a function of mixing two climatically distinct regions. Why not split it into at least two regions? Then you can clearly state the improvements for the southern parts of Australia.
2. The evaluation conducted here focuses on rainfall and temperature, which I agree are the most important variables to consider. However, some investigation of the circulation state in the RCMs may be of use to help understand the systematic biases, for example over northern Australia (R3-7), and over SE Australia (R2-4).
3. The statements around general improvements in precipitation are not well-founded in my view, since the dry biases over northern Australia are large compared to NARCliM1.5 runs. I would prefer if the claims were either made specific to the inner domain, or else more carefully described in this manuscript. Alternatively the authors can show whether the biases in the NARCliM2.0 runs (especially for northern Australia) are actually smaller as a percentage of annual mean climatological rain.
4. It is outside the scope of this study, however it would be interesting to know if the different physics configurations and their associated regional climate biases have much bearing on the future change signal in the model when holding the driving global model data constant. Similarly, it would be interesting to know if the evaluation of the ERA5 runs with different physics configurations translates in an evaluation of the CMIP6 historical scenario runs.
5. Also outside the scope of this study, but it would be interesting to intercompare the dry-bias tendency over northern Australia in the NARCliM2.0 runs with other regional simulations using different models that introduced similar dry biases for the Australian monsoon. Although such a bias may be undesirable, there is a real opportunity here to shed light on some fundamental characteristics of the dynamics Australian monsoon, in particular the feedback mechanisms associated with land surface behaviour, convection, and large-scale circulation.

Specific comments:

L63-64: It’s now May 2024 and this statement is outdated; I believe the BARPA paper is now published (https://gmd.copernicus.org/articles/17/731/2024/gmd-17-731-2024-discussion.html) and there may be others by now. Might be worth a quick search.

L205-208: Did you follow the same experiment design in all other respects except for run length? Fig 13 shows the inner domain. Are these ERA5 swapped with ERA Interim sensitivity experiments conducted at the fine-scale for the inner domain? Maybe it’s specified somewhere but I couldn’t see it. Worth clarifying here.

L210-211: The short periods are probably fine, but why not just do a quick bootstrap check to see how representative 14-month periods are for rainfall in the longer run period using either AGCD or your simulations?

L231-233: Based on Fig 3 it seems Fig 2 might obscure some compensating biases between north and south. Is this the case?

Fig 3: The stippling is difficult to see. Can you improve somehow?

L270 and Fig 4: Is the bimodality due to mixing different climate zones?

Fig 6: Why not show log(P)? Might be easier to see differences and similarities.

Fig 7: Would the biases over northern Australia look this dramatic if you showed it as a percent of AGCD climatology? It’s hard to know for example which absolute bias is more concerning between runs, since a small absolute bias in the dry regions may matter more than a large absolute bias in the monsoon region.

Fig 13: If this is not 4km explicit convection runs than perhaps show larger domain? Otherwise, see comment for L205-208 above.

L459: The claim of general improvement in precipitation and even max temperature is not quite true in my opinion. The bias over northern Australia appears large and systematic. I think it is appropriate to claim general improvement over the inner domain though. See main comment 3.

Section 4.1: you note the dry bias vs wet bias may relate to microphysics scheme. Looking at fig 7 the three runs (R2-4) with MYNN2 boundary layer scheme are all wet biased over SE Australia. Is this a coincidence?

L492: suggest “especially over northern Australia where all other runs contain a systematic dry-bias”.

L568: Again, I don’t agree with this claim of general improvement for precipitation.

L577: It also appears important here at coarser scales when precipitation is parameterised, based on Fig 7.

L584-585: Potentially also in the rainfall biases, especially where dynamical feedbacks are known to occur in the real world such as over northern Australia during the summer monsoon season.

This paper provides an evaluation of the representation of precipitation and diurnal screen-level temperatures from a set of 7 model configurations of the NARCLIM2.0 regional climate model. By benchmarking model performance against a previous version of NARCLIM and repeating the analysis of a previous paper, the authors follow an objective, pre-determined framework, which is to be commended. NARCLIM2.0 is shown to have a reduction in outlier model configurations with large temperature biases in excess of 2K. Their results highlight model dependence, particularly of precipitation, on the choice of parametrisation schemes and identify a pervasive dry bias in northern Australia.

I have comments around the description and justifications of model configuration choices and some more minor comments on the presentation of the results. Overall, this is an important and well written manuscript suitable for publication following these revisions.

General comments

There are a large number of models, statistics and maps presented in this paper which makes it difficult to form an overall view of the improvement in model performance across generations. I would suggest you include a summary table of the mean absolute error, bias magnitudes and Perkins Skill Scores reported across the paper and supplementary materials.

The text at lines 137-140 suggests that the NARCLIM2.0 model configurations have been selected based on empirical performance during a single year, and that compatibility between parametrisation schemes or recommendations from the WRF model developers may not have been considered. Please add some text to provide assurance that these for each of the 7 selected configurations, the combination of parametrisation schemes is physically sensible. For example, have these combinations been used and recommendations by separate studies, developed and tested in combination, or at least not contain schemes developed specifically for use with a different setup or combinations precluded in the WRF user guide? Are the PBL schemes compatible with the surface schemes, and is shallow convection appropriately dealt with by the combination of PBL and convection schemes?

More details on common aspects of the experimental design would be welcome: how are ozone and aerosols represented in these models? How frequently does the SST update? What datasets have been used as static inputs to the land-surface schemes (vegetation fraction etc)?

As the differences between the parametrisation schemes forms a large component of this paper, please provide references for the schemes. Some explanation of the dynamic vegetation scheme would also be welcome.

The selection of RCMs for this study comes across as ad-hoc and incomplete: HadRM3P, RegCM4 and REMO2015 also contributed ERA-interim driven simulations to CORDEX-CMIP5 Australasia but have not been evaluated. Additionally, three additionally, three ERA5-driven CORDEX-CMIP6 Australasia simulations appear to have also been recently published before the submission date. While including extra models at this stage may be out of scope, the paper may sit better in the literature if it focuses purely on NARCLIM/WRF-based models.

On a similar note, you may consider acknowledging that NARCLIM2.0 will contribute to an ensemble of downscaled climate projections for Australasia. (e.g. https://www.sciencedirect.com/science/article/pii/S2405880723000298)

Map quality: stippling is hard to see, while coastlines and state boundaries show up as inconsistently rendered, adding to confusion. Can these be improved? Perhaps the figures would be easier to read if the stippling density was decreased and line thicknesses increased.

Can you provide a recommendation of which of R1-R7 you would recommend to be used in downscaling GCMs going forward?

Line-specific comments

Line 10:  Please be more explicit about what these statistics (0.54K; 0.81K) are. They seem to be from R5 but I'm not sure why (R1 has a lower mean absolute error for the p99).

Lines 11-12 and lines 479-486: I can't see systematic improvement in mean state precipitation of the 7 CORDEX-CMIP6 RCMs over the 6 CORDEX-CMIP5 RCMs. Certainly, WRFJ has a very large wet bias, however the performance of WRFL is comparable to R3 and R4.

Line 194: Please specify the bin width used when calculating the Perkins Score.

Lines 380-383: Please review the meaning in this paragraph as it's confusing. In the first sentence you say the ERA5-driven and ERA-interim driven simulations are similar, in the second you say that the ERA5-driven show large reductions in biases.

Lines 341 and elsewhere: consider saying 'bias magnitudes' (or mean absolute error) over |biases| in the text.

Lines 488-490: I don't agree that the convection-permitting P99s from R3-R7 are markedly improved over WRFK and WRFL: perhaps a little along the coast but it's fairly marginal.

Figure 8-13: are you able to include cutouts of the 20km outer domains of ERA5 R1-R7 in these figures?