the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 27 Mar 2023
The second Met Office Unified Model–JULES Regional Atmosphere and Land configuration, RAL2
Mike Bush
Ian Boutle
John Edwards
Anke Finnenkoetter
Charmaine Franklin
Kirsty Hanley
Aravindakshan Jayakumar
Huw Lewis
Adrian Lock
Marion Mittermaier
Saji Mohandas
Rachel North
Aurore Porson
Belinda Roux
Stuart Webster
Mark Weeks
Download
- Final revised paper (published on 27 Mar 2023)
- Preprint (discussion started on 30 Sep 2022)
Interactive discussion
Status: closed
-
RC1: 'Comment on gmd-2022-209', Anonymous Referee #1, 30 Oct 2022
General comments
This paper describes the changes to the regional atmosphere land (RAL) configuration of the Met Office Unified Model and demonstrates the impact of the changes for various variables, regions of the world, seasons, and case studies. Effective use is made of scorecards to highlight the nature of the changes from RAL1 to RAL2. Overall RAL2 provided more improvements than detriments.
These results demonstrate the large effort required by the Met Office and its partners to continuously improve the simulation of the atmosphere-land system. The fact that the differences between RAL1 and RAL2 were not very large (even if they were statistically significant) highlights just how far the community has come in recent decades in producing excellent numerical weather forecasts. This paper makes a useful contribution to documenting progress in NWP development.
When writing about a complex NWP model it’s difficult to be brief and still convey complete information (or at least cite where the reader can find out more). Most of my specific comments relate to incomplete information that needs to be better described or defined.
The English was of good quality, and the graphics were well presented.
Specific comments
- p.4, lines 20-27 – The overlap between liquid water and ice phases is “limited” but you don’t describe how. Is there a cap on the percent overlap? Is this documented elsewhere and could be referenced?
- p.5, lines 12-14 – since you show results for addition of the Leonard terms in Figs. 5 and 6, it would be good to give more explanation here of what the Leonard terms actually refer to.
- Table 2 – what are bLEM and cLEM?
- p.6, line 12 – seems odd to have “(no reference)” – consider omitting
- p.6, eq.(1) – what is Γ?
- p.6, lines 24-25 – why does fixing the multilayer snow scheme allow reintroduction of graupel? Does it form on the snow surface?
- p.7, bottom – the scorecards in the figures use 10.5 km scale (7 grid-lengths). It would be good to explain why this particular scale was chosen for evaluation.
- p.7, line 18 – define GPM
- p.10, line 1 – this should be “the improvement in performance in Winter is much better than the improvement in performance in Summer”.
- p.10, lines 13-14 and Fig. 12 – do the model results in Fig. 12 correspond to the large rectangular domain over Darwin or for the circular domain of the radar? Please clarify, including in the caption for Fig. 2. If the model results are for the larger domain, how much difference from the radar could be explained by sampling different areas?
- p.11, lines 4, 9, 10, 14 – define GWP, LMI, ENLS, QCF
- p.11, lines 17-18 – readers may not know where these regions are – it would be better to say what part of India (northern, etc.)
- p.12, lines 6-10 – do you think the implementation of RAL2 in operations with only 70 levels rather than 90 levels (as shown in the results in this paper) has much effect on the improvements over RAL1? If this has been tested it would be good to say a bit more about it.
- p.A table of acronyms is provided in Appendix 2 but never referred to in the text.
Technical corrections
- p.1, line 18 and elsewhere – There are too many parentheses in the in-line citations. This should be (e.g. Baldauf et al., 2011; Brousseau et al., 2016; Bengtsson et al., 2017; Klasa et al., 2018).
- p.2, line 6 – Fix spelling of ‘the’
- p.2, line 16 – RAL has not yet been defined in the body of the paper
- p.2, lines 30, 32 and elsewhere – Instead of “Sect” write “Section”
- p.4, line 10 – fix “but extensively modified is used”
- p.4, line 20 – do you mean “a change in RAL2-M”?
- p.5, line 21 – change “RAL2 to use” to “RAL2 uses”
- Tables – remove “%” from captions
- p.9, line 6 and 8, and elsewhere in case studies – remove the “0” from “04th December 2019”
- Results sections – no need to capitalise Winter and Summer
- p.9, line 23 – should be “Figure 11 shows RAL2 outperforms RAL1…”
- p.10, line 20 – rather than “in the longer time” give the hours for which RAL2-T performance is better than RAL1-T
Citation: https://doi.org/10.5194/gmd-2022-209-RC1 -
AC1: 'Reply on RC1', Mike Bush, 21 Dec 2022
Thank you very much for your comments. Corrections have now been made to the manuscript. My replies to each of the specific comments are in bold.
Specific comments
- p.4, lines 20-27 – The overlap between liquid water and ice phases is “limited” but you don’t describe how. Is there a cap on the percent overlap? Is this documented elsewhere and could be referenced?
see Appendix of Abel et al. (2017) for more details of the modification to the cloud scheme.
- p.5, lines 12-14 – since you show results for addition of the Leonard terms in Figs. 5 and 6, it would be good to give more explanation here of what the Leonard terms actually refer to. The "Leonard term" is an extra subgrid vertical flux that accounts 15 for the tilting of horizontal flux into the vertical by horizontal gradients in vertical velocity. Hanley et al. (2019) found that including this extra term in the Met Office UKV model reduces the peak vertical velocity within updrafts, leading to a reduction in condensation. As a result, the number of grid points with moderate to high rainfall rates, which are overrepresented by the UKV, are also reduced.
- Table 2 – what are bLEM and cLEM?
Replaced Stability function definitions involving bLEM and cLEM in Table2 with "conventional" and "standard". Text now reads: There are two differences in the representation of turbulence between RAL2-M and RAL2-T, namely in the form of the unstable stability functions and in the free-atmospheric mixing length. Both give enhanced turbulent mixing in RAL2-T compared to RAL2-M. RAL2-M uses the Brown (1999) “conventional” function, the same as GA7, while RAL2-T uses the Brown (1999) “standard” function.
- p.6, line 12 – seems odd to have “(no reference)” – consider omitting
omitted
- p.6, eq.(1) – what is Γ?
Equation deleted and text rewritten as there is a very small impact in RAL2. We don’t have a convection scheme, so the parametrized downdraught gustiness velocity scale, w_c is zero (and so gamma is irrelevant). Text now reads: For lower mean wind speeds, the effect of subgrid convective boundary layer gusts on the surface turbulent fluxes is included via a term proportional to the convective velocity scale in the calculation of the friction velocity. For RAL2 we reduce the strength of that term by a half, to then match GL7.0.
- p.6, lines 24-25 – why does fixing the multilayer snow scheme allow reintroduction of graupel? Does it form on the snow surface?
Improvements to the treatment of lying snow in RAL2 are achieved by introducing a representation of melting of the snow pack from the base over warm ground, as the original code in JULES allows melting only from the surface. Previously it was necessary to remove graupel from the precipitation reaching the surface as the omission of melting from the base resulted in unrealistically prolonged retention of thin layers of frozen precipitation. This modification allows the reintroduction of graupel into the precipitation reaching the surface.
- p.7, bottom – the scorecards in the figures use 10.5 km scale (7 grid-lengths). It would be good to explain why this particular scale was chosen for evaluation.
This particular scale was chosen for evaluation as Mittermaier and Csima (2017) showed that all variables benefited from the use of at least a 3 x 3 neighbourhood, whilst neighbourhoods which are too large may be detrimental for some variables, including temperature.
-
p.7, line 18 – define GPM Global Precipitation Measurement (GPM) IMERG satellite data based product (Huffman, 2015, 2017), Skofronick-Jackson et al., (2017) is used. - p.10, line 1 – this should be “the improvement in performance in Winter is much better than the improvement in performance in Summer”. Corrected.
- p.10, lines 13-14 and Fig. 12 – do the model results in Fig. 12 correspond to the large rectangular domain over Darwin or for the circular domain of the radar? Please clarify, including in the caption for Fig. 2. If the model results are for the larger domain, how much difference from the radar could be explained by sampling different areas?
Figure 2 caption now reads: The CPOL radar location is denoted by the black triangle and its coverage by the area within the circle of dashed lines, which is the area used for the analysis presented in Figure 12.
- p.11, lines 4, 9, 10, 14 – define GWP, LMI, ENLS, QCF
ENLS : Earth Networks Lightning Sensor; QCF: Cloud Frozen ice content; GWP: Graupel Water Path; LMI : Lightning Mapping Image
- p.11, lines 17-18 – readers may not know where these regions are – it would be better to say what part of India (northern, etc.)
Bihar and Uttar Pradesh (Eastern India) and also very few flash strikes over the Rajasthan-Madhya Pradesh border (north-west India).
- p.12, lines 6-10 – do you think the implementation of RAL2 in operations with only 70 levels rather than 90 levels (as shown in the results in this paper) has much effect on the improvements over RAL1? If this has been tested it would be good to say a bit more about it.
Both level sets have a very similar number of near-surface levels with both having 28 levels below 3km and L90(67t,23s)40 only having one extra level by 10km asl. As a consequence of this, the impact of 10 L90(67t,23s)40 was found to be very small for this UK specific application.
- p.A table of acronyms is provided in Appendix 2 but never referred to in the text.
Now referenced at the beginning of chapter 2
Technical corrections
All technical corrections have been made.
- p.1, line 18 and elsewhere – There are too many parentheses in the in-line citations. This should be (e.g. Baldauf et al., 2011; Brousseau et al., 2016; Bengtsson et al., 2017; Klasa et al., 2018).
- p.2, line 6 – Fix spelling of ‘the’
- p.2, line 16 – RAL has not yet been defined in the body of the paper
- p.2, lines 30, 32 and elsewhere – Instead of “Sect” write “Section”
- p.4, line 10 – fix “but extensively modified is used”
- p.4, line 20 – do you mean “a change in RAL2-M”?
- p.5, line 21 – change “RAL2 to use” to “RAL2 uses”
- Tables – remove “%” from captions
- p.9, line 6 and 8, and elsewhere in case studies – remove the “0” from “04th December 2019”
- Results sections – no need to capitalise Winter and Summer
- p.9, line 23 – should be “Figure 11 shows RAL2 outperforms RAL1…”
- p.10, line 20 – rather than “in the longer time” give the hours for which RAL2-T performance is better than RAL1-T
Citation: https://doi.org/10.5194/gmd-2022-209-AC1 - p.4, lines 20-27 – The overlap between liquid water and ice phases is “limited” but you don’t describe how. Is there a cap on the percent overlap? Is this documented elsewhere and could be referenced?
-
RC2: 'Comment on gmd-2022-209', Anonymous Referee #2, 31 Oct 2022
*General Comments*
This manuscript describes a new regional atmospheric and land-surface modeling configuration of the UK Met Office's weather forecasting system. The paper is well structured and provides an excellent summary of the new configuration. I believe it will be of significant interest to the wider NWP modeling community and vital for users of the UM/Jules. I recommend this be published with minor corrections as laid out below.
*Specific Comments*
Where case studies and trials of the complete RAL2 configuration are described for the UK (Sec 3.3, 3.4, 3.5) these have been related back to the one or more of the individual changes (i.e. the performance described in Sec 3.2). However, the other cases are not so well linked. For example:
a) in the MCS case it is not clear whether the improvement in fractional coverage shown in Fig 12 is to be expected, nor which of the science changes might have caused this.
b) in the South East Asia cases, what might be causing the degradation in FSS during spinup, or rather, why does RAL2-T take longer to spinup than RAL1-T. Is it due to the BL stochastic perturbations (Table 2) or was there no change in this from RAL1 to RAL2?
c) for the Indian lightning cases, I assume the changes seen here are partly/mainly?? due to the liquid and ice phases in the cloud scheme (Sec 2.6). There is mention of a reduction in graupel and ice water paths, but it's not obvious what is causing this. Also, I assume the results shown are after the reduction in GWP threshold was applied to the RAL2-M configuration; if so it would be helpful to see what the results from the 'standard' configuration looked like. This threshold adjustment seems arbitrary, why tune RAL2 to RAL1 output? From Fig 14 & 15, it looks like both RAL1 and RAL2 are producing higher flash counts than the obs (albeit with lower spatial coverage) so its not clear why they should be increased.
*Technical Corrections*
(Apologies, but some of these get rather pedantic...)
Pg2, Ln 5: The 'regional model' you refer to here has not been defined yet (except in the abstract). It needs to be defined as UM/JULES or similar. (Also 'trhe world' sp.)
Pg2, Para 3/4: I think it would make sense to introduce RMED at this point (rather than just as a prefix when introducing the toolbox). Or if not here, then around line 7 on page 7 where the regional model evaluation process is introduced.
Pg3, Ln2: When introducing section 2 I think it is implied that when not stated explicitly, then the definitions given haven't changed from RAL1 to RAL2 (e.g. Sec 2.2, 2.3, 2.4). It would be helpful to make this explicit.
Pg3, Ln6: The SOCRATES url in the footnote is not accessible (I assume this is an internal website). Remove.
Pg3, Ln 19: Delete 'basic underlying'.
Pg6, Ln25: Thin snow albedo bugfix: this should either be expanded on or removed. Did the bug have a noticeable effect?
Pg6, Ln26: Similarly the snow grain growth. Is there a reference for this? Why was the treatment revised?
Pg8, Ln4: Delete "simple"
Pg8, Sec 3.2: Can you confirm that all the scorecards and results (Fig 4, 5, 6, 7 & 8) are based on all 100 UK cases? And do they include the Darwin cases? How many Darwin cases are included?
Pg9, Ln1: Clarify that "performance" in winter is not necessarily better than summer. "... by season reveals that the IMPROVEMENT IN performance in winter is LARGER than in summer..."
Pg9, Ln6-8: "04th" shouldn't have the "0"
Pg10, Ln7: I think "though" is meant to be "through"
Pg11, Ln22/Ln25: "RAL2 Science" and "Data Assimilation" don't need capitals
Pg11, Ln25: "(the latter we refer to as case studies)" This is probably not necessary. But if it is important should be pointed out in the relevant earlier section, not in the conclusion.
Pg12, Ln7: change "RAL2-M science is running 24/7 in" to "where RAL2-M science is used in"
Pg12, Ln6-8: "Operationally" doesn't need capitals and '04th' should be '4th'
Pg12, Ln13: change "the partnership" to "the UM Partnership"
Acronyms not defined:
Pg2, Ln4: UM (except in abstract where an acronym isn't needed, should be defined in main text)
Pg6, Ln20: LES & PBL
Pg7, Ln18: GPM
Pg11, Ln4: GWPSeasons shouldn't be capitalised (e.g. winter and summer throughout page 9, and Fig 9-11 captions)
Captions to Fig 3-6 have unnecessary capitals. I don't think they need to match the 'Description' field in table A3. Also too many capitals in caption of Fig 14.
Citation: https://doi.org/10.5194/gmd-2022-209-RC2 -
AC2: 'Reply on RC2', Mike Bush, 28 Dec 2022
Thank you very much for your comments. Corrections have now been made to the manuscript. My replies to each of the specific comments are in bold.
*Specific Comments*
Where case studies and trials of the complete RAL2 configuration are described for the UK (Sec 3.3, 3.4, 3.5) these have been related back to the one or more of the individual changes (i.e. the performance described in Sec 3.2). However, the other cases are not so well linked. For example:
a) in the MCS case it is not clear whether the improvement in fractional coverage shown in Fig 12 is to be expected, nor which of the science changes might have caused this.
The increase in fractional coverage of cloud and rain in the MCS case is likely to be predominately a result of the introduction of the Leonard terms (ticket 27). This science change has the impact of reducing the number of small convective cells and heavy rain rates (Fig.6) and helps to produce better cloud cover, as indicated in Figure 5. However, to be able to quantify the contribution of each science change to the change in, for example cloud cover, would be require many more simulations that include both individual and combinations of changes, which is outside the scope of this study. Therefore we propose that the text remains unchanged as we don't have strong evidence to confidently address this comment.
b) in the South East Asia cases, what might be causing the degradation in FSS during spinup, or rather, why does RAL2-T take longer to spinup than RAL1-T. Is it due to the BL stochastic perturbations (Table 2) or was there no change in this from RAL1 to RAL2?
There was no change to the BL stochastic perturbations between RAL1 and RAL2. The change is likely to be predominately a result of the introduction of the Leonard terms (ticket 27). The spin-up with RAL2-T is more muted than with RAL1-T. The reduced rainfall amounts means that the absolute thresholds are worse (not significantly though), but the percentile thresholds are significantly better. We propose that the text remains unchanged as we don't have strong evidence to confidently address this comment.
c) for the Indian lightning cases, I assume the changes seen here are partly/mainly?? due to the liquid and ice phases in the cloud scheme (Sec 2.6).
The change to limit the overlap between the liquid water and ice phases was only applied to RAL2-M and is not applicable to RAL2-T.
There is mention of a reduction in graupel and ice water paths, but it's not obvious what is causing this. Also, I assume the results shown are after the reduction in GWP threshold was applied to the RAL2-M configuration; if so it would be helpful to see what the results from the 'standard' configuration looked like.
The GWP threshold reduction was not applied to RAL2-M. All experiments were performed using RAL2-T.
This threshold adjustment seems arbitrary, why tune RAL2 to RAL1 output? From Fig 14 & 15, it looks like both RAL1 and RAL2 are producing higher flash counts than the obs (albeit with lower spatial coverage) so its not clear why they should be increased.
The threshold adjustment to RAL2-T was made to increase the area coverage of the moderate lighting flash counts distribution which was for many cases even less than observations at many locations. By reducing the threshold, not only the area coverage, but the intensity also increases at some locations.
Agree that both RAL1-T and RAL2-T are producing higher flash counts compared to observations after the tuning of RAL2-T. Our strategy is to reduce the missing events at the cost of some false alarms. However, we have conducted experiments with many lightning events of light, moderate and extreme intensity, and the overall objective scores (Reply-to-reviewer Table, see attachment) are favouring RAL2-T compared to RAL1-T. Hence definitely there is an improvement over RAL1-T.
Reply-to-reviewer Table (see attachment) Objective scores of daily accumulated lightning for RA1T and RA2T (8-12 August 2019) over Indian domain (Root mean Square error, Correlation coefficient, Bias, Multiplicative bias, Mean forecast, Mean observation).
*Technical Corrections*
These have been made.
-
AC2: 'Reply on RC2', Mike Bush, 28 Dec 2022
-
RC3: 'Comment on gmd-2022-209', Anonymous Referee #3, 31 Oct 2022
General Comments:
The paper is a good follow-on paper to the UK Met Office’s regional forecasting system RAL1 science configuration published by the same author in 2020. The case studies covered a wide range of phenomena and sufficient sample size to make convincing conclusions on model performance. The paper also highlights the specialised and wide range of physics research which can improve model performance over and above the dynamical core of NWP systems. Worth noting is that the proposed microphysics changes do not improve performance in all model outputs; in some aspects, RAL1 outperforms RAL2. This information is always valuable for operational centres to consider when planning model upgrades. Furthermore, the paper highlights the benefits of consortia in NWP development.
Specific Comments:
To fully understand some references and model configurations, one needs to have a good understanding and knowledge of the contents and results of the 2020 paper. Therefore, some technical references or specific comments might be seen as unnecessary.
- During the explanation of rotated v. unrotated grids, it is mentioned that Australia is at lower latitudes (p 3 line 7). However, the continent lies between 10-43 degrees south. Therefore, it might be worthwhile to give a short explanation/reasoning why regional domains for parts of Australia does/does not apply a rotated grid. Alternatively, since the domain is not yet defined y p3, mention that the domain of interest in this paper lies within the tropics; hence the model is unrotated.
- Remove “%” in all table headings
- P2, L13: Do forecasters also conduct subjective assessments at NCHWRF and BoM, or is this done only at the Met Office?
- P5, Table 2: Define bLEM and cLEM .
- P6 Equation 1: Define Γ.
- P6,L24: Briefly explain on how/why changes in surface snow settings affect graupel in precipitation.
- P7,L20: Explain why scorecards use 10.5 km (7 grid-lenghts); possibly related to the scale of the synoptic observations?
- In Section 3.2: It is overall not clear which case studies are evaluated and presented in Figures 4-8. It is stated that RAL1-M were evaluated for UKV and Darwin, but not the number of cases for Darwin.
- Which case studies are included in the results shown in the scorecards, and how many cases from each partner were included? Apart from the 100 cases at the Met Office, it is not stated how many cases were conducted by NCMWRF and BoM.
- In the scorecards, does “precipitation” include snowfall?
- Figure 6: Does the possible weakness of GPM capturing higher rain rates contribute to this result? Or is this not applicable in higher latitudes?
- Figure 6: Was the model resolution upscaled to the GPM resolution?
- P8,L26: To which Figure does this paragraph refer? How did you distinguish between Met Office and Darwin cases?
- P9,L1: No plausible explanation is given for the performance increase in winter and not summer months.
- P9,L23: Define the grid resolution of MOGREPS-UK since, in Figure 11, 7 grid-lengths now equal 15 km as opposed to 10.5 km in previous scorecards.
- P10,L4: Is the more extensive fog in the RAL-M configs more accurate? Also, is it more extensive in both temporal and spatial scales compared to the observed?
- P10,L13: “Figure 2 shows..” – is the results in 3.7/Figure 12 only for the circle (radar coverage) in Figure 2 or for the whole domain as shown in Figure 2? Please clarify.
- P11: Define GWP, LMI, ENLS, QCF
- Table A2 is not referred to in the text
Technical corrections:
- Figures 1 and 2: use either “orography” or “height” in both figures for consistency.
- P1, L18: It is suggested that the term “National Hydrological and Meteorological Services” (NHMS) be used to align with international practice (also used by the WMO).
- P1, L19: Correct the reference syntax; include all references in one bracket.
- P2, L6: Typing error “trhe”
- P2, L8: Suggest shortenening the sentece: “…10 years. This strategy includes…”
- P2, L21: “The systems run in variable…”
- P2: L23: Suggest defining UK (might be pedantic)
- P3,L5: “…centre of the regional model domain…”
- P3,L10: Swap RAL2 and RAL1 in the sentence to agree with the order as shown in Table 1.
- P3,L17-8: Add a reference for this statement
- P4,L6: The link is to an internal site at the Met Office and is inaccessible to the public. Suggest using the University of Leeds site for the PDF document (http://homepages.see.leeds.ac.uk/lecsjed/winscpuse/socrates_techguide.pdf) website OR the SOCRATES github page (https://execlim.github.io/Isca/modules/socrates.html) similar.
- P4,L10: Give an example of how the single moment microphysics has been “extensively modified.”
- P4,L20: Provide a reference or briefly explain why a limit is applied and the possible ranges between liquid water and ice phases.
- P5,L3: Reference for statement ending with “… kilometre scale models”
- P5,L5: Suggestion for consistency, use “s” in both parameterisation” and “parameterise”, although “z” is an acceptable British spelling.
- P6,L8: Has GL been defined?
- P7,L31: Again remove “%” in table title
- P7,L9: Suggest replacing “variety” with “diverse” to eliminate the use of “variety” in consecutive sentences.
- P7,L10: Define RMED earlier since it refers not only to the Toolbox but also to the larger collaboration in the UM Partnership on regional model evaluation.
- P7,L18: Define “GPM”.
- P8,L20: “observations” and not “obs”.
- P8,L28: Typing error “togther”.
- P9,L1: “reveals that almost all parameters improved during the winter months, as opposed to summer”.
- P9,L8: 4th
- P9,L6, L11 and L12: Use dates syntax consistently and throughout the rest of the paper.
- P9,L11: What were the results for the winter months, or was the sample size too small?
- P10,L24: Suggest changing to “model for a large domain covering …”, indicated in Figure 14.
- P10,L28: It might be worth mentioning why the results from t+72 onwards are not of significance or not described in the results.
- P11,L7 and 17: Where are these regional areas located? Possibly show on a map to support the reader in interpreting the results.
- P11,L26: U.K to UK
- P11,L19: “The maximum flash counts…”
- P31: Figure 11, 13 – Reduce the space between the title and graph.
- P34: Figure 14: Panels a) to i) is not defined or referred to in the text/Figure title. Suggest using the panel references when discussing the results for ease of reference to the maps. Conversely, Figure 15 for the second case study does not indicate panel numbering.
Citation: https://doi.org/10.5194/gmd-2022-209-RC3
Peer review completion
