Review of "Forecastin contrail climate forcing...pycontrails 0.51.0"

by Engberg et al.

General comments:

The authors describe the implementation of a new tool for contrail-avoiding flight-routing that is aimed at producing forecasts of fields of the so-called Energy Forcing (EF), similar in format to other fields of standard weather maps. These fields can then be used for flight-routing that optimizes flight tracks in a way that the total path-integrated EF is minimal. I am pleased that the authors see the necessity to test their predictions thoroughly with independent data and to investigate the sensitivity to uncertain parameters of their approch. Thus, this development is on a good way and the description of the code fits well to GMD. However, I have two major reservations to the current approach, which should be adressed in the final paper.

Major comments:

1) All the results that this method produces and will produce and many results that are cited are based on the "parametric RF model" by Schumann et al. (2012). This "model" is not a model but a fit to results from 1000s radiative transfer calculations using a large set of profiles as input. It must be recognized that a fit is not a model. I suggest to search the internet for "fit vs. regression". What would be required for the current purpose is a regression rather than a fit. The fit that is used so far has more than 10 free parameters. It is quite possible that this leads to overfitting (that is, fitting of noise). Testing of the fit against independent test profiles has never been performed, as far as I know. Thus it is unknown whether there is overfitting or not. Moreover, there is to my knowledge no analysis of the residuals, whether they are distributed homogeneously or not over the range of input variables. Thus, it is in particular not known, how the fit behaves under conditions that lead to the strongest warming.  There are often statements that 2-3% of contrails contribute 80% of the overall EF, but whether this statement is tenable cannot be judged without an analysis of the residuals. 

I do not expect that the "RF model" will be thoroughly tested for the current paper, but these tests should certainly by placed on the agenda for the near future. For the present paper I expect to see a section in the discussion where these issues are discussed and I expect that statements that are based on results of this RF model are turned moderate.

2) Some parts of the model are much more detailed than others. For instance, details on aircraft/engine combinations up to engine details are required as input in order to make a precise prediction of the emission rate of NvPMs. At the same time, the precision of the weather input is certainly much lower (vertical resolution is low, hourly output, problems with the field of relative humidity, etc.), the ERF/RF ratio can only be estimated, other quantities have a very wide 5-95% confidence interval. It seems that this combination of very precise vs quite imprecise parts may lead to funny results. I was puzzled, on page 22, that in the first paragraph some quite uncertain paprameters are used while in the next paragraph results are given with a very high precision, e.g. 213,357 +- 0.03 kg. Considering, for instance, the range of social carbon cost, roughly 44 to 410 USD, I would suggest that of the 213,357 kg maybe the first digit is valid, but not more. 

I would like to see what the authors think about this mixture of very detailed vs. very uncertain parts of their model.

Special comments and questions:

General: Please be careful to distinguish between strong radiative/energy forcing vs. warming/climate impact. As contrails might have a low efficacy and as that may depend on location and situational circumstances (feedbacks), strong forcing and strong warming are not equivalent. 

L 42: Isn't a negative exponential distribution simply an exponential distribution?

L 44: What exactly is meant with the word "localised"? 

L 50ff: The sentence is a bit misleading. Both satellite images and ground based cameras cannot only observe contrail formation, they see old contrails as well when they move through the field of view. That one is currently not able to integrate RF over a contrail's lifetime, is another - independent - issue. Perhaps it is just infeasible for long-living contrails, but in principle it seems possible to me. I have also problems to see the connection between this sentence and the remaining ones in this paragraph.

L 66 ff: The first contrail avoidance trial was the MUAC/DLR trial, not the American Airlines trial. Moreover, the MUAC/DLR trial is, as far as I am aware of, the only one that was thorougly analysed and the experiment and analysis is published in a peer-reviewd paper by Sausen et al. (Can we successfully avoid persistent contrails by small altitude adjustments of flights in the real world? Meteorol. Z., 33(1), 83-98. 10.1127/metz/2023/1157). This paper instead of the grey literature should be cited here. If you know of other peer-reviewed analyses of such trials, please let the reader know.

L 68 ff: I find the rest of this paragraph a bit too optimistic. It appears as when the list of current problems is quite short and that they are easily solvable by selecting a certain kind of format for model output.  

Figure 3: Please explain the strange structures around x,y=+-10(-7). 

L 365 ff: Please reformulate this sentence "The 2019 ...". It is not clear what you mean.

L 386 ff: It is counterintuitive that areas with high cirrus coverage lead to strongly warming contrails. Please explain.

P 22: The precision of the quoted input and output values does not fit together, see major comment above. 

L 521/22: I am pleased that the authors acknowledge this necessity and agree completely!

L 626: Please try to find a combination of entries in a contingency table that results in ETS=-1. If you find one, please let the reader know.

This study describes a gridded version of the Contrail Cirrus Prediction model and illustrates how it could be used in various applications, including aircraft rerouting for contrail avoidance.

The paper is very well written and presents several important findings for contrail avoidance. Using Cocip to compute a grid of cumulative contrail energy forcing per flown distance is a simple and clever idea. The Monte-Carlo framework for calculating uncertainties is very welcome, because the lack of uncertainty propagation in Cocip is a strong limitation of the original model. The comparison between Cocip and CocipGrid is important to inform a potential operational use of the latter. The examples of CocipGrid-based contrail avoidance are informative illustrations. I have installed and tried CocipGrid and it was straightforward – congratulations to the developers.

Yet, there are aspects of the study that require improvement. Like much of Cocip literature, the paper does not discuss the impact of the choice of model timestep. That discussion needs to happen now, since it has direct implications for the avoidance examples given in section 5. The comparison between Cocip and CocipGrid also needs to be deepened, because the differences are large. For those reasons, and to also address the other comments below, I recommend major revisions.

Major comments:

• Line 116: Cocip timestepping has received little attention in the literature. This study uses a timestep of 300 s (5 min), following Teoh et al. (2023). Schumann (2012) used much longer timesteps but noted that some aspects of the model are timestep-dependent (their section 2.9), unfortunately without discussing the impact on simulated contrail energy forcing. So, I ran CocipGrid on ERA5 data for the case shown in Fig 5, 7 Jan 2019 at 3am, focusing on the North Atlantic at 200 hPa. With a 5-min timestep (dt_integration parameter), I get a domain-averaged energy forcing of 6.7 10⁷ J km⁻¹ and a maximum of 4.8×10⁹ J km⁻¹. Increasing the timestep to 15 min, the numbers become 6.4×10⁷ and 4.1×10⁹ J km⁻¹, respectively. At 30 min, we are down to 6.1×10⁷ and 3.8×10⁹ J km⁻¹. So simply changing the integration timestep decreases the domain average by almost 10% and maximum energy forcing by more than 20%. If CocipGrid is to become the basis for contrail avoidance, that needs to be addressed – otherwise actors will use whichever timestep gives the most convenient answer. Why such a large impact? Can that impact be reduced with further developments? If not, what guidance can be given for choosing the timestep? Are there other unexplored parameters with similarly dramatic impacts on simulated energy forcing?
• Section 4 is very good at describing the different metrics used to compare CocipGrid to Cocip, which is a very important comparison to make, but the discussion of the results feels incomplete. Section 4.2 focuses on the impact of the choice of the number of aircraft groupings. But differences shown on Fig 3a are sizeable and raise several questions:
  • What causes the differences? Is that mainly alpha and f_shear? Line 222 mentioned a calibration of f_shear. Has that been done here?
  • Many flights have zero energy forcing in either Cocip or CocipGrid when the other model has non-zero energy forcing. Why is that? Difference in contrail persistence due to different effective wind shear?
  • And why are the datapoints arranged in square patterns? Is it an artefact of the selection of comparison cases?

Other comments:

• Abstract: The abstract does not say anything of the differences between Cocip and CocipGrid. Those differences are not negligible and that could impact the operational use of CocipGrid, so it is important that the abstract acknowledges that fact.
• Line 125: So Cocip does not account for the impact of underlying clouds other than cirrus? I thought it dealt with underlying cloudiness by using outgoing longwave radiation as input. Is that not the case?
• Line 134: It would be useful to note here that “generally consistent” is a low bar, and, as acknowledged in the conclusion, a proper quantification of Cocip skill compared to observations, all the way to simulated energy forcing, remains needed.
• Tables 2 and 3: It would be useful to have a graphical version of those Tables, showing the mass and nvPM of the individual aircraft types on a plot, to see how well separated the different aircraft groups are. Like Figure 1, but before transformation by the aircraft performance model.
• Line 201: Could point out that the ICAO emission databank is for LTO emissions, hence the need to translate them to the more relevant cruise emissions.
• Line 204: Which aircraft type has the largest market share in each group? It could be good to indicate it in italics in Table 2.
• Figure 1 could do with a more detailed discussion. If I understand well, it was obtained by applying the four assumptions listed in lines 205-212 onto the input data for one aircraft group. I see that the aircraft needs to be lighter to fly higher up, which makes sense. But why are the distributions multimodal? Is that because of the different aircraft types within the group? And why does the nvPM EI distributions change with altitude? Because mass has changed?
• In addition, the use of Figure 1 in Section 3.4 is ambiguous. My understanding is that it is an example of a multivariate distribution, and that the Monte Carlo analysis relies on many similar figures. Is that correct? It would be good to clarify the role of Figure 1 in that section.
• Line 225: What is meant by “is set up”? Is that something you did for that section, or some built-in capability of the model? Some practical information would be useful here.
• Line 234: Are 100 Monte-Carlo simulations enough to get robust uncertainties? That seems like a small number given the number of uncertain parameters and their uncertainty ranges.
• Line 260: But to determine the proportion of flights that exert 80% of total annual energy forcing, the model needs to be able to simulate the whole distribution properly. Unless you take an approximated view of the percentile boundaries?
• Lines 377-378: Is this statement an introduction to what follows? What is the consistency with Bier and Burkhardt (2022) and Gettleman et al. (2021)? Are you talking about qualitative or quantitative consistency?
• Line 476: Are those findings based on the one-day case shown on Figure 9b? How generic are they?
• Line 477: Regions of lower uncertainties seem to be also located at the edges and in pockets.
• Line 771: How is the fuel cost of changing altitude calculated? Is that part of the performance model?