Review of “DeepPhenoMem V1.0: Deep learning modelling of canopy greenness

dynamics accounting for multi-variate meteorological memory effects on vegetation phenology” by Liu et al. for consideration in EGUsphere.

Based on in situ observation data of plant phenology at 50 sites distributed across Northern American, the authors trained a deep learning model LSTM which has the potential to capture and model the meteorological memory effects on vegetation phenology. The topic of this study is very important and interesting. It provides a new pathway to simulate and investigate the complex impacts of environmental factors on plant canopy dynamics. In addition, the manuscript is overall well organized, and the methods used in this study have been introduced detailedly.

Nonetheless, I still have some questions on the method and results of this study:

• The DeepPhenoMem model in this study is trained at 45 sites, and evaluated at 5 sites. Some random factors (e.g. the specific climate and species in the 4 validation sites compared to the training sites) might strongly affect the evaluation results. To give a more robust evaluation on the model, I would suggest the authors to do a 10-fold (or 5-fold) cross validation.
• There are 3 unseen sites for deciduous broadleaved forests were used to test the trained model in this study. Why not show the evaluation results at all of these 3 sites in Figs. 5 & 6. The readers might wonder that only the site with best model performance for PFT DB was showed in Figs. 5 & 6. The evaluation results might thus be biased.
• The deciduous broadleaved forests (DB) generally show stronger seasonal variability compared to evergreen needle-leaved (EN) forests. I am wondering why the DeepPhenoMem model performs better for EN, compared to DB (Table 1, Fig. 3). If it is only because the BD has been tested at three sites, while the EN was only tested in 1 sites? In theory, the observed and simulated GCC for DB and grassland could be low to 0. Why the GCC for DB and grassland are all higher than 0.3. Are there any evergreen plants living in the DB and grassland sites. In addition, the EN keeps being green across the year, is it accurate enough to extracted the GCC from the digital images photographed by automated and high-frequency digital cameras. Even in the end of growing season, the GCC for EN should still be high.
• Based on the observed data, the authors can actually calculate the sensitivities of SOS and EOS to warming using linear regression (e.g. Fu et al., 2015, Nature). I am wondering if the temperature sensitivities simulated from the trained DeepPhenoMem model in this study are comparable to the values calculated based on observations. I would suggest the authors to do a comparison/evaluation.

Specific comments:

L147-148: Is it accurate enough to use the sum of precipitation over the previous month as a proxy of daily SW? Is there any reference of Eq. 3. Maybe it is better to simply say the sum of precipitation over the previous month has been included in the model, without mentioning the SW.

L163-164: Why not conduct a cross-validation?

Table 1: Please add RMSE of each model for each vegetation type

Fig. 4: What does the rhombus represent here? The significance of difference between two versions of the model? If no, please provide a significance test between results from the M0 and Mfull. The sub-plots a, b, c show results for DB, EN, GR, respectively?

Figs. 5 & 6: There are 3 unseen sites for DB (Fig. 1), why only results for harvardbarn2 was presented?

L376-378:  I did not find what result in this study indicate the cumulative thermal summation, rather than daily temperature alone, determines vegetation phenology

L406-411: Not fully true. The phenology module of Earth System models (ESMs) indeed only focuses on a few specific phenological events (e.g. start and end of the growing season). However, the ESMs also simulate the whole time series of canopy development/evolution across the whole growing season, by mechanically simulating the photosynthesis, autotrophic respiration, carbon allocation, etc. To have a closed mass balance of carbon, the canopy evolution has to be simulated mechanically, rather than using an empirical model or machine learning model.

Liu et al.,

DeepPhenoMem V1.0: Deep learning modelling of canopy greenness dynamics accounting for multi-variate meteorological memory effects on vegetation phenology

General comments:

This study aims to train an LTSM model to simulate the temporal evolution of a measure of canopy greenness observed with in-situ repeat, digital photography across three plant functional types, testing performance against the final year of observations at the ~50 training sites, and across multiple years at several sites not used in training. This is an interesting proposal and, given the current limitations in long established phenology models, potentially very useful to a wide community of vegetation modelers. The primary conclusions that the LTSM model seems to capture some of the underlaying controls on phenology, that incorporating meteorological memory effects improves performance, and the models exhibits plausible relationships are sound.

However, the results as presented are difficult to interpret beyond this – the reliance on R2 statistics is potentially misleading considering the marked biases exhibited by the model.  Incorporating RMSE or some other measure of bias throughout the results and discussion is required to better understand where and when the model performs well or otherwise. Contrasting and explaining (lack of) model performance across space/time/PFT would potentially be more informative than the current approach which tends towards endorsing model performance in very vague terms. At a minimum, this would include adding RMSE values to Table 1 and including in Figure 4 and moving Figure 9 from the Discussion to the Results and maybe incorporating more discussion of Figure S2, as well as adding additional interpretation and discussion of these results related to the bias. 

Specific comments:

L126: Need some additional information that explains the limited and very similar dynamical range in GCC across the three PFTs. Why is there not more annual variation in deciduous v. evergreen trees in particular?

L153: It doesn’t seem appropriate to call weighted mean monthly precipitation “soil water” – it isn’t and should be renamed. Also, it was an unfortunate choice of variable to include when trying to tease out the difference between models with and without memory effects as by design it will capture an approximation of soil moisture memory that won’t be removed by the shuffling in the M0 model and leads to a mixture of instantaneous and time integrating variables in the regression model. If this can be included, then why not some cumulative temperature term, or day of year etc. which are known to strongly influence phenology.

L251: Figures 1, 4 and Figure S1 indicate there are three deciduous test sites? It’s unclear which (maybe all?) are being shown in Figure 3

L270: Here, and elsewhere in several places in the manuscript, reference to figure numbers is incorrect. Please check all these carefully.

L276: There is evidently zero/minimal skill in simulating daily anomalies, whilst overall R2 values indicate some skill in seasonal variability/monthly times scales. Is attempting to simulate what are presumably noisy daily data a valuable test? Can some smoothing be applied to investigate if there is any model skill between daily and monthly time scales. Or is there only really skill in seasonal variability - making additional analysis of variation of a few days in SOS and EOS difficult to interpret?

L280: Here is a clear example of the bias that needs to be quantified and examined more thoroughly throughout the whole analysis.