Response to Reviewer 1:

The manuscript documents the updated multi-model large ensemble archive and the climate variability diagnostics package version 6 (CVDPv6). Examples of different analyses available with CVDPv6 are also provided.

Overall the paper provides a reference for two extremely valuable resources and should be published. However, the analysis highlighted does not stress the need to critically evaluate the fidelity of the modelled forced response. In particular, potential signal to noise errors in the forced response would directly impact the validity of the modelled internal variability, with major implications for any comparison to observations. The multi-model large ensemble archive is a key tool to make progress here and I recommend adding some discussion on this.

We fully agree with the Reviewer's points and will add some discussion to address these issues.

Line 19: state that it is the modelled forced response and internal variability that can be quantified - the real world could be very different.

We will change this on revision as suggested.

Line 29: evaluating whether the observations sit within the model spread is insufficient - the fidelity of the model response to external forcings (and relatedly its representation of internal variability) is crucial.

In our revision we will add a statement that the MMLEAv2 could also help to evaluate the amplitude of internal variability in models, which influences the results of the percentile analysis.

Line 74: I believe there are several other LEs in the CMIP6 archive. at least over the historical period - are there plans to include these?

We made the choice to add any LEs we could obtain the data to from CMIP6 that has historical + ssp370 or ssp585. We do not plan to add additional LEs at this point.

Line 85: including winds at different levels would be a great addition - most extremes are related to atmospheric circulation and winds are crucial for assessing the forced response and internal variability of extremes.

This would indeed be useful as would some other variables. We have chosen to make the archive public now to enable scientific research with what we have rather than spend more time adding additional variables.

Line 138: detrending does not separate external forcing and internal variability since some external forcing including aerosols, solar and volcanoes, produce variability beyond the trend.

We will clarify the word ‘detrending in the revision.

Line 143: please add 'modelled' to 'internal variability'.

We will change this on revision as suggested.

Line 151: could you provide a link to the CVDPv6 output webpage please?

We will add this on revision as suggested.

Line 152: should this say Table 3?

Yes – we will fix it thanks.

Line 153: would that be the average pattern correlation for individual members?

To be clarified on revision

Line 175: please check the link - it didn't work for me

Apologies – it works but the website went offline for a week and this likely when you tried to access it.

Figure 1: apologies I don't understand this, what does a pattern correlation mean for an index such as ENSO which is a timeseries? Also, are all panels needed or would it be better to illustrate with one or two?

This is the ENSO pattern – i.e. the spatial Pattern in the tropical Pacific.

Figure 2: same comment as Fig 1, also what are the colours?

The colours are the RMS value – we will add this to the caption.

Line 220: but presumably evaluating more members is better than a single member?

Yes – but perhaps not necessary – we will clarify what we meant by this on revision.

Figure 3: Please provide a legend for the ensemble summary panel

The colours come from the titles in the plots above – we will clarify in the caption.

Figure 4: There are far too many panels and the text is impossible to read. I suggest just showing a couple of models to illustrate the type of output. It appears that the model patterns are based on the ensemble means which should not be compared to observations - if this is not the case please clarify. The observations are the same in every row so really do not need to be repeated.

This is already a subset of models and our preference is to keep it as is.

Lines 247-267: This ignores potential impacts of forcings on the modes that could be very important. I am certain many of the authors are aware of this, indeed I believe the leadrecieved author has written about volcanic impacts on ENSO, so I am puzzled why this is ignored here.

Line 250 already addresses this point by saying that ‘We note that changes in the variability itself are not removed using this methodology, so changes in the variability itself can be assessed using this method.’

We will add due to external forcing e.g. GHG/volcanoes to the revised text.

Figure 5: This appears to illustrate the same output as Fig 4 so could be removed

Figure 5 is the PDV while Figure 4 is the NAO so they show different modes of variability.

Figure 6: I presume the dark blue curve shows the model power spectrum rather than the timeseries? Again, I think the average of ensemble members rather than the ensemble mean should be compared with observations. The caption states that observations are grey but it looks more like brown to me.

We are not sure what you mean by ‘average of ensemble members’ is this not the ensemble mean? The caption will be changed to say grey/brown.

Line 271: also worth pointing out that La Nina rainfall impacts are not well represented over up to 40% of the globe. Is there a way to highlight, perhaps with stippling, where the observations lie outside the range of ensemble members?

This is already shown in the ranks on the RHS.

Figure 8: GISS-E2-G,CESM2 and UKESM1-0-LL are not shown

This is because they are not in set2 models (see caption).

Lines 278-293: Highlighting model differences is very important, but I suggest also highlighting the need to address these differences perhaps through emergent constraints. 

We will add a sentence on this point in revision.

Figure 9: the period in the caption (1950-1958) is not consistent with the plots (1950-1959). Again, I suggest showing fewer models to illustrate the output more clearly.

As previously we would like to keep all models here, caption will be fixed.

Line 296: Can CVDPv6 also compare different scenarios?

Yes, the CVDP can compare different scenarios (across different times) in the same comparison. The CVDP can read in data from any simulation if the data follows CMIP's file convention format.

Response to Reviewer 2:

General comments

The authors present a valuable contribution to the field of climate model evaluation, the Climate Variability Diagnostics Package (CVDP), which is developed for collective evaluation of simulated climate variability modes. The manuscript is well-structured, and the availability of CVDP results for the Multi-Model Large Ensemble Archive (MMLEA) alongside the tool itself represents an important resource for the community. I appreciate the authors' efforts in developing CVDPv6 and see its value. However, it would be helpful to clarify what is novel beyond the integration of CVDP and CVDP-LE. Does CVDPv6 introduce any new scientific advancements or methodological improvements? Highlighting these aspects in a more clear way would further strengthen the manuscript.

Thank you for your positive feedback. There are indeed methodological improvements such as the inclusion of multiple detrending methods in the CVDPv6. We will add text to highlight this on revision.

Specific comments

Section 3.1: It would be helpful to include some technical details about the CVDP, such as the programming language it is implemented in and its compatibility with different operating systems.

The CVDP is written in NCL (The NCAR Command Language). NCL can be installed on most commonly used operating systems. We hope to have a python version of the CVDP available within the next year.

Section 3.3: While I was able to access the CVDPv6 output for MMLEAv2 at https://webext.cgd.ucar.edu/Multi-Case/MMLEA_v2/ as indicated in the section, the current hosting approach—being a directory on a web server—raises concerns about long-term stability and version control. Compared to more permanent data archiving solutions such as DOI-based repositories (e.g., Zenodo), this method may not ensure robust traceability. Are there any plans to improve the long-term accessibility and versioning of these data products?

This is a good point. CVDP output (available from the CVDP Data Repository) has been available from NCAR's website for 10+ years now, and up to this point no one has questioned whether this is the best way to go about distributing it. Currently we do not offer versioning of the output data. We will consider both versioning and increasing accessibility in the future.

It would be beneficial to discuss potential linkages and/or interaction plans with other established evaluation tools, such as the Earth System Model Evaluation Tool (ESMValTool; Eyring et al., 2016), PCMDI Metrics Package (PMP; Lee et al., 2024), and Climate Model Assessment Tool (CMAT; Fasullo, 2020). In my knowledge, CVDP contributes to ESMValTool, and similar capability is also available in the PMP that can help (or already helped?) cross validation of the tools. Addressing these (potential) connections recievedwould provide readers with a comprehensive understanding of CVDP's role within the broader ecosystem of climate model analysis and evaluation tools.

An earlier version of the CVDP can indeed be run through ESMValTool. The CVDP can also be run from CESM's AMWG Diagnostics Framework (ADF) package and soon from within the  CESM Unified Postprocessing and Diagnostics (CUPiD) package. Amongst many other uses, the CVDP is used alongside the CMAT package to evaluate CESM within the model's development cycle.

Figures:Some multi-panel figures appear overly complex and difficult to read due to their small size. Is there a way to simplify these figures while maintaining clarity? For instance:

• In Figure 7, the middle column seems to repeat the same plot (ERA5_1) across multiple rows.

• In Figure 8, the second column (MEM) appears to contain redundant plots.

• In Figure 9, the third column (MEM Change) also seems repetitive.

• Figures 4 and 5 may similarly benefit from reorganization.

The authors could consider reducing the number of plots by eliminating redundancies and restructuring the panel layout to improve readability.

We understand the redundancy but want to show what is in the CVDP and how it plots so we prefer to leave these as they are.

Response to Editor comments below:

1. Climate model code would need to be provided. This has now been further clarified and this is not required.

Thank you for clarifying this point.

2. The embargo on MMLEAv2 data. The confusion has come about due to having the full MMLEAv2 dataset that is not yet hosted (you have embargoed it) and the additional data which has been provided in the zenodo repo (at my request to comply with GMD's policy) in order to reproduce the results on the manuscript. I believe there was confusion about what was in the zenodo repo and what is pending. I would like to suggest that you publish your data and update the data availability to reflect this. Could you also comment on if any permanent data archiving is planned or any way of version control or data DOIs may be managed?

We apologise for the delay. The data has been submitted for permanent publication (which can include a DOI) we were advised it would be online in early January, but there have been substantial delays. We will ensure it is published and fully accessible before we submit the revision.

3. The reference/sources to model data are not yet complete. The manuscript needs to provide sufficient information so that someone reading your paper could reproduce your dataset and results. Are their persistent DOI's, data citations or similar you could include? You mention data from two colleagues, could you expand on these runs please? How would someone get the data, is it available, would someone need to contact the author etc (anything relevant really)? Please note that the following link does not work https://esgf-data.dkrz.de/projects/mpi-ge

We have provided citations to the model description papers in the table and the links to where we originally obtained the data in text to make sure the original data can be found. Only one dataset is only available on request (via email; GFDL-ESM2M), although we note a reference paper for this dataset is available. All other model data is now published. We have followed up with the model developers regarding the broken links – and are waiting for a response. To this point in the revision we will add source code links and persistent URLs for source datasets that theMMLEAv2 is based on, where they are available. We note that DOIs do not exist for all datasets, but will add them wherever possible. We additionally note that the ETH cmip6-ng dataset is based on data from ESGF and then processed following Brunner et al. 2020 as part of the ETH cmip6-ng effort; we will make this clear that the original data source is CMIP6 ESGF in the revision.

Example DOI to be added (for model CESM2):

Danabasoglu, Gokhan(2019).NCAR CESM2 model output prepared for CMIP6 CMIP.Version YYYYMMDD[1].Earth System Grid Federation.https://doi.org/10.22033/ESGF/CMIP6.2185

We thank the editor for their time and comments which are addressed below and have been added to the revised manuscript, and updated response to review and track changes. Line numbers in this document refer to the track changed version of the manuscript.

Dear Dr Maher and co-authors,

Thank you very much for your revised manuscript. Well done on getting the data released.

I have reviewed the manuscript changes and the responses from the reviewers. The majority of the responses and changes are satisfactorily completed. At this point, the paper needs minor revisions before I send the paper back to the reviewers.

In response to reviewer 1:

The reviewer felt the areas of the modelled forced response, internal variability and signal to noise had not been addressed sufficiently. While I can see you added text to the introduction on this area, I feel a broader discussion of these areas in the introduction is needed.

The reviewers comment "Line 29: evaluating whether the observations sit within the model spread is insufficient - the fidelity of the model response to external forcings (and relatedly its representation of internal variability) is crucial. " has not yet been address sufficiently. Please review their comment and revise.

We have updated the text in response to your two comments above as follows on line 36:

‘While evaluating whether the observations sit within the model spread is crucial for model evaluation as it takes into account the fact that observations are only one possible realisation of the Earth system, understanding whether a model’s forced response and internal variability are similar to observations is also vital. Quantifying the forced response and internal variability in observations is non-trivial and an active area of research. Ongoing efforts such as the “Forced Component Estimation Statistical Method Intercomparison Project” (ForceSMIP), are expected to provide guidance on how best to extract forced response and internal variability from the observational record. LEs are critical to such projects. Quantifying these metrics in LEs themselves is more straightforward, but can be limited by ensemble size. The ensemble size needed to quantify any given metric is found to be dependent on the metric itself and the acceptable error on the quantification (Milinski et al., 2020; Lee et al., 2021; Planton et al., 2024). Progress has been made in the assessment of these quantities in combination in comparison to observations with studies such as Suarez-Gutierrez et al. (2021) using LEs and a rank-histogram method to evaluate whether LEs have a realistic forced response and internal variability of annual temperatures compared to the observed record. Such evaluations are becoming increasingly possible with the lengthening observational record and increasing forced trends (Simpson et al.,2025). Additionally, further understanding when the forced response (signal) is likely to emerge from the internal variability (noise) is vital to understanding the climate we observe (e.g. Hawkins and Sutton, 2012). This concept is important for climate communication and quantifying when we expect to see climate change signals appear in our observed record. This is another avenue where LEs are vital tools to further research in the area (e.g. Schlunegger et al., 2020).’

The reviewer suggested winds would be helpful in the archive. I am satisfied with your comment about why you are not in a position to expand the archive at this point, but do you think it would be worth adding to the discussion what other vars are of interest and why? Major ones only, I am not suggesting you include a long list. This could be used by others to understand what variables are of interest in the community. This is an optional change, and I am happy for you to decide if you feel it helpful to include.

We have added the following on line 120 to highlight which variables we plan to add if it becomes possible “If possible, other variables that could be added to the archive in the future are winds at multiple pressure levels in the atmosphere, surface and top-of-atmosphere fluxes, and thermocline depth.“

Fig 3: Your change "(colour codes by the colour of each model’s title in the previous panels)" is not clear. Please rephrase. Are the colours related to Fig 1 or different.

The caption has been updated to read ‘each model's line colour corresponds to the colour of each model's title in the previous panels of this figure’.

In response to reviewer 2:

On the reviewers comment about the ESMValTool and others, I think this should be mentioned in the introduction. Not all of these tools will be well known, and it help put into context what tools are available and what is new. I think your paper would benefit from a sentence or two commenting on this. This is an optional change. You decide.

This information has been added on line 86 as follows: ‘ We note that multiple packages exist that provide ways to investigate climate model output. Examples include: Earth System Model Evaluation Tool (ESMValTool; Eyring et al., 2016), PCMDI Metrics Package (PMP; Lee et al., 2024), and Climate Model Assessment Tool (CMAT; Fasullo, 2020). An earlier version of the CVDP can be run through ESMValTool and the CVDP can also be run from CESM’s AMWG Diagnostics Framework (ADF) package and is planned to be run from within the CESM Unified Postprocessing and Diagnostics (CUPiD) package. Amongst many other uses, the CVDP is used alongside the CMAT package to evaluate CESM within the model’s development cycle.’

Combined concerns about the plots:

Both reviewers raised concerns about the plots and I feel similarly. I understand your desire to keep the paper plots in line with the code output, in this instance I agree with that sentiment. But both reviewers has said they are hard to read and there is too much information on each plot. As you progress the code, and perhaps in the stage where you update to python, could I suggest you take the reviewers comments into consideration and get broad feedback on the images generated.

Thank you for this comment – this can be taken into consideration for future versions of the package.

Can you I suggest you edit or remove "The MMLEAv2 Archive (this published archive; xx add here)" if this was missed in copy editing it would be disappointing.

Apologies we missed this in the revision and have updated it now.

The links for the ESGF metagrid did not work as expected. Can I suggest you include 1 reference to the ESGF grid and then include "search []" or similar to indicate how to find it. 

This has been updated as suggested.

Also https://portal.nersc.gov/archive/home/c/ccsm/www/E3SMv2/FV1/atm/proc/tseries/month_1) did not work

Apologies the ) stopped his link from working – we have removed it.

Please consider these point, resubmit your manuscript, and I will send it to the reviewers for another round of review.

This has been done – we have updated the response to review and submitted material to reflect the above changes.

Many thanks for your submission and I look forward to getting your next revision,

Penny