Cheng, X., Nitsche, G., and Wallace, J. M.: Robustness of Low-Frequency Circulation Patterns Derived from EOF and Rotated EOF Analyses, J. Climate, 8, 1709–1713,
https://doi.org/10.1175/1520-0442(1995)008<1709:ROLFCP>2.0.CO;2, 1995.
a
Danabasoglu, G., Lamarque, J.-F., Bacmeister, J., Bailey, D. A., DuVivier, A. K., Edwards, J., Emmons, L. K., Fasullo, J., Garcia, R., Gettelman, A., Hannay, C., Holland, M. M., Large, W. G., Lauritzen, P. H., Lawrence, D. M., Lenaerts, J. T. M., Lindsay, K., Lipscomb, W. H., Mills, M. J., Neale, R., Oleson, K. W., Otto-Bliesner, B., Phillips, A. S., Sacks, W., Tilmes, S., van Kampenhout, L., Vertenstein, M., Bertini, A., Dennis, J., Deser, C., Fischer, C., Fox-Kemper, B., Kay, J. E., Kinnison, D., Kushner, P. J., Larson, V. E., Long, M. C., Mickelson, S., Moore, J. K., Nienhouse, E., Polvani, L., Rasch, P. J., and Strand, W. G.: The Community Earth System Model Version 2 (CESM2), J. Adv. Model. Earth Sy., 12, e2019MS001916,
https://doi.org/10.1029/2019MS001916, 2020.
a
Dijkstra, E. W.: Why numbering should start at zero, University of Texas,
http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF (last access: 3 April 2025), 1982. a
Easterbrook, S., Edwards, P., Balaji, V., and Budich, R.: Guest Editors' Introduction: Climate Change – Science and Software, IEEE Software, 28, 32–35, 2011. a
ESCOMP: CAM: The Community Atmosphere Model, GitHub [code],
https://github.com/ESCOMP/CAM, last access: 3 April 2025a.
ESCOMP: CESM: The Community Earth System Model, GitHub [code],
https://github.com/ESCOMP/CESM, last access: 3 April 2025b.
Gokhale, M., Gopalakrishnan, G., Mayo, J., Nagarakatte, S., Rubio-González, C., and Siegel, S. F.: Report of the DOE/NSF Workshop on Correctness in Scientific Computing, June 2023, Orlando, FL,
https://doi.org/10.48550/arXiv.2312.15640, 2023.
a
Jackson, J. E. and Jackson, J. E.: A user's guide to principal components, Wiley series in probability and mathematical statistics, 1. print edn., Wiley, New York,
https://doi.org/10.1002/0471725331, 1991.
a,
b
Krock, M. L., Kleiber, W., Hammerling, D., and Becker, S.: Modeling Massive Highly Multivariate Nonstationary Spatial Data with the Basis Graphical Lasso, J. Comput. Graph. Stat., 32, 1472–1487,
https://doi.org/10.1080/10618600.2023.2174126, 2023.
a
Lawley, D. N.: Tests of Significance for the Latent Roots of Covariance and Correlation Matrices, Oxford University Press, Biometrika Trust, 128–136,
https://doi.org/10.2307/2333586, 1956.
a
Mahajan, S.: Ensuring statistical reproducibility of ocean model simulations in the age of hybrid computing, in: Proceedings of the Platform for Advanced Scientific Computing Conference, PASC '21, Geneva, Switzerland, 5–9 July 2021, Association for Computing Machinery, New York, NY, USA,
https://doi.org/10.1145/3468267.3470572, pp. 1–9, 2021.
a,
b
Mahajan, S., Gaddis, A. L., Evans, K. J., and Norman, M. R.: Exploring an Ensemble-Based Approach to Atmospheric Climate Modeling and Testing at Scale, Procedia Comput. Sci., 108, 735–744,
https://doi.org/10.1016/j.procs.2017.05.259, 2017.
a
Mahajan, S., Evans, K. J., Kennedy, J. H., Xu, M., and Norman, M. R.: A Multivariate Approach to Ensure Statistical Reproducibility of Climate Model Simulations, in: Proceedings of the Platform for Advanced Scientific Computing Conference, Zurich, Switzerland, 12–14 June 2019, ACM, Zurich Switzerland,
https://doi.org/10.1145/3324989.3325724, pp. 1–10, 2019a.
a
Mahajan, S., Evans, K. J., Kennedy, J. H., Xu, M., Norman, M. R., and Branstetter, M. L.: Ongoing solution reproducibility of earth system models as they progress toward exascale computing, Int. J. High Perform. C., 33, 784–790,
https://doi.org/10.1177/1094342019837341, 2019b.
a
Massonnet, F., Ménégoz, M., Acosta, M., Yepes-Arbós, X., Exarchou, E., and Doblas-Reyes, F. J.: Replicability of the EC-Earth3 Earth system model under a change in computing environment, Geosci. Model Dev., 13, 1165–1178,
https://doi.org/10.5194/gmd-13-1165-2020, 2020.
a
Milroy, D. J., Baker, A. H., Hammerling, D. M., Dennis, J. M., Mickelson, S. A., and Jessup, E. R.: Towards Characterizing the Variability of Statistically Consistent Community Earth System Model Simulations, Procedia Comput. Sci., 80, 1589–1600,
https://doi.org/10.1016/j.procs.2016.05.489, 2016.
a,
b
Milroy, D. J., Baker, A. H., Hammerling, D. M., and Jessup, E. R.: Nine time steps: ultra-fast statistical consistency testing of the Community Earth System Model (pyCECT v3.0), Geosci. Model Dev., 11, 697–711,
https://doi.org/10.5194/gmd-11-697-2018, 2018.
a,
b,
c,
d,
e,
f,
g,
h,
i
Milroy, D. J., Baker, A. H., Hammerling, D. M., Kim, Y., Jessup, E. R., and Hauser, T.: Making Root Cause Analysis Feasible for Large Code Bases: A Solution Approach for a Climate Model, in: Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing, HPDC '19, Phoenix, AZ, USA, 22–29 June 2019, Association for Computing Machinery, New York, NY, USA,
https://doi.org/10.1145/3307681.3325399, p. 73–84, 2019.
a
Molinari, S., Milroy, D., and Hammerling, D.: A Statistical Investigation of the CESM Ensemble Consistency Testing Framework – Part II, Tech. rep., NSF National Center for Atmospheric Research,
https://doi.org/10.5065/Y541-X174, 2024.
a
Molinari, S. J., Milroy, D. J., and Hammerling, D. M.: A Statistical Investigation of the CESM Ensemble Consistency Testing Framework, NCAR Technical Note (No. NCAR/TN-554+STR), National Center for Atmospheric Research,
https://doi.org/10.26024/bfdr-nz31, 2018.
a,
b,
c
MPAS-Dev: MPAS-Model: Model for Prediction Across Scales models and shared framework releases, GitHub [code],
https://github.com/MPAS-Dev/MPAS-Model, last access: 3 April 2025.
NCAR: Cheyenne Supercomputer, National Center for Atmospheric Research, https://doi.org/10.5065/D6RX99HX, 2017.
NCAR: Derecho Supercomputer, National Center for Atmospheric Research, https://doi.org/10.5065/qx9a-pg09, 2024.
North, G. R., Bell, T. L., Cahalan, R. F., and Moeng, F. J.: Sampling Errors in the Estimation of Empirical Orthogonal Functions, Mon. Weather Rev., 110, 699–706,
https://doi.org/10.1175/1520-0493(1982)110<0699:SEITEO>2.0.CO;2, 1982.
a
Pollio, V.: Vitruvius: ten books on architecture/translated by Ingrid D. Rowland; commentary and illustrations by Thomas Noble Howe, with additional commentary by: Ingrid D. Rowland and Michael J. Dewar, Cambridge University Press, ISBN 0521553644, 1999. a
Skamarock, W. C., Klemp, J. B., Duda, M. G., Fowler, L. D., Park, S.-H., and Ringler, T. D.: A Multiscale Nonhydrostatic Atmospheric Model Using Centroidal Voronoi Tesselations and C-Grid Staggering, Mon. Weather Rev., 140, 3090–3105,
https://doi.org/10.1175/MWR-D-11-00215.1, 2012.
a,
b
Wan, H., Zhang, K., Rasch, P. J., Singh, B., Chen, X., and Edwards, J.: A new and inexpensive non-bit-for-bit solution reproducibility test based on time step convergence (TSC1.0), Geosci. Model Dev., 10, 537–552,
https://doi.org/10.5194/gmd-10-537-2017, 2017.
a,
b
