The title of the paper is: "Copula-based synthetic data generation for machine learning emulators in weather and climate: application to a simple radiation model".  The major conclusion of the paper is that using emulated data may be of particular interest to scientists and practitioners developing ML emulators in weather and climate.  In my opinion both the title and the main conclusion are unfocused and the main conclusion is not something new. 

First of all, authors should specify how they understand "machine learning emulators in weather and climate". There is a long history of developing ML emulators for numerical weather and climate models, including emulations of sophisticated state-of-the-art radiation models: 

Chevallier, F., Morcrette, J.-J., Chéruy, F., Scott, N. A. (2000). Use of a neural-network-based longwave radiative transfer scheme in the ECMWF atmospheric model. Quarterly Journal of the Royal Meteorological Society, 126, 761-776

Krasnopolsky, V. M., Fox-Rabinovitz, M. S., Chalikov, D. V. (2005). New Approach to Calculation of Atmospheric Model Physics: Accurate and Fast Neural Network Emulation of Long Wave Radiation in a Climate Model.  Monthly Weather Review, 133, 1370-1383

Krasnopolsky, V. M., Fox-Rabinovitz, M. S., Belochitski, A. (2008a). Decadal climate simulations using accurate and fast neural network emulation of full, long- and short wave, radiation. Monthly Weather Review, 136, 3683-3695. doi:10.1175/2008MWR2385.1 

Krasnopolsky, V. M., Fox-Rabinovitz, M. S., Hou, Y. T., Lord, S. J., & Belochitski, A. A. (2010). Accurate and Fast Neural Network Emulations of Model Radiation for the NCEP Coupled Climate Forecast System: Climate Simulations and Seasonal Predictions, Monthly Weather Review, 138, 1822-1842. doi: 10.1175/2009MWR3149.1

Krasnopolsky, V. M., Fox-Rabinovitz, M. S., & Belochitski, A. A. (2013). Using Ensemble of Neural Networks to Learn Stochastic Convection Parameterization for Climate and Numerical Weather Prediction Models from Data Simulated by Cloud Resolving Model, Advances in Artificial Neural Systems, 2013, Article ID 485913, 13 pages. doi:10.1155/2013/485913

Brenowitz, N. D., & Bretherton, C. S. (2018). Prognostic Validation of a Neural Network Unified Physics Parameterization, Geophysical Research Letters, 45, 6289-6298. https://doi.org/10.1029/2018GL078510

Gentine, P., Pritchard, M., Rasp, S., Reinaudi, G., Yacalis G. (2018). Could Machine Learning Break the Convection Parameterization Deadlock? Geophysical Research Letters, 45, 5742–5751. https://doi.org/10.1029/ 2018GL078202

O’Gorman, P. A., & Dwyer, J. G. (2018). Using machine learning to parameterize moist convection: potential for modeling of climate, climate change and extreme events.  Journal of Advances in Modeling Earth Systems. https://doi.org/10.1029/2018MS001351

Pal, A., Mahajan, S., and Norman, M. R. (2019). Using deep neural networks as cost-effective surrogate models for Super-Parameterized E3SM radiative transfer. Geophysical Research Letters, 46, 6069–6079. https://doi.org/10.1029/2018GL081646

Rasp, S., Pritchard, M. S., Gentine, P. (2018). Deep learning to represent subgrid processes in

climate models, PNAS Latest Articles. https://doi.org/10.1073/pnas.1810286115

Rasp, S. (2020).  Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations: general algorithms and Lorenz 96 case study (v1.0) arXiv:1907.01351v4 [physics.ao-ph] 

Scher, S. (2018). Toward data‐driven weather and climate forecasting: Approximating a simple general circulation model with deep learning. Geophysical Research Letters, 45. https://doi.org/10.1029/2018GL080704

Scher, S., Messori G. (2019). Weather and climate forecasting with neural networks: using GCMs with different complexity as study-ground. Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2019-53

plus a quickly growing amount of publications during 2019, 2020, and even already during 2021. 

In all these works, data simulated by GCM, CRM, or LES are used.  Researchers, working in these fields of weather and climate, long ago recognized that they must use simulated data because there is no sufficient amount of observations, collocated in space and time to use for training their ML emulations.  In some cases ECMWF or NCEP analysis or reanalysis are used to integrate in an optimal way simulated and observed data. The same sources of data are used to develop forward models. 

Thus, I suggest to authors:

1. to clearly specify what subfield of "weather and climate" they target

2. to edit the title and the text correspondingly

3. to explain how their results provide researchers, working in the targeted field, with a new information 

The authors have written an interesting paper on the use of copulas for synthetic data generation for ML emulators in weather and climate applications. This is a valuable and, as far as I know, novel contribution to the ML in weather and climate modelling field. The paper is well-written and for the most part, clear and concise.

Besides some improvements that could be made with regards to literature review, I found a larger issue with regards to the methodology, which appears inconsistent in the way it is defined and implemented. The authors initially appear to use both observation-based training (OBT) and emulation-based training (EBT), but strictly speaking no OBT is done as the outputs come from a physical model (just like in the authors’ definition of EBT). Perhaps the implementation of method B (OBT with data generation) is intended only to mimic the method as it is defined, but then this should be made explicit and the validity of this approach discussed.

While some readers may find no problem with the conceptualization and presentation, since it is open to interpretation, I found it nonintuitive. That being said, this a publishable paper if the clarity is improved. I attach my specific comments as a PDF supplement.

## Comments:

- It is my impression that the nomenclature Emulator-based training does not accurately reflect the principles of what is done in the setting described by the authors. Independently of how we train our ML models (using observations or simulations), both general settings provide a surrogate (an emulator) to make predictions. I believe the term "Simulation-based training (SBT)" reflects better the principles of training a surrogate model based on simulations being done by, what the authors call, physical models. This distinction, although a bit superficial, has been carefully proposed in earlier work in general scientific endeavors. See for example: 

  - Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989). Design and analysis of computer experiments. Statistical science, 409-423. 

  which has led to a more mature framework of modeling emulators based on simulation results to predict real world processes. See: 

  - Kennedy, M. C., & O'Hagan, A. (2001). Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(3), 425-464.

- I believe that to refer to the parameters of a ML model as _weights_ lessens the generalizability of the work of the authors. This is because the term "weight" is particular to Deep Learning (DL) instances such as Artificial Neural Networks or Multilayer Linear Perceptron models and their subsequent generalizations. My issue with this is that ML is a much broader discipline than just DL. _Any_ ML used for prediction is looking for the best possible association of $X$ (features, properties, descriptors) to $Y$ (the target). In this sense, we are looking for the best candidate $h$ that can achieve $Y \approx h(X)$ in some sense (for example, as measured by Mean-Square Error). We choose parametrized models due to our ability ---mainly, through iterative optimization algorithms--- to learn such approximations. Although, for an ML researcher/practitioner this is not new. Someone with no such background would not make the immediate connection in the text with $w$ to "the best function approximation for a specific model architecture".

- This leads me to my next concern: are simulations contrasted to data? Even though, a surrogate is built upon simulation-based observations, at some point it needs to be contrasted to real world data to measure the validity of using a specific model configuration (either for the physical model or the emulator). There is work being done in this direction for climate predictions. See, for example: 

  - Cleary, E., Garbuno-Inigo, A., Lan, S., Schneider, T., & Stuart, A. M. (2020). Calibrate, emulate, sample. Journal of Computational Physics, 424, 109716. 

  as an example of a complete simulation-observation based strategy to learn surrogate models. 

- The goal of learning ML models for climate applications is a hot topic in research. You can also see: 

  - Rasp, S., Pritchard, M. S., & Gentine, P. (2018). Deep learning to represent subgrid processes in climate models. Proceedings of the National Academy of Sciences, 115(39), 9684-9689. 

  Which leads me to question what is the novelty of the proposed manuscript. 

- The stochasticity of the training of an MLP can be accounted, for example, with $K$-fold cross validation. It should also be used as it internally provides a measure of generalization error that can help guide the selection of certain hyper-parameters (for example the optimization-related parameters). Why not use such a strategy for this manuscript?

- As a final comment, it is not clear to me the setting and the intention of the strategy presented by the authors. I believe it is not clear how to interpret these results. It seems like a particular instance of a data-augmentation strategy, with the potential benefit of preserving the observed probabilistic relationships among training data. I believe the authors does not provide clear evidence that such an augmentation achieves better results when compared to real world observations.

## Minor comments:

- Typo found at line 151. It reads "...all variables a continuous..." it should be "...all variables are continuous...".