the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 14 Nov 2024
Intercomparison of bias correction methods for precipitation of multiple GCMs across six continents
Abstract. This study, conducted across six continents, evaluated and compared the effectiveness of three Quantile Mapping (QM) methods: Quantile Delta Mapping (QDM), Empirical Quantile Mapping (EQM), and Detrended Quantile Mapping (DQM) for correcting daily precipitation data from 11 CMIP6 General Circulation Models (GCMs). The performance of corrected precipitation data was evaluated using ten evaluation metrics, and the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) was applied to calculate performance-based priorities. Bayesian Model Averaging (BMA) was used to quantify model-specific and ensemble prediction uncertainties. Subsequently, this study developed a comprehensive index by aggregating the performance scores from TOPSIS with the uncertainty metrics from BMA. The results showed that EQM performed the best on all continents, effectively managing performance and uncertainty. QDM outperformed other methods in specific regions and was selected more frequently than DQM when greater weight was given to uncertainty. It suggests that daily precipitation corrected by QDM is more stable than DQM. On the other hand, DQM effectively reproduces dry climate but shows the highest uncertainty in certain regions, suggesting potential limitations in capturing long-term climate trends. This study emphasizes that both performance and uncertainty should be considered when choosing a bias correction method to increase the reliability of climate predictions.
- Preprint
(7142 KB) - Metadata XML
-
Supplement
(3534 KB) - BibTeX
- EndNote
Status: open (until 09 Jan 2025)
-
CEC1: 'Comment on gmd-2024-177', Juan Antonio Añel, 08 Dec 2024
reply
Dear authors,
Unfortunately, after checking your manuscript, it has come to our attention that it does not comply with our "Code and Data Policy".
https://www.geoscientific-model-development.net/policies/code_and_data_policy.htmlYour "Code and Data Availability" statement does not contain the link to a permanent repository with the code and data used to produce your manuscript. I am sorry to have to be so outspoken, but this is something completely unacceptable, forbidden by our policy, and your manuscript should have never been accepted for Discussions given such flagrant violation of the policy. All the code and data must be published openly and freely to anyone in one of the repositories listed in our policy before submission of a manuscript.
Therefore, we are granting you a short time to solve this situation. You have to reply to this comment in a prompt manner with the information for the repositories containing all the models, code and data that you use to produce and replicate your manuscript. The reply must include the link and permanent identifier (e.g. DOI). Also, any future version of your manuscript must include the modified section with the new information.
Note that if you do not fix these problems as requested, we will have to reject your manuscript for publication in our journal.
Juan A. Añel
Geosci. Model Dev. Executive Editor
Citation: https://doi.org/10.5194/gmd-2024-177-CEC1 -
CC1: 'Reply on CEC1', Young Hoon Song, 08 Dec 2024
reply
Dear Chief editor
We have identified an oversight on our part regarding the absence of a DOI for data availability. Accordingly, we have revised the Code and Data Availability section as follows:
Code availability
Codes for benchmarking the xclim of python package are available from https://xclim.readthedocs.io/en/stable/. Furthermore, the Comprehensive Index proposed in this study, along with the performance and uncertainty indices used within it, is available at https://doi.org/10.6084/m9.figshare.27987665.v2 (Song, 2024).
Data availability
The data used in this study are publicly available from multiple sources. CMIP6 General Circulation Models (GCMs) outputs were obtained from the Earth System Grid Federation (ESGF) data portal at https://esgf-node.llnl.gov/search/cmip6/. Users can select data types such as climate variables, time series, and experiment ID, which can be downloaded as NC files. Furthermore, CMIP6 GCMs output can also be accessed in Eyring et al. (2016) The ERA5 reanalysis dataset used in this study is available through the Copernicus Data Store (CDS) provided by ECMWF (https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=overview). ERA5 is available at https://doi.org/10.24381/cds.bd0915c6 (Hersbach et al., 2023).
Additionally, the newly added DOI in the references is as follows:
Eyring, V., Bony, S., Meehl, G., Senior, C., Stevens, B., Stouffer, R., and Taylor, K.: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization [Dataset]. Geoscientific Model Development, 9(5), 1937–1958. 2016. https://doi.org/10.5194/gmd-9-1937-2016
Hersbach, H., Bell, B., Berrisford, P., Biavati, G., Horányi, A., Muñoz Sabater, J., Nicolas, J., Peubey, C., Radu, R., Rozum, I., Schepers, D., Simmons, A., Soci, C., Dee, D., and Thépaut, J.-N.: ERA5 hourly data on pressure levels from 1940 to present, Copernicus Climate Change Service (C3S) Climate Data Store (CDS) [Data set], https://doi.org/10.24381/cds.bd0915c6, 2023.
Song, Y.H.: Comprehensive Index and Performance-Related Code, Figshare [Code], https://doi.org/10.6084/m9.figshare.27987665.v2
Additionally, I am attaching the revised article. The modified sections are marked in red text. Thank you. 
Â
-
CEC2: 'Reply on CC1 - compliance with policy no solved', Juan Antonio Añel, 10 Dec 2024
reply
Dear authors,
I'm sorry, but your reply does not address the issues I have pointed out. You should read more carefully our policy. It clearly states which are the acceptable repositories to publish assets. The readthedocs.io is not acceptable (and moreover it only contains documentation, not the code). Also, generic data portals for reanalyses or datasets such as those from the CMIP6 do not provide the exact data that you use in your manuscript, that is what you have to publish. Also, papers could have DOIs, but they are not data or code repositories.
I ask you to read again our policy, and follow the rules listed there, and store your assets in adequate repositories to comply with our policy.
To be clear, currently, your submission continues to be non-compliant with our policy.Â
Juan A. Añel
Geosci. Model Dev. Executive Editor
Citation: https://doi.org/10.5194/gmd-2024-177-CEC2
-
CEC2: 'Reply on CC1 - compliance with policy no solved', Juan Antonio Añel, 10 Dec 2024
reply
-
RC1: 'Comment on gmd-2024-177', Anonymous Referee #1, 08 Dec 2024
reply
This study proposes a Comprehensive Index that considers both performance and uncertainty, using 11 CMIP6 GCMs and compares bias correction methods across six continents. Such an approach can be evaluated as a highly significant method for GCMs and bias correction, which are essential for future predictions. The reviewer, therefore, would like to recommend that this manuscript be returned to the authors for major revisions. My comments are given below;
The authors need to address several key questions:
- Why was only the historical period used? If there is a specific reason, it must be clearly stated.
- Developing a comprehensive index is a significant strength of this study. However, there is insufficient explanation of the differences from previous methods to highlight its originality. Please address this. Additionally, the comprehensive index appears to be overly limited. Such a restrictive methodology may reduce the efficiency of the approach, which warrants further discussion.
- DQM performs worse overall compared to other methods. The reasons for this must be explained. However, as QM methods are not newly proposed in this study but adopted from existing ones, it is unnecessary to deeply analyze the cause (this could be a topic for a future study). A brief explanation in the context of specific climate phenomena should suffice.
- The authors used entropy as a criterion weight for TOPSIS. Is there a reason for this choice? Since MCDA is widely used in other fields, incorporating more complex weights might be more appropriate.
- Model performance can vary significantly depending on the climate zone, but in most cases, the performance is reported to be very high (e.g., EVS: 0.98 (DQM)). Such results indicate exceptional performance. This should be mentioned in the Discussion section.
- Why was only ERA5 selected for comparison? It may not pose a significant issue without a valid reason, but considering multiple reference datasets is essential for evaluating performance.
- Why does DQM yield higher estimates for extreme precipitation? Could this be because DQM emphasizes variability in the data due to detrending?
This study provides valuable insights into climate change research by comparing bias correction methods and incorporating an integrated uncertainty evaluation. Therefore, sincerely addressing the above comments will significantly enhance the scientific value of this study. Additionally, the findings are anticipated to be actively applicable to new stages of GCMs, or models developed after CMIP7.
https://onlinelibrary.wiley.com/doi/10.1029/2021EF002240
https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2022EF002830
Citation: https://doi.org/10.5194/gmd-2024-177-RC1 -
CC2: 'Reply on CEC2', Young Hoon Song, 11 Dec 2024
reply
Dear Editor,
We have actively utilized Zenodo to address your feedback. For the code, we were able to provide example datasets on Zenodo due to their small size. However, the datasets used in this study, such as CMIP6 GCM and ERA5, exceed 50 GB in size. Therefore, we utilized Springer Nature's Figshare to make these datasets available. Additionally, we have updated the version of xclim we used to reflect it on Zenodo. Accordingly, we have revised the Code and Data Availability sections as follows.
Code availability
Codes for benchmarking the xclim of python package are available from https://doi.org/10.5281/zenodo.10685050 (Bourgault et al., 2024). Furthermore, the CI proposed in this study, along with the TOPSIS and BMA used within it, is available at https://doi.org/ 10.5281/zenodo.14351816 (Song, 2024b).Data availability
The data used in this study are publicly available from multiple sources. CMIP6 General Circulation Models (GCMs) outputs were obtained from the Earth System Grid Federation (ESGF) data portal at https://esgf-node.llnl.gov/search/cmip6/. Users can select data types such as climate variables, time series, and experiment ID, which can be downloaded as NC files. Furthermore, CMIP6 GCMs output can also be accessed in Eyring et al. (2016) The ERA5 reanalysis dataset used in this study is available through the Copernicus Data Store (CDS) provided by ECMWF (https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=overview). ERA5 is available at https://doi.org/10.24381/cds.bd0915c6 (Hersbach et al., 2023). The daily precipitation datasets from CMIP6 GCM and ERA5 used in this study are available at https://doi.org/10.6084/m9.figshare.27999167.v5 (Song, 2024c).ÂThe additional and revised references are as follows:
Bourgault, P., Huard, D., Smith, T.J., Logan, T., Aoun, A., Lavoie, J., Dupuis, É., Rondeau-Genesse, G., Alegre, R., Barnes, C., Beaupré Laperrière, A., Biner, S., Caron, D., Ehbrecht, C., Fyke, J., Keel, T., Labonté, M.P., Lierhammer, L., Low, J.F., Quinn, J., Roy, P., Squire, D., Stephens, Ag., Tanguy, M., Whelan, C., Braun, M., Castro, D.: xclim: xarray-based climate data analytics (0.48.1). Zenodo [Code], https://doi.org/10.5281/zenodo.10685050, 2024.
Eyring, V., Bony, S., Meehl, G., Senior, C., Stevens, B., Stouffer, R., and Taylor, K.: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geoscientific Model Development, 9(5), 1937–1958. 2016. https://doi.org/10.5194/gmd-9-1937-2016
Hersbach, H., Bell, B., Berrisford, P., Biavati, G., Horányi, A., Muñoz Sabater, J., Nicolas, J., Peubey, C., Radu, R., Rozum, I., Schepers, D., Simmons, A., Soci, C., Dee, D., and Thépaut, J.-N.: ERA5 hourly data on pressure levels from 1940 to present, Copernicus Climate Change Service (C3S) Climate Data Store (CDS), https://doi.org/10.24381/cds.bd0915c6, 2023. 
Song, Y.H.: Comprehensive Index and Performance-Related Code, Zenodo [Code], https://zenodo.org/records/14351816. 2024b
Song, Y.H.: Historical Daily Precipitation Data of CMIP6 GCMs and ERA5, Figshare [Dataset], https://doi.org/10.6084/m9.figshare.27999167.v5. 2024c
Â
-
CC3: 'Comment on gmd-2024-177', Shamsuddin Shahid, 13 Dec 2024
reply
This study presents an innovative approach by systematically comparing the performance and uncertainty of Quantile Mapping (QM) methods across six continents using 11 CMIP6 GCMs. Specifically, it makes an academically significant contribution by quantitatively evaluating the performance of three major QM methods—QDM, EQM, and DQM—and developing a Comprehensive Index (CI) that integrates performance and uncertainty using TOPSIS and Bayesian Model Averaging (BMA).
In climate change research, bias correction of GCMs is a critical step in determining the reliability of climate projections and impact assessments. While previous bias correction studies have often been limited to single regions or small datasets, this study attempts to compare bias correction methodologies on a global scale, considering continental differences and regional characteristics. Additionally, the development of CI, which integrates uncertainty quantification and performance evaluation, is expected to improve the reliability of climate models and provide substantial contributions to the formulation of adaptation and mitigation policies for climate change.
I have the following major questions:
Why did this study only use the historical period? For example, do the authors believe that bias correction performance during historical periods sufficiently accounts for uncertainty in future projections?
What are the strengths of the Comprehensive Index? The study claims to have developed this index, but could you provide a detailed explanation of its strengths and how it differs from previous methods?
Why were TOPSIS and BMA specifically used for performance and uncertainty in the Comprehensive Index? Could other methods have been equally applicable? Please discuss whether alternatives might be feasible.
Why does DQM perform worse than other methods? Could the authors explain why this method shows lower performance compared to QDM and EQM?
Why was ERA5 chosen as the reference dataset without comparing it to other reanalysis datasets (e.g., CHIRPS, GPCP)? What criteria led the authors to determine ERA5 as the most suitable for the study’s regions and precipitation characteristics?
While unifying data resolution to 1°x1° is advantageous, it might obscure detailed regional characteristics that can be captured with higher resolutions. Why was this resolution chosen, and do the authors believe that bias correction performance could differ at higher resolutions?
Entropy theory was clearly used to determine the weights, but could the authors explain the impact and significance of low weights for certain metrics (e.g., EVS, NSE)?EQM showed the lowest uncertainty. Is this result due to the characteristics of the methodology itself, or is it influenced by specific regional or data traits?
Each QM method shows strengths and weaknesses in specific regions. Was there an attempt to develop a hybrid approach that combines these methods? For instance, could DQM’s ability to remove long-term trends and EQM’s stable performance be integrated?
The authors claim that CI considers both performance and uncertainty. However, there is insufficient explanation on whether region-specific weights were applied. Was CI calculated for each grid? For example, regions with extreme precipitation distributions (e.g., the Sahara Desert) differ significantly from moderate regions (e.g., Northern Europe). If CI was calculated at a larger scale, would that be appropriate? Alternatively, is it reasonable to apply uniform weights?
Why was TOPSIS chosen over other MCDA techniques like AHP (Analytic Hierarchy Process) or VIKOR? A discussion on this decision is necessary.
Lastly, some sentences could benefit from structural refinement. To enhance the clarity and flow of the text, it would be helpful to revise the sentences throughout the manuscript.Citation: https://doi.org/10.5194/gmd-2024-177-CC3
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 157 | 43 | 10 | 210 | 10 | 2 | 1 |
- HTML: 157
- PDF: 43
- XML: 10
- Total: 210
- Supplement: 10
- BibTeX: 2
- EndNote: 1
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1