SICNet<sub>season</sub> V1.0: a transformer-based deep learning model for seasonal Arctic sea ice prediction by integrating sea ice thickness data

Ren, Yibin; Li, Xiaofeng; Wang, Yunhe

doi:https://doi.org/10.5194/gmd-2024-200

Preprints

https://doi.org/10.5194/gmd-2024-200

Preprints

Submitted as: development and technical paper

04 Dec 2024

Submitted as: development and technical paper |

| 04 Dec 2024

Status: this preprint is currently under review for the journal GMD.

SICNet_season V1.0: a transformer-based deep learning model for seasonal Arctic sea ice prediction by integrating sea ice thickness data

Yibin Ren, Xiaofeng Li, and Yunhe Wang

Abstract. The Arctic sea ice is suffering dramatic retreating in summer and fall, which has far-reaching consequences on the global climate and commercial activities. Accurate seasonal sea ice predictions are significant in inferring climate change and planning commercial activities. However, seasonally predicting the summer sea ice encounters a significant obstacle known as the spring predictability barrier (SPB): predictions made later than May demonstrate good skill in predicting summer sea ice, while predictions made on or earlier than May exhibit considerably lower skill. This study develops a transformer-based deep-learning model, SICNet_season(V1.0), to predict the Arctic sea ice concentration on a seasonal scale. Including spring sea ice thickness (SIT) data in the model significantly improves the prediction skill at the SPB point. A 20-year (2000–2019) testing demonstrates that the detrended anomaly correlation coefficient (ACC) of Sep. sea ice extent (sea ice concentration > 15 %) predicted by our model at May/Apr. is improved by 7.7 %/10.61 % over the ACC predicted by the state-of-the-art dynamic model from the European Centre for Medium-Range Weather Forecasts (ECMWF). Compared with the anomaly persistence benchmark, the mentioned improvement is 41.02 %/36.33 %. Our deep learning model significantly reduces prediction errors of Sep.'s sea ice concentration on seasonal scales compared to ECMWF and Persistence. The spring SIT data is key in optimizing the SPB, contributing to a more than 20 % ACC enhancement in Sep.'s SIE at four to five months lead predictions. Our model achieves good generalization in predicting the Sep. SIE of 2020–2023.

Received: 25 Oct 2024 – Discussion started: 04 Dec 2024

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Yibin Ren, Xiaofeng Li, and Yunhe Wang

Status: open (until 15 Feb 2025)

Post a comment Subscribe to comment alert

CEC1:
'Comment on gmd-2024-200', Juan Antonio Añel, 27 Dec 2024 reply

Dear authors,

Unfortunately, after checking your manuscript, it has come to our attention some lack of compliance with our "Code and Data Policy".

https://www.geoscientific-model-development.net/policies/code_and_data_policy.html
To assure the replicability of your work it is necessary to have all the input and output data that you use and produce, respectively. You have provided a repository hosted by Zenodo for part of the input data and the code. However, in the repository it reads "Because the well-trained model is too large (10GB), it cannot be supported by the github". I understand that this is a sentence that comes from a previously existing GitHub site where you hosted these assets. However, it is clear that does not correspond to the current Zenodo repository. Moreover, Zenodo does not have problems with hosting 10 GB of data, so you must include in it all the input data that you have used. Also, in your "Code and Data Availability" section you link several webpages that provide access to the data that you use for your work. These are not valid and should be replaced by the repositories containing the exact input data. The sites that you list are only main "portals" to data, not the repositories for the exact input data that you use, and what you have to provide is the exact input data used to produce your work. Moreover, such sites are not valid repositories for long-term archival or scientific publication.
Additionally, you do not provide the output data produced by your model, and you must do it.
Therefore, you must reply to this comment as soon as possible with the relevant information (link and DOI) for the new repositories, as we request that you make it already available before submission and, of course, before the Discussions stage. Moreover, you must include in a potentially revised version of your manuscript the modified 'Code and Data Availability' section, with the DOI of the code (and another DOI for the dataset if necessary).
Note that if you do not fix these problems as requested, we will have to reject your manuscript for publication in our journal. In the meantime, I advice the Topical Editor to stall the peer-review process until these issues are solved, as this manuscript should not have been accepted in Discussion given the lack of compliance with the policy of the journal.
Juan A. Añel
Geosci. Model Dev. Executive Editor

Reply

Citation: https://doi.org/10.5194/gmd-2024-200-CEC1
- AC1:
  'Reply on CEC1', Yibin Ren, 27 Dec 2024 reply
  Thanks for the comment.
  We have uploaded the exact input data, output data, and the well-trained model weights to the new Zenodo repository with a DOI: https://doi.org/10.5281/zenodo.14561423. We have added detail introductions for the upload files to make sure the model can be run:
  The codes of the model are in the model_utils folders.
  
  The jupyter file "SICNet_season_training.ipynb" contains the flow of loading data, defining a model, and training the model.
  
  The jupyter file "SICNet_season_evaluating.ipynb" includes evaluating the trained model and drawing the ACC/BACC result.
  
  The training and testing data are in the Data/Data.zip file and can be loaded for training and evaluation of the model. The files "SIC_1979_2020_Jan_May_96.npy", "SIT_1979_2020_Jan_May_96.npy", "SIC_climate_mean_96.npy", and "mask_96.npy" are input data for model training. The "Output data" file contains the predicted results output by SICNetseason model, "Predicted_SIC_output.npy". The "Ground_SIC_output.npy" in this file is the ground truth from NSIDC.
  
  The Model_SICNet.zip contains the saved well-trained models. Please unpackage the zip file and copy the .h5 files to the corresponding file path "/Result/Model_SICNet". Then, the "SICNet_season_evaluating.ipynb" can be run to genernate the predicted values by the well-trained models.
  
  We also revised the “Code and data availability” session as follow:
  The satellite-observed sea ice concentration is obtained from the following site https://nsidc.org/data/NSIDC-0081/versions/1 (Cavalieri et al., 1996). The reanalysis of sea ice thickness data is open access (http://psc.apl.uw.edu/research/projects/arctic-sea-ice-volume-anomaly/data/model_grid) (Zhang and Rothrock, 2003). The seasonal predictions of SEAS5 are obtained from the ECMWF (https://cds.climate.copernicus.eu/cdsapp#!/dataset/seasonal-monthly-single-levels?tab=form) (Johnson et al., 2019). The code, the exact input/output data, and the saved well-trained weights of the developed model SICNetseason are available at https://doi.org/10.5281/zenodo.14561423.
  We upload the revised manuscript with the new “Code and data availability” section as supplement.
  
  Reply
  
  Citation: https://doi.org/10.5194/gmd-2024-200-AC1
  - CEC2: 'Reply on AC1', Juan Antonio Añel, 28 Dec 2024 reply
    
    Dear authors,
    Thanks for replying so quickly to my previous comment. However, I do not understand why you continue including in the Code and Data Availability section links to the sites that do not comply with the policy. It is my understanding that all the input data is in the Zenodo repository, and therefore, linking other sites is unnecessary and only generates doubts among readers about where to find the data to replicate your work. Therefore, the section should read simply "The code, the exact input/output data, and the saved well-trained weights of the developed model SICNetseason are available at https://doi.org/10.5281/zenodo.14561423."
    If my interpretation is wrong, and it is necessary to use data that is not included in the Zenodo repository, and that is hosted in the other sites, then your manuscript is not in compliance with the policy of the journal, because such sites are not accepted as suitable repositories for scientific publication.
    Please, clarify the situation regarding the input data, and delete the mentions to the sites that do not comply with the policy.
    Juan A. Añel
    Geosci. Model Dev. Executive Editor
    
    Reply
    
    Citation: https://doi.org/10.5194/gmd-2024-200-CEC2
    
    AC2: 'Reply on CEC2', Yibin Ren, 28 Dec 2024 reply
    
    Thanks for the comment. Sorry for my misunderstanding of the last comment. We have deleted other sites and revised the "Code and data availability" section as:
    The code, the exact input/output data, and the saved well-trained weights of the developed model SICNet_season are available at https://doi.org/10.5281/zenodo.14561423.
    The new revised manuscript is attached with the supplement.
    
    Reply
    
    Citation: https://doi.org/10.5194/gmd-2024-200-AC2
    
    CEC3: 'Reply on AC2', Juan Antonio Añel, 28 Dec 2024 reply
    
    Dear authors,
    Many thanks for the quick reply again. We can consider now the current version of your manuscript in compliance with our code and data policy, and therefore the Discussions stage and the peer-review process can continue.
    Juan A. Añel
    Geosci. Model Dev. Executive Editor
    
    Reply
    
    Citation: https://doi.org/10.5194/gmd-2024-200-CEC3
RC1:
'Comment on gmd-2024-200', Anonymous Referee #1, 09 Jan 2025 reply

For comments - see pdf attached

Reply

Citation: https://doi.org/10.5194/gmd-2024-200-RC1
- AC3: 'Reply on RC1', Yibin Ren, 22 Jan 2025 reply
  
  General comments:
  In summary, this research aims to develop a transformer-based U-net model to improve prediction
  skill of Arctic sea ice concentration at seasonal lead times. The primary novel contribution of this
  work is the use of a state-of-the-art deep learning architecture with a custom domain-specific loss
  function as well as including spring sea ice thickness as an input predictor to overcome the common
  spring predictability barrier (SPB) faced by previous numerical and DL-based models that decrease
  prediction skill of predictions made before May.
  The authors choose to use a transformer based architecture to capture spatiotemporal patterns in the
  input data. The model inputs include multiple sea-ice related variables over 3-6 months and outputs
  the sea ice concentration of the next six months, specifically looking at predictions for June-Sept at
  6 month lead times. The authors develop a novel loss function that combines both a standard DL
  loss function (mean squared error) and a domain-specific NIEE loss which accounts for spatial
  similarity in model predictions and ground truth. The model follows a standard DL training
  procedure and includes three years of unseen data to test their model performance. The authors
  evaluate their model using standard DL performance metrics along with Binary Accuracy (BACC)
  which accounts for accurate spatial distributions.
  The authors succeed in improving prediction skill when compared to the benchmark Persistence
  model and dynamical ECMWF model. For Sept ice prediction specifically, the ECMWF model
  performs better at smaller lead times but the author’s model shows marked performance
  improvement at a lead time of 3-6 months, thus overcoming the spring predictability barrier. The
  authors perform necessary ablation tests to investigate the role of sea ice thickness in their model.
  They show a model that includes sea ice thickness as input outperforms a model without sea ice
  thickness at seasonal lead times of 3-6 months. When testing their model on unseen data, the authors
  use an ensemble model approach by averaging the results of 20 separately trained models. The
  authors also test their model against a previous state of the art deep learning CNN based approach,
  IceNet. They show their model produces better ACC scores at longer lead times. Their model
  produced lower BACC scores but can capture more individualized/local or extreme characteristics
  in comparison to IceNet which tends to produce smoothed out results.
  Overall, this manuscript provides a significant improvement to the modeling of sea ice concentration
  by providing a novel approach to increase prediction skill at seasonal lead times to overcome the
  SPB. The authors clearly state their motivations for the project and outline their novel contributions.
  The authors discuss previous modeling approaches in the field and clearly showcase their significant
  scientific results where their model outperforms previous approaches. The contributions of this
  paper is twofold, one in utilizing a novel transformer based approach that the authors state has not
  been previously used in this field and second in highlighting the impact of spring sea ice thickness
  on improving prediction accuracies, revealing potential scientific insights that need to be studied
  further. Below are a few suggestions to improve reproducibility and presentation quality.
  Response: Thanks for the comment.Specific comments:
  Comment 1: In line 52, the authors state that experiments show the reason for decrease in
  prediction skill before spring is due to ice motion and growth in the winter. It is unclear which
  experiments in the literature the authors are referring to. Including a citation here would help
  justify this statement.
  Response: Thanks for the comment. We cite the following reference in this sentence:
  Bushuk, M., Winton, M., Bonan, D. B., Blanchard-Wrigglesworth, E., and Delworth, T. L.: A
  mechanism for the Arctic sea ice spring predictability barrier, Geophys Res Lett, 47,
  https://doi.org/10.1029/2020GL088335, 2020.
  Comment 2: In line 79, a citation for the IceNet model (Andersson et al 2021) is missing. Since
  working with the IceNet model is a significant portion of the manuscript, perhaps including
  more information about the IceNet model and how it differentiates from the author’s
  transformer based model would provide more context to the reader in this section.
  Response: Thanks for the comment. We added the citation for IceNet in line 79. A brief introduction
  about IceNet has also been added. The revised sentences are as follows:
  Finally, we compare our SICNetseason model with the published deep learning model IceNet
  (Andersson et al., 2021). IceNet is a probability prediction model for Arctic SIE based on
  convolutional neural network (CNN) units and the U-Net architecture. It achieved state-of-the-art
  performance in predicting the SIE for six months (Andersson et al., 2021). Therefore, we chose
  IceNet as a comparison model.
  Besides, more information about the IceNet model, such as the model’s inputs and outputs, was
  presented in Section 4.7, lines 263-267:
  The IceNet is a seasonal sea ice prediction model that performs state-of-the-art SIE prediction
  (Andersson et al., 2021). It is a CNN-based classification model, and it outputs the probability of
  three classes: open water (SIC≤15%), marginal ice (15% < SIC < 80%), and full ice (SIC≥80%).
  Differently, our SICNetseason outputs the 0-100% range SIC values. The IceNet's inputs consist of 50
  monthly mean variables, including SIC, 11 climate variables, statistical SIC forecasts, and metadata.
  Comment 3: In lines 108-109, the authors state that multiple experiments were conducted to
  determine the length of the model input features. It is unclear what experiments or methods the
  authors tried. Was this determined by manual tuning using their cross-validation strategy or did the
  authors employ an automated grid-search type strategy. The authors can also include if domain
  knowledge or previous literature informed the choice of input lengths of the data.
  Response: Thanks for the comment. We determine the length of input factors by combining domain
  knowledge and manual tuning experiments. The main domain knowledge we considered is the sea
  ice reemergence mechanism. The spring-fall reemergence occurs between pairs of months where
  the ice edge is in the same position, such as in May and December (Blanchard-Wrigglesworth et al.,
  2011; Day et al., 2014). The spring sea ice anomaly is positively correlated with fall sea ice
  anomalies, and there is also a weaker reemergence between fall sea ice anomalies and anomalies
  the following spring (Bushuk et al., 2015). Therefore, we set the initial input length of the
  SIC/SIT/SIC anomaly as six months. Then, we change the length manually to fine-tune the deep learning model to find the best-matched length for each factor. The SIC climatology of the target
  months provides an essential mean state of the prediction SIC, so we input it into the model. It can
  also represent the month number signal that IceNet has considered. We explain more details about
  this issue in the revised manuscript, and the new revision is as follows:
  The input for SICNetseason is a 96×96×18 SIC and SIT sequence, composed of SIT of the last three
  months, SIC of the last six months, SIC anomaly of the last three months, and SIC climatology of
  the six target months (Fig. 1a). We determine the length of input factors by combining domain
  knowledge and manual tuning experiments. The primary domain knowledge we considered is the
  spring-fall reemergence mechanism. It occurs between pairs of months where the ice edge is in the
  same position, such as in May and December (Blanchard-Wrigglesworth et al., 2011; Day et al.,
  2014). The spring sea ice anomaly is positively correlated with fall sea ice anomalies, and there is
  also a weaker reemergence between fall sea ice anomalies and anomalies the following spring
  (Bushuk et al., 2015). Therefore, we set the initial input length of the SIC/SIT/SIC anomaly as six
  months. We change the input length manually (from six to one in step one) to fine-tune the deep
  learning model to find the best-matched length for each factor. The SIC climatology of the target
  months provides an essential mean state of the prediction SIC. It represents the monthly cycle signal
  that IceNet has considered.
  [1] Blanchard-Wrigglesworth, E., Armour, K. C., Bitz, C. M., and Deweaver, E.: Persistence and
  inherent predictability of arctic sea ice in a GCM ensemble and observations, J Clim, 24, 231–250,
  https://doi.org/10.1175/2010JCLI3775.1, 2011.
  [2] Day, J. J., Tietsche, S., and Hawkins, E.: Pan-arctic and regional sea ice predictability:
  Initialization month dependence, J Clim, 27, 4371–4390, https://doi.org/10.1175/JCLI-D-13-
  00614.1, 2014.
  [3] Bushuk, M., Giannakis, D., and Majda, A. J.: Arctic Sea Ice Reemergence: The Role of Large
  Scale Oceanic and Atmospheric Variability*, https://doi.org/10.1175/JCLI-D-14-00354.s1, 2015.
  Comment 4: In section 4.7, when comparing the transformer based model to the CNN-based
  IceNet model, the authors state they used identical training and testing settings to perform fair
  comparisons. It is unclear whether the authors used the same 20 trained ensemble model
  approach they had used for their transformer model for the IceNet model. If so, specifying
  whether they used an ensemble approach or singular-model approach for IceNet would clarify
  this for the reader. If the authors did not use a similar ensemble approach, the authors should
  justify this choice.
  Response: Thanks for the comment. We are sorry for the confusion. We did not use an ensemble
  approach for both SICNetseason and IceNet. The training procedure is a leave-one-year-out strategy
  for the 20 testing years (2000-2019). For example, if the testing year is 2019, the training set is data
  from 1979-2018, and the testing data is 2019. Then, the testing year moves to 2018, and the
  corresponding training set is data from 1979-2017 and 2019. The model is trained 3 times for each
  training-testing pair to eliminate randomness, and the prediction for each testing year is the mean
  value of the three trained models. We explain the leave-one-year-out training procedure in Section
  4.1. Further, we clarify the training strategy in the revision:
  The training and testing settings of IceNet are the same as those of SICNetseason. The IceNet is trained
  using the same leave-one-year-out strategy as the SICNetseason. For example, if the testing year is 2019, the training set is data from 1979-2018, and the testing data is 2019. Then, the testing data
  moves to 2018; the remaining data (1979-2017, 2019) is the training set. For each training/testing
  pair, the model is trained three times to eliminate randomness, and the final prediction for testing
  data is the mean value of the three models.
  Comment 5: In line 269, it is unclear how the authors transformed the IceNet output to match
  the continuous scale (for e.g restructuring only the final output layer). Having this additional
  context would help with reproducibility of this experiment.
  Response: Thanks for the comment. We reconstruct IceNet's output layer by replacing the softmax
  with the sigmoid activation function. The sigmoid function outputs continuous values of 0-100%.
  We clarify this point in the revision:
  We reconstruct IceNet’s output layer by replacing the original softmax with the sigmoid activation
  function. The sigmoid function outputs continuous values of 0-100%.
  Comment 6: For Figures 2, 4, 6 and 8, it is slightly confusing to the reader at first how to
  interpret these results, specifically the Difference column. Perhaps including the caption that
  red signifies improvement in accuracy and blue signifies a decrease would aid in understanding
  especially because in Figures 3 and 5 the opposite color scheme is used (red = high error, blue
  – lower error).
  Response: Thanks for the comment. We have added the following statement to the captions of
  Figures 2, 4, 6, and 8: The red signifies a high/improvement in ACC/BACC, and the blue signifies
  a decrease.
  
  Reply
  
  Citation: https://doi.org/10.5194/gmd-2024-200-AC3

Yibin Ren, Xiaofeng Li, and Yunhe Wang

Viewed

Total article views: 266 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
211	45	10	266	1	2

HTML: 211
PDF: 45
XML: 10
Total: 266
BibTeX: 1
EndNote: 2

Views and downloads (calculated since 04 Dec 2024)

Month	HTML	PDF	XML	Total
Dec 2024	160	30	9	199
Jan 2025	51	15	1	67

Cumulative views and downloads (calculated since 04 Dec 2024)

Month	HTML	PDF	XML	Total
Dec 2024	160	30	9	199
Jan 2025	51	15	1	67

Viewed (geographical distribution)

Total article views: 257 (including HTML, PDF, and XML) Thereof 257 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 22 Jan 2025

Short summary

This study developed a deep learning model to predict the Arctic sea ice seasonally. By integrating the sea ice thickness data into the model, the spring prediction barrier of Arctic sea ice is optimized significantly. The model achieves better prediction skills than the typical numerical model in predicting September’s sea ice concentration seasonally. The sea ice thickness data plays a key role in reducing the prediction errors of the Beaufort Sea, the East Siberian Sea, and the Laptev Sea.


Total:	0
HTML:	0
PDF:	0
XML:	0

SICNetseason V1.0: a transformer-based deep learning model for seasonal Arctic sea ice prediction by integrating sea ice thickness data

Viewed

Viewed (geographical distribution)

SICNet_season V1.0: a transformer-based deep learning model for seasonal Arctic sea ice prediction by integrating sea ice thickness data