Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

Journal metrics

  • IF value: 5.240 IF 5.240
  • IF 5-year value: 5.768 IF 5-year
    5.768
  • CiteScore value: 8.9 CiteScore
    8.9
  • SNIP value: 1.713 SNIP 1.713
  • IPP value: 5.53 IPP 5.53
  • SJR value: 3.18 SJR 3.18
  • Scimago H <br class='hide-on-tablet hide-on-mobile'>index value: 71 Scimago H
    index 71
  • h5-index value: 51 h5-index 51
Volume 9, issue 6
Geosci. Model Dev., 9, 2099–2113, 2016
https://doi.org/10.5194/gmd-9-2099-2016
© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.

Special issue: Model infrastructure integration and interoperability (MI3)

Geosci. Model Dev., 9, 2099–2113, 2016
https://doi.org/10.5194/gmd-9-2099-2016
© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.

Development and technical paper 09 Jun 2016

Development and technical paper | 09 Jun 2016

A new adaptive data transfer library for model coupling

Cheng Zhang1,2, Li Liu1,3, Guangwen Yang2,1,3, Ruizhe Li1,2, and Bin Wang1,3,4 Cheng Zhang et al.
  • 1Ministry of Education Key Laboratory for Earth System Modeling, Center for Earth System Science (CESS), Tsinghua University, Beijing, China
  • 2Department of Computer Science and Technology, Tsinghua University, Beijing, China
  • 3Joint Center for Global Change Studies (JCGCS), Beijing, China
  • 4State Key Laboratory of Numerical Modelling for Atmospheric Sciences and Geophysical Fluid Dynamics (LASG), Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, China

Abstract. Data transfer means transferring data fields from a sender to a receiver. It is a fundamental and frequently used operation of a coupler. Most versions of state-of-the-art couplers currently use an implementation based on the point-to-point (P2P) communication of the message passing interface (MPI) (referred to as “P2P implementation” hereafter). In this paper, we reveal the drawbacks of the P2P implementation when the parallel decompositions of the sender and the receiver are different, including low communication bandwidth due to small message size, variable and high number of MPI messages, as well as network contention. To overcome these drawbacks, we propose a butterfly implementation for data transfer. Although the butterfly implementation outperforms the P2P implementation in many cases, it degrades the performance when the sender and the receiver have similar parallel decompositions or when the number of processes used for running models is small. To ensure data transfer with optimal performance, we design and implement an adaptive data transfer library that combines the advantages of both butterfly implementation and P2P implementation. As the adaptive data transfer library automatically uses the best implementation for data transfer, it outperforms the P2P implementation in many cases while it does not decrease the performance in any cases. Now, the adaptive data transfer library is open to the public and has been imported into the C-Coupler1 coupler for performance improvement of data transfer. We believe that other couplers can also benefit from this.

Publications Copernicus
Download
Short summary
We propose a butterfly implementation for data transfer. Although the butterfly implementation outperforms the existing implementation (the P2P implementation) in many cases, it degrades the performance in some cases. So we design and implement an adaptive data transfer library that automatically chooses an optimal implementation between the P2P one and the butterfly one and also further improves the performance based on the butterfly implementation through skipping some butterfly stages.
We propose a butterfly implementation for data transfer. Although the butterfly implementation...
Citation