Developing a common, flexible and efficient framework for weakly coupled ensemble data assimilation based on C-Coupler2.0

Sun, Chao; Liu, Li; Li, Ruizhe; Yu, Xinzhu; Yu, Hao; Zhao, Biao; Wang, Guansuo; Liu, Juanjuan; Qiao, Fangli; Wang, Bin

doi:https://doi.org/10.5194/gmd-14-2635-2021

Articles | Volume 14, issue 5

https://doi.org/10.5194/gmd-14-2635-2021

© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

Special issue:

Model infrastructure integration and interoperability (MI3)

https://doi.org/10.5194/gmd-14-2635-2021

© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 14, issue 5

Development and technical paper

|

12 May 2021

Development and technical paper |

| 12 May 2021

Developing a common, flexible and efficient framework for weakly coupled ensemble data assimilation based on C-Coupler2.0

Chao Sun, Li Liu, Ruizhe Li, Xinzhu Yu, Hao Yu, Biao Zhao, Guansuo Wang, Juanjuan Liu, Fangli Qiao, and Bin Wang

Download

Final revised paper (published on 12 May 2021)
Preprint (discussion started on 11 May 2020)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

SC1: 'Comment regarding requirements of PDAF', Lars Nerger, 29 May 2020
- SC2: 'First reply to Dr. Lars Nerger', Li Liu, 30 May 2020
  - SC3: 'Some more clarifications on PDAF', Lars Nerger, 15 Jun 2020
    - SC4: 'Second reply to Dr. Lars Nerger', Li Liu, 18 Jun 2020
      - SC5: 'One ease of compilation, process setup and distribution of the analysis step', Lars Nerger, 29 Jun 2020
        
        SC6: 'Third reply to Dr. Lars Nerger', Li Liu, 30 Jun 2020
RC1: 'Referee comment', Anonymous Referee #1, 29 Jun 2020
- SC7: 'First reply to Reviewer 1', Li Liu, 02 Jul 2020
RC2: 'Review of "Developing a common, flexible and efficient framework for weakly coupled ensemble data assimilation based on C-Coupler2.0"', Anonymous Referee #2, 01 Jul 2020
- SC8: 'First reply to Reviewer #2', Li Liu, 04 Jul 2020
AC1: 'Final response', Chao Sun, 04 Aug 2020

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by Chao Sun on behalf of the Authors (13 Oct 2020) Author's response Manuscript

ED: Referee Nomination & Report Request started (29 Oct 2020) by Wolfgang Kurtz

RR by Anonymous Referee #1 (12 Nov 2020)

RR by Anonymous Referee #2 (16 Nov 2020)

ED: Reconsider after major revisions (02 Dec 2020) by Wolfgang Kurtz

Dear authors,

thank you for submitting a revised version of your manuscript. Your paper has been send out again to the two former referees. While referee #2 is fine with your changes, referee #1 points out that some major recommendations from the interactive discussion were not taken into account yet, namely regarding structure and clarity of the content.

I agree with referee #2 that the clarity and structure of the manuscript still needs to be improved. This is especially the case for sections 2 and 3. A suggestion from my side would be to first provide a general introduction how the C-Coupler works in a technical sense (e.g. with an example for two models) mentioning required coding steps for coupling, details on data transfer etc. Then you could introduce the changes you made for allowing ensemble propagation and the use of DA algorithms and explain the details of the implementation. Some additional comments and suggestions are given below.

I would like to ask you to include these comments in a further revision of your manuscript.

Kind regards,
Wolfgang Kurtz

Specific comments:

Line 35-36: This statement is a bit misleading as it gives the impression that from a methodological point of view variational methods are a subset of ensemble methods, which is not quite correct. To me it seems that the statement is more from a technical perspective (--> managing the ensemble propagation). I would suggest to rephrase this statement in order to avoid confusion.

Line 40: involving --> evolving

Line 52-53: What do you mean with 'coupled DA system' in this context? Does 'coupling' refer here to DA with coupled models or to the coupling of a DA framework to a model? For example, DART is a quite generic DA framework, that can do DA with any kind of model (be it a single compartment model or a coupled earth system model). Please specify.

Line 93-96: I would not call this 'extra coding'. The choice of processor/ communicator layout is a regular part of the implementation of a DA system (e.g. the coupling of PDAF with a user code) and this somehow depends on the users needs and preferences. It's just that the template files in PDAF include a standard example with the described processor layout that can be relatively easily adapted to user needs as suggested by Dr. Nerger's short comments.

Line 96: COMM_COUPLE, COMM_FILTER, etc. is a quite specific nomenclature for PDAF which is probably not known to the reader. Please explain this in a more general sense.

Line 101-102: '..which does not use global communications,...': It is not quite clear to me what you mean here (i.e. what 'global communication' you refer to). Ensemble filters always require a step where the data from the different realizations are collected. How would the implementation with C-Coupler differ in that respect (as compared to other already existing frameworks)?

Line 102-103: '...and does not require users to develop extra codes...': With reference to your statements in lines 93-96, how does the implementation with C-Coupler differ in that respect? Please provide more details how your implementation can handle different processor layouts for models and DA algorithms. For example, in Figure 2, how would C-Coupler handle a case where you want to run DA algorithm 1 with all available processors instead of the ones shown in the figure?

Line 92/103: It is not clear what you mean with 'process sets'. Please provide more details.

Line 123: Please explain in more detail what you mean with the DLL technique.

Line 168: What do you mean with 'ensemble-set component model'? Please explain more clearly.

Line 201-207: Does that mean that the DA software component is restarted at every assimilation cycle (as suggested by keywords initialize, run, finalize)?

Section 5.4: This is rather short. Please provide a more detailed analysis and description here as a main focus of the paper (already mentioned in the title) is on weakly coupled DA systems.

Line 468-470: '...while still guaranteeing software independence between model and the DA method.': As I understand the coupling approach, you would still need to adapt the model code when using a different DA software (as suggested e.g. by lines 364-377). Please clarify.

Hide

AR by Chao Sun on behalf of the Authors (15 Dec 2020) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (23 Dec 2020) by Wolfgang Kurtz

RR by Anonymous Referee #1 (29 Dec 2020)

Suggestions for revision or reasons for rejection

My concerns about the general structure, especially the sections 2 and 3, of the paper have not been addressed. The text is still extremely difficult to understand. Below are my specific comments to the text of the section 2 and 3 to demonstrate the frustration of a reader. Besides, the WCDA implementation has not been properly introduced despite my previous concerns. I don’t think that this article may be published in its current state in the GMD journal.
Specific comments:
l.84: Maybe like this? Such an implementation maintains the independence between the DA and model modules. But the global communications are generally inefficient in the sequential DA systems because of idle processes due to sequential running of model and DA modules.
L86: I don’t understand these two sentences: In PDAF, a DA method is transformed into a native procedure that is called by the corresponding models via the PDAF application programing interfaces (APIs). Thus, a DA method can share the processes of the model ensemble.
L89: DA method shares all processes of the first ensemble member of the corresponding model… Not clear.
L101. …enables a DA method to share almost all the processes of the model ensemble… not clear
L102. When a DA algorithm uses processes different from a model ensemble member… not clear
l.106. I would modify this sentence like this: Fortunately, such a challenge has already been overcome by most existing couplers (Craig et al., 2012; Valcke, 2012; Liu et al., 2014; Craig et al., 2017; Liu et al., 2018b). Each of these couplers (?) can transfer data between different process sets with different parallel decompositions without the global communications.
L116-118. Something shorter like: Combining multiple components into one MPI program is challenging because CC2 can handle exchanges between two components only.
L120. Please explain how these three new steps help to solve the mentioned problem.
L121. The sentence ‘These three steps enable all members in a model ensemble to use a DA algorithm cooperatively.’ is not clear. Please reformulate it.
l.127 A DA algorithm can include a set of procedures such as observation operators and analysis modules, each of which can be called by the model separately. I would only say: Each module of the DA algorithm may be called separately in the PDAF system. Is this what you are saying here?
L128-130. Instead of ‘The framework uses the dynamic link library (DLL) technique for the connection of a DA algorithm program to a model, where a DA algorithm program is compiled into a DLL that is dynamically linked to a model when an instance of the DA algorithm is initialized.’ I would say: ‘The dynamic link library (DLL) technique is used to connect a DA algorithm program to a model program. The DA algorithm program is compiled into a DLL that is dynamically linked to a model program.’

L130-133. Instead of ‘With the DLL technique, a new DA algorithm can be used by a model without modifying and recompiling the model codes, and the original configuration and compilation systems of a DA algorithm can generally be preserved for greater independence of the DA algorithms from the models and for less work in integrating a DA algorithm.’ I would say ‘Using the DLL technique allows us to couple DA algorithm and model codes without modifying the codes.’

L133-134. The ensemble component manager governs the communicators of ensemble members.

L134-135. The online DA procedure manager provides several APIs that enable the ensemble members of a component model to initialize, run and finalize a DA instance cooperatively. Not clear. And this part of the sentence may be a separate sentence ‘automatically handles the data exchanges between the ensemble members and the DA algorithm’. You can delete ‘with a set of operations’.

L136-138. This is not clear: The ensemble DA configuration manager enables the user to flexibly specify the DA algorithm, DA frequency and the operations for the data exchange in a DA simulation through a configuration file. What is DA frequency? What is ‘to specify the DA algorithm’, what are ‘the operations for the data exchange’?

L139. Not clear: Guided by the architecture in Fig. 2, we implemented the new framework

L143. DA algorithms at different frequencies… What do you mean?
L143. while component 3 does not use DA. Not clear.

L158-161. Not clear.

L169-170. Given an ensemble run of a coupled model, all ensemble members of the component models of the coupled model can be organized as one level of models. Not clear.

L170-174 although we recommend constructing two hierarchical levels of models with 170 the first level corresponding to all ensemble members of the coupled model and each ensemble member including the component models at the second level (Fig. 5b), because the hierarchical organization retains the original architecture of the coupled model through a simple additional registration of the coupled model to C-Coupler2.0.. Not clear.

L174-176. As a DA algorithm that handles ensemble fields can run on the MPI processes of all ensemble members of a component model (Fig. 3), a special C-Coupler2.0 component model that covers all ensemble members of the component model (this 175 special component model is called ensemble-set component model hereafter) is required for using the DA algorithm (Fig. 5b). Not clear.

L177-178. The ensemble component manager provides the capability to generate an ensemble-set component model, which does not introduce global synchronization and only involves the ensemble members of the corresponding component model. Not clear.

L180. A pair of a model and a DA algorithm have essentially the relationship between a caller and a callee in a program. Why is so?

L183 what is native code?

L183 a corresponding compiler… not clear

L185. What is host model?

L186. To address the above challenge... The challenge is not clear to me.

I stopped reading here. This article would have been an excellent reference for the future users of such complicated system. However, I don’t see how it can be helpful for them due to such unclear text.

Hide

ED: Reconsider after major revisions (19 Jan 2021) by Wolfgang Kurtz

AR by Chao Sun on behalf of the Authors (02 Feb 2021) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (17 Feb 2021) by Wolfgang Kurtz

RR by Anonymous Referee #1 (16 Mar 2021)

ED: Publish as is (08 Apr 2021) by Wolfgang Kurtz

Short summary

Data assimilation (DA) provides better initial states of model runs by combining observations and models. This work focuses on the technical challenges in developing a coupled ensemble-based DA system and proposes a new DA framework DAFCC1 based on C-Coupler2. DAFCC1 enables users to conveniently integrate a DA method into a model with automatic and efficient data exchanges. A sample DA system that combines GSI/EnKF and FIO-AOW demonstrates the effectiveness of DAFCC1.