P-CSI v1.0, an accelerated barotropic solver for the high-resolution ocean model component in the Community Earth System Model v2.0

Huang, Xiaomeng; Tang, Qiang; Tseng, Yuheng; Hu, Yong; Baker, Allison H.; Bryan, Frank O.; Dennis, John; Fu, Haohuan; Yang, Guangwen

doi:https://doi.org/10.5194/gmd-9-4209-2016

Articles | Volume 9, issue 11

https://doi.org/10.5194/gmd-9-4209-2016

© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.

https://doi.org/10.5194/gmd-9-4209-2016

© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 9, issue 11

Development and technical paper

|

22 Nov 2016

Development and technical paper |

| 22 Nov 2016

P-CSI v1.0, an accelerated barotropic solver for the high-resolution ocean model component in the Community Earth System Model v2.0

Xiaomeng Huang, Qiang Tang, Yuheng Tseng, Yong Hu, Allison H. Baker, Frank O. Bryan, John Dennis, Haohuan Fu, and Guangwen Yang

Download

Final revised paper (published on 22 Nov 2016)
Preprint (discussion started on 01 Jul 2016)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Review of "P-CSI v1.0, an accelerated barotropic solver for the high resolution ocean model component in the Community Earth System Model v2.0"', Eike Müller, 25 Aug 2016
- AC1: 'Reply to Eike Müller', Xiaomeng Huang, 07 Sep 2016
RC2: 'Review of "P-CSI v1.0, an accelerated barotropic solver for the high-resolution ocean model component in the Community Earth System Model v2.0"', Anonymous Referee #2, 30 Aug 2016
- AC2: 'Reply to anonymous referee', Xiaomeng Huang, 07 Sep 2016

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by Xiaomeng Huang on behalf of the Authors (05 Oct 2016) Author's response Manuscript

ED: Referee Nomination & Report Request started (06 Oct 2016) by David Ham

RR by Eike Müller (13 Oct 2016)

Suggestions for revision or reasons for rejection

I think the document has been improved significantly and the authors argue convincingly as to how their work extends what is presented in the SC proceedings article. I would like to thank the authors for addressing the comments in the reviews. Below are a few further (minor) comments which I suggest taking into account before publication. In particular I would still suggest clarifying the definition and discussion of the CFL number in section 4.1 (see also comments on original submission by reviewer 2).

* The overview of changes with respect to the SC paper in lines 104 - 124 is good, but I'd suggest shortening this a bit. Comments like "we rewrote most of the sentences" (line 121) or ``After our presentation at the SC conference in November 2015, much helpful advice was
gathered'' (lines 104 - 105) don't really add any useful information and I don't think the reader will be interested in all the details as to why and how exactly the material was reorganised.

* last sentence in caption of Fig. 10: ``can refer to Fig. 3'' -> ``can be deduced from Fig. 3'' (or something similar)

* In general I think some of the new material would benefit from polishing the English to make sure it is of the same standard as the rest of the article.

* I think it is good that some of the Figures have been removed/merged

* Fig 3: It should be made clear that this figure refers to the number of unknowns per processor, so I would replace ``Number of Grid points'' -> ``Number of unknowns per processor''. The rule of thumb is that strong scaling breaks down once you have $\approx1000-5000$ unknowns per processor, so on the right you are deep in the strong scaling limit. This is consistent with Fig. 9: the halo exchange time is of the same size or even larger than the computation time for $>2000$ cores.

* In section 4 $\Delta t$ is used for the time step size, but in section 2.1 it is $\tau$. Could the same notation be used throughout the paper to avoid confusion?

* I think the discussion of the CFL number in section 4.1 is clearer now, but there are still bits that are very confusing, in particular the exact definition of the CFL number. In line 311 the CFL number in introduced as $CFL=v\Delta t/\Delta x$, which seems to imply that $v$ is some other velocity (i.e. \textit{not} the barotropic velocity $\sqrt{gH}$, which appears separately in the definite of $\Phi$ in line 310. However, then at the end of the section (line 335) the CFL number is redefined as $CFL=c\Delta t/\Delta x$, which is the CFL number of gravity waves. The CFL number of explicitly treated modes is $\approx 100$ smaller than $3.46$ (otherwise they would not be stable). Can you therefore clarify that $CFL=v*\Delta t/\Delta x$ is the CFL number for the \textit{non-barotropic} modes in the system, i.e. modes which propagate with $v \ll \sqrt{gH}$? I understand that in this section you concentrate on variation in the CFL number (due to variations in time step size, grid resolution and $v$) and their impact on the solver performance, but still think that a discussion of the relative size of $v$ and $\sqrt{gH}$ would be useful here. E.g. saying something like: ``$\sqrt{gH}\sim200m/s$ is the speed of the gravity waves which are treated implicitly in the barotropic solver. $v$ is the speed of other, explicitly treated processes in the system and typically $v<\sqrt{gH}$. While in the POP model at 0.1 resolution $v\sim 2m/s and $CFL \approx 0.0346$, in this section we also consider a set of velocitities $v$ and a wider range of CFL numbers to make more general statements on the algorithmic performance of the solver.''
Alternatively, to avoid the confusion above, the (barotropic) CFL number could be defined as $CFL=c\Delta t/\Delta x=3.46$ throughout the section. Then simply $\Phi=1/CFL^2$, as also noted by reviewer 2.

* top of page 18: typo, "extrem" -> "extreme"

* line 158/158: equation is broken very awkwardly at the $\cdot$

* How is $\overline{H}$ defined in appendix A? If it is the average depth, and $\min{H}\le\overline{H}\le \max{H}$, then not both inequalities in (A3) and (A4) can be true. The last inequality in (A4) should be $\ge 2\min(\alpha-1/\alpha,1/\alpha-\alpha)\min{H}+\Phi\max{H}$. Eq. (17) would have to be adjusted accordingly.

Hide

RR by Anonymous Referee #2 (24 Oct 2016)

ED: Publish subject to minor revisions (Editor review) (25 Oct 2016) by David Ham

AR by Xiaomeng Huang on behalf of the Authors (03 Nov 2016) Author's response Manuscript

ED: Publish as is (03 Nov 2016) by David Ham

AR by Xiaomeng Huang on behalf of the Authors (04 Nov 2016) Manuscript

Short summary

Refining model resolution is helpful for representing climate processes. With resolution increasing, the computational cost will become very huge. We designed a new solver to accelerate the high-resolution ocean simulation so as to reduce the computational cost and make full use of the computing resource of supercomputers. Our results show that the simulation speed of the improved ocean component with 0.1° resolution achieves 10.5 simulated years per wall-clock day on 16875 CPU cores.