Porting the WAVEWATCH III (v6.07) Wave Action Source Terms to GPU

Abstract. Surface gravity waves play a critical role in several processes, including mixing, coastal inundation and surface fluxes. Despite the growing literature on the importance of ocean surface waves, wind-wave processes have traditionally been excluded from Earth system models due to the high computational costs of running spectral wave models. The Next Generation Ocean Model Development in the DOE’s (Department of Energy) E3SM (Energy Exascale Earth System Model) project partly focuses on the inclusion of a wave model, WAVEWATCH III (WW3), into the E3SM. WW3, which was originally developed for operational wave forecasting, needs to be computationally less expensive before it can be integrated into ESMs. To accomplish this, we take advantage of heterogeneous architectures at DOE leadership computing facilities and the increasing computing power of general-purpose graphics processing units (GPU). This paper identifies the wave action source terms as the most computationally intensive module in WW3 and then accelerates them via GPU. Using one GPU, our experiments on two computing platforms, Kodiak (P100 GPU & Intel(R) Xeon(R) CPU E5-2695 v4) and Summit (V100 GPU & IBM POWER9), show speedups of up to 2.4x and 6.6x respectively over one MPI task on CPU. Using different combinations of multiple CPUs and GPUs, we obtained an average speedup of 2x and 4x on Kodiak and Summit. We also discuss how the trade off between occupancy, register and latency affects the GPU performance of WW3.