Preprints
https://doi.org/10.5194/gmd-2021-382
https://doi.org/10.5194/gmd-2021-382
Submitted as: model description paper
01 Dec 2021
Submitted as: model description paper | 01 Dec 2021

Massive-Parallel Trajectory Calculations version 2.2 (MPTRAC-2.2): Lagrangian transport simulations on Graphics Processing Units (GPUs)

Lars Hoffmann1, Paul F. Baumeister1, Zhongyin Cai1, Jan Clemens1,2, Sabine Griessbach1, Gebhard Günther2, Yi Heng3, Mingzhao Liu1, Kaveh Haghighi Mood1, Olaf Stein1, Nicole Thomas2, Bärbel Vogel2, Xue Wu4,5, and Ling Zou1 Lars Hoffmann et al.
  • 1Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany
  • 2Institut für Energie- und Klimaforschung (IEK-7), Forschungszentrum Jülich, Jülich, Germany
  • 3School of Computer Science and Engineering, Sun Yat-Sen University, Guangzhou, China
  • 4Key Laboratory of Middle Atmosphere and Global Environment Observation, Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, China
  • 5University of Chinese Academy of Sciences, Beijing, China

Abstract. Lagrangian models are fundamental tools to study atmospheric transport processes and for practical applications such as dispersion modeling for anthropogenic and natural emission sources. However, conducting large-scale Lagrangian transport simulations with millions of air parcels or more can become numerically rather costly. In this study, we assessed the potential of exploiting graphics processing units (GPUs) to accelerate Lagrangian transport simulations. We ported the Massive-Parallel Trajectory Calculations (MPTRAC) model to GPUs using the open accelerator (OpenACC) programming model. The trajectory calculations conducted within the MPTRAC model were fully ported to GPUs, i.e., except for feeding in the meteorological input data and for extracting the particle output data, the code operates entirely on the GPU devices without frequent data transfers between CPU and GPU memory. Model verification, performance analyses, and scaling tests of the MPI/OpenMP/OpenACC hybrid parallelization of MPTRAC were conducted on the JUWELS Booster supercomputer operated by the Jülich Supercomputing Centre, Germany. The JUWELS Booster comprises 3744 NVIDIA A100 Tensor Core GPUs, providing a peak performance of 71.0 PFlop/s. As of June 2021, it is the most powerful supercomputer in Europe and listed among the most energy-efficient systems internationally. For large-scale simulations comprising 108 particles driven by the European Centre for Medium-Range Weather Forecasts' ERA5 reanalysis, the performance evaluation showed a maximum speedup of a factor of 16 due to the utilization of GPUs compared to CPU-only runs on the JUWELS Booster. In the large-scale GPU run, about 67 % of the runtime is spent on the physics calculations, conducted on the GPUs. Another 15 % of the runtime is required for file-I/O, mostly to read the large ERA5 data set from disk. Meteorological data preprocessing on the CPUs also requires about 15 % of the runtime. Although this study identified potential for further improvements of the GPU code, we consider the MPTRAC model ready for production runs on the JUWELS Booster in its present form. The GPU code provides a much faster time to solution than the CPU code, which is particularly relevant for near-real-time applications of a Lagrangian transport model.

Journal article(s) based on this preprint

Lars Hoffmann et al.

Interactive discussion

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on gmd-2021-382', Anonymous Referee #1, 05 Feb 2022
  • RC2: 'Comment on gmd-2021-382', Anonymous Referee #2, 08 Feb 2022
  • AC1: 'Comment on gmd-2021-382', Lars Hoffmann, 08 Mar 2022

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision
AR by Lars Hoffmann on behalf of the Authors (08 Mar 2022)  Author's response    Author's tracked changes    Manuscript
ED: Publish as is (10 Mar 2022) by Christoph Knote

Interactive discussion

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on gmd-2021-382', Anonymous Referee #1, 05 Feb 2022
  • RC2: 'Comment on gmd-2021-382', Anonymous Referee #2, 08 Feb 2022
  • AC1: 'Comment on gmd-2021-382', Lars Hoffmann, 08 Mar 2022

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision
AR by Lars Hoffmann on behalf of the Authors (08 Mar 2022)  Author's response    Author's tracked changes    Manuscript
ED: Publish as is (10 Mar 2022) by Christoph Knote

Journal article(s) based on this preprint

Lars Hoffmann et al.

Lars Hoffmann et al.

Viewed

Total article views: 1,068 (including HTML, PDF, and XML)
HTML PDF XML Total Supplement BibTeX EndNote
862 188 18 1,068 39 14 8
  • HTML: 862
  • PDF: 188
  • XML: 18
  • Total: 1,068
  • Supplement: 39
  • BibTeX: 14
  • EndNote: 8
Views and downloads (calculated since 01 Dec 2021)
Cumulative views and downloads (calculated since 01 Dec 2021)

Viewed (geographical distribution)

Total article views: 1,009 (including HTML, PDF, and XML) Thereof 1,009 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 10 Jan 2023
Download

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Short summary
We describe the new Version 2.0 of the Lagrangian transport model MPTRAC, which has been ported for application on GPUs. The model was verified by comparing kinematic trajectories and synthetic tracer simulations for the troposphere and stratosphere from GPUs and CPUs. Benchmarking showed a speed-up of a factor of 16x of GPU-enabled simulations compared to CPU-only runs, indicating the great potential of applying GPUs for Lagrangian transport simulations on upcoming HPC systems.