Meteorological processes and advection of atmospheric constituents in the WRF-GC coupled model are simulated by the WRF model (v3.9.1.1 or later versions), a mesoscale numerical weather model for research and operational applications . WRF offers its users many options for model configurations and physical schemes. WRF uses the Advanced Research WRF (ARW) dynamical solver, which solves fully compressible, Eulerian non-hydrostatic equations on either hybrid sigma–eta (default) or terrain-following vertical coordinates defined by the user. Horizontal grids in WRF are staggered Arakawa C grids, which can be configured by the user using four map projections: latitude–longitude, Lambert conformal, Mercator, and polar stereographic. WRF supports the use of multiple nested domains to simulate the interactions between large-scale dynamics and mesoscale meteorology. WRF supports grid, spectral, and observational nudging . This allows the WRF model to produce meteorological outputs that mimic assimilated meteorological fields for use in air quality hindcasts. The WRF model offers many options for land surface physics, planetary boundary layer physics, radiative transfer, cloud microphysics, and cumulus parameterization, for use in meteorological studies, real-time numerical weather prediction, idealized simulations, and data assimilation on meso- to regional scales . Table lists the configuration and physical options supported by WRF-GC v1.0. In particular, only the hybrid sigma–eta vertical coordinate is currently supported in WRF-GC.

The WRF model incorporates a highly modular software framework that is portable across a range of computing platforms. WRF supports two-level domain decomposition for distributed-memory (Message Passing Interface; MPI) and shared-memory (OpenMP) parallel computation. Distributed parallelism is implemented through the Runtime System Library lite (RSL-lite) module, which supports irregular domain decomposition, automatic index translation, distributed input/output, and low-level interfacing with MPI libraries .

Our development of WRF-GC was made possible by a recent structural overhaul of GEOS-Chem , which enabled the use of GEOS-Chem as a self-contained chemical module within the WRF-GC model. The original GEOS-Chem CTM (prior to v11.01) was structured specifically for several static sets of global or regional 3-D grids at prescribed horizontal and vertical resolutions . Parallelism for the original GEOS-Chem was implemented through OpenMP, such that the original GEOS-Chem can only operate on single-node hardware with large shared memory. restructured the core processes in GEOS-Chem, including emission, chemistry, convective mixing, planetary boundary layer mixing, and deposition processes, to work in modular units of atmospheric vertical columns. Information about the horizontal grids, formerly fixed at compile time, is now passed to the GEOS-Chem chemical module at runtime. This development enabled the use of the GEOS-Chem chemical module with any horizontal grid structure and horizontal resolution.

The new, modularized structure of the GEOS-Chem has been implemented in two types of configurations. The first type of configuration uses GEOS-Chem as the core of offline CTMs. For example, in the GEOS-Chem “Classic” implementation (GCC), the GEOS-Chem chemical module is driven by the GEOS assimilated meteorological data and is parallelized using OpenMP. This implementation treats the prescribed global or regional model domains as contiguous sets of atmospheric columns placed at regular latitude/longitude intervals, with vertical layers pre-defined to match those of the GEOS model. In essence, the GCC implementation mimics the “original” GEOS-Chem model before the structural overhaul by . Other grid systems can also be used with the GEOS-Chem chemical module. For example, the GEOS-Chem High Performance (GCHP) implementation calls the GEOS-Chem chemical module on the native cubed-sphere coordinates of the NASA GEOS model via a column interface in GEOS-Chem (GIGC_Chunk_Run). This column interface was built on the Earth System Modeling Framework (ESMF) and permits runtime specification of the horizontal grid parameters. The GCHP implementation uses MPI to parallelize GEOS-Chem across nodes through the Model Analysis and Prediction Layer framework (MAPL) , which is a wrapper on top of ESMF specifically designed for the NASA GEOS model system.

Alternatively, GEOS-Chem can be used as a module coupled to weather models or Earth system models to perform online chemical calculations. Using this capability, developed an online implementation of GEOS-Chem by coupling it to the NASA GEOS-5 model to simulate global atmospheric chemistry. coupled GEOS-Chem to the Beijing Climate Center Atmospheric General Circulation Model (BCC-AGCM). However, both the NASA GEOS-5 model and the BCC-AGCM are proprietary.

WRF-GC is the first implementation that couples the GEOS-Chem chemical module to an open-access high-resolution meteorological model. We developed a modular coupler between WRF and GEOS-Chem that draws from the technology of GCHP but does not rely on ESMF (described in Sect. ). We also made changes to GEOS-Chem to accept arbitrary vertical discretization from WRF at runtime and to improve physical compatibility with WRF (described in Sect. ). These changes have been incorporated into the mainline GEOS-Chem code. Our coupler and code modifications can be adapted in the future to couple GEOS-Chem to other non-ESMF Earth system models.

Chemical calculations in WRF-GC v1.0 use GEOS-Chem version 12.2.1 (10.5281/zenodo.2580198, ). The standard chemical mechanism in GEOS-Chem v12.2.1, used by default in WRF-GC, includes detailed Ox–NOx–VOC–ozone–halogen–aerosol in the troposphere, as well as the unified tropospheric–stratospheric chemistry extension (UCX) for stratospheric chemistry and stratosphere–troposphere exchange. The standard gas-phase mechanism includes 208 chemical species and 981 reactions. Reactions and rates follow the latest recommendations from the Jet Propulsion Laboratory and the International Union of Pure and Applied Chemistry. In addition, GEOS-Chem uses the FlexChem pre-processor (a wrapper for the Kinetic PreProcessor, KPP; ) to configure chemical kinetics . This allows users to add or modify gaseous species and reactions to develop custom mechanisms and diagnostic quantities in GEOS-Chem. GEOS-Chem also supports the optional “Tropchem” (troposphere-only chemistry) mechanism, where UCX is disabled and replaced by a parameterized linear chemistry in the stratosphere . GEOS-Chem uses the Rosenbrock Rodas3 solver for gas-phase chemistry by default, but users may choose other solvers through KPP.

Aerosol species in GEOS-Chem include secondary inorganic aerosols (sulfate, nitrate, ammonium), elemental carbon aerosol (EC), primary organic carbon (POC), secondary organic aerosol (SOA), dust, and sea salt. By default, secondary inorganic aerosols, EC, POC, and SOA are simulated as speciated bulk masses. Dust aerosols are represented in 4 size bins (0.1–1.0, 1.0–1.8, 1.8–3.0, and 3.0–6.0 µm) , while sea salt aerosols are represented in two size bins (0.1–0.5 and 0.5–4.0 µm) . The thermodynamics of secondary inorganic aerosol are coupled to gas-phase chemistry and computed by the ISORROPIA II module . EC and POC are represented in GEOS-Chem as partially hydrophobic and partially hydrophilic, with a conversion timescale from hydrophobic to hydrophilic of 1.2 d . The organic matter to organic carbon (OM / OC) mass ratio is assumed to be 2.1 for POC by default, with an option to use seasonally and spatially varying OM / OC ratios . GEOS-Chem has two options to describe the production of SOA, and both options are supported in WRF-GC. By default, SOA is produced irreversibly using simple yields from anthropogenic and biogenic volatile organic precursors . Alternatively, GEOS-Chem can simulate SOA production via the aging of semi-volatile and intermediate volatility organic precursors using a volatility basis set (VBS) scheme , as well as via the aqueous reactions of the oxidation products from isoprene . The GEOS-Chem model also has the option of simulating detailed, size-dependent aerosol microphysics using the TwO-Moment Aerosol Sectional microphysics (TOMAS) module or the Advanced Particle Microphysics (APM) module , but these two modules are not yet supported in WRF-GC.

Emissions of chemical species in WRF-GC are calculated using the Harvard-NASA Emissions Component (HEMCO) in GEOS-Chem . HEMCO allows users to select emission inventories from the HEMCO data directory or add their own inventories, and interpolate the emission fluxes to the model domain and resolution at runtime. The HEMCO data directory currently includes more than 20 global and regional emission inventories, mostly at their respective native resolutions (http://wiki.seas.harvard.edu/geos-chem/index.php/HEMCO_data_directories, last access: 24 April 2020). By default, the Community Emissions Data System (CEDS) inventory (0.5∘×0.5∘ resolution, monthly) is used for most of the world; over Asia and the US, the CEDS is superseded by the MIX inventory (0.25∘×0.25∘ native resolution, monthly; ) and the 2011 National Emission Inventory (NEI 2011) (0.1km×0.1km native resolution, hourly; ), respectively. HEMCO also has extensions to compute emissions with meteorological dependencies, such as the emissions of biogenic species , soil NOx , lightning NOx, sea salt , and dust . With the exception of lightning NOx, these meteorology-dependent emissions are supported in WRF-GC v1.0. Further details about the use of HEMCO in WRF-GC are given in Sect. .

Other physical calculations in GEOS-Chem are coupled to WRF meteorological fields in WRF-GC; we describe the coupling in detail in Sect. . Convective transport of chemical species is calculated using a single-plume parameterization , which is in turn driven by the cumulus parameterization in WRF. Boundary layer mixing is calculated using a non-local scheme, driven by the WRF-simulated atmospheric instability and boundary layer height . Dry deposition is based on a resistance-in-series scheme . Aerosol deposition is as described in , with updates to account for size dependency for dust and sea salt . Wet scavenging of gases and water-soluble aerosols is as described in and and driven by WRF-simulated precipitation rates.

Figure gives an architectural overview of the WRF-GC coupled model. Our development of WRF-GC uses many of the existing infrastructure in the WRF-Chem model that couples WRF to its chemistry module . The subroutine calls between WRF and the chemistry components are exactly the same in WRF-GC and in WRF-Chem. Within WRF-GC, the WRF model and the GEOS-Chem model remain entirely intact. The WRF-GC coupler consists of interfaces with the two parent models, as well as a state conversion module and a state management module. The coupler is separate from both parent models and is written in a manner similar to an application programming interface, enabling independent updates of the parent models within WRF-GC.

The WRF-GC model is initialized and driven by WRF, which sets up the simulation domain, establishes the global clock, handles the input and output of data, sets the initial and boundary conditions (IC/BC) for meteorological and chemical variables, and manages cross-processor communication for parallelization. Users configure the WRF-GC model structure in the WRF configuration file (namelist.input), including the domain, projection, horizontal resolution, vertical coordinates, simulation time, dynamic time step, external chemical time step, and other physical and dynamical options. Users also “turn on” GEOS-Chem in WRF-GC by specifying chem_opt = 233 in namelist.input, similar to the way that users specify the chemical mechanism in WRF-Chem. GEOS-Chem is initialized by the WRF model using the WRF-to-chemistry interface described in Sect. . Chemical options within GEOS-Chem, including the choice of standard or custom chemical mechanisms, emission inventories in HEMCO, and diagnostic quantities to be output, are defined by users in the GEOS-Chem configuration files (input.geos, HEMCO_Config.rc, and HISTORY.rc). IC/BC for meteorological and chemical variables are prepared by the user in NetCDF format and read by WRF. Meteorological IC/BC can be prepared using the WRF pre-processor system (WPS) from datasets available from NCAR's Research Data Archive (https://rda.ucar.edu, last access: 24 April 2020). IC/BC of chemical species concentrations are taken from GEOS-Chem Classic global model outputs and interpolated to the WRF-GC model grids using a modified version of the WRF-Chem pre-processor tool mozbc (available along with the WRF-GC code).

Dynamical and physical calculations are performed in WRF-GC exactly as they are in the WRF model. WRF also calculates the grid-scale advection of chemical species. At the beginning of each chemical time step, WRF calls the WRF-GC chemistry component through the WRF-to-chemistry interface, chem_driver, which is a chemical driver similar to that in WRF-Chem. Spatial parameters and the internal state of WRF are translated at runtime to GEOS-Chem by the state conversion and management modules. The GEOS-Chem chemical module then performs convective transport, dry deposition, wet scavenging, emission calculations, boundary layer mixing, and chemistry calculations. This operator splitting between WRF and GEOS-Chem is identical to that between WRF and the chemistry module in WRF-Chem. Then, the GEOS-Chem internal state is translated back to WRF, and the WRF time stepping continues. At the end of the WRF-GC simulation, WRF outputs all meteorological and chemical diagnostic quantities in WRF's standard format.

WRF-GC v1.0 supports all the existing input and output functionality of the WRF model, including serial/parallel reading and writing of NetCDF, HDF5, and GRIB2 datasets. WRF and WRF-Chem users can use existing data pre- and post-processing tools to prepare input data and analyze results from WRF-GC.

Below, we describe the operator splitting between WRF and GEOS-Chem within WRF-GC, as well as the treatments of some of the key processes. The general Eulerian form of the continuity equation for m chemical species with number density vector n=(n1,…,nm)T is 1∂ni∂t=-∇⋅(niU)+Pi(n)+Li(n)i∈[1,m]. U is the wind vector, which is provided by the WRF model in WRF-GC. The first term on the right-hand side of Eq. () indicates the transport of species i, which includes grid-scale advection, as well as subgrid turbulent mixing and convective transport. Pi(n) and Li(n) are the local production and loss rates of species i, respectively .

In WRF-GC, WRF simulates the meteorological variables using the dynamic equations and meteorological IC/BC. These meteorological variables are then passed to the GEOS-Chem chemical module (Table ) to solve the local production and loss terms of the continuity equation. Large-scale (grid-scale) advection of chemical species is grid-aware and is calculated by the WRF dynamical core. Local (subgrid) vertical transport processes, including turbulent mixing within the boundary layer and convective transport from the surface to the convective cloud top, are calculated in GEOS-Chem using WRF-simulated meteorology. Dry deposition and wet scavenging of chemical species are also calculated in GEOS-Chem and driven by WRF. This operator-splitting arrangement is identical to that in the WRF-Chem model.

The dynamic and chemical time steps are specified by the user in the WRF configuration file namelist.input. The dynamic time step is constrained by the Courant–Friedrichs–Lewy stability criterion and should be short for high-resolution simulations. WRF-Chem recommends that the chemical time step be set the same as the dynamic time step as best practice . Because GEOS-Chem uses a Rosenbrock solver, which adapts its internal chemical time step to the stiffness of the chemical mechanism, a larger chemical time step may be used. However, it is recommended that the results be compared to a control simulation with the chemical time step set to the dynamic time step .

Figure a shows the domain of the GEOS-Chem Classic nested-China simulation, which was driven by the GEOS-FP assimilated meteorological dataset at its native horizontal resolution of 0.25∘ latitude ×0.3125∘ longitude. The vertical resolution of the GEOS-FP dataset was reduced from its native 72 levels to 47 levels by lumping levels in the stratosphere. This treatment is standard in GEOS-Chem Classic simulations. The 47 vertical layers extended from the surface to 0.01 hPa, with seven levels in the bottom 1 km. The top edge of the bottom layer was at approximately 120 m over eastern China. Meteorological variables were updated every 3 h (every hour for surface variables). IC/BC of chemical species concentrations were taken from the outputs of a global GEOS-Chem Classic simulation and updated at the boundaries of the nested-China domain every 3 h. The dynamic time step and the external chemistry time step were 5 and 10 min, respectively.

Figure b shows the WRF-GC simulation domain, with a horizontal resolution of 27km×27km. We chose this domain and horizontal resolution to be comparable to those of the GEOS-Chem Classic nested-grid simulation. There were 50 vertical levels in our WRF-GC simulation, which extended from the surface up to 10 hPa with 7 levels below 1 km. The top edge of the bottom layer was at approximately 60 m over eastern China. The WRF model does not have the option of using the GEOS-FP dataset for meteorological IC/BC. Instead, we used the NCEP FNL dataset (10.5065/D6M043C6, ) at 1∘×1∘ resolution as IC/BC for WRF-GC; the FNL dataset was interpolated to WRF vertical levels and updated every 6 h. In addition, we nudged the WRF-simulated meteorological fields with surface (every 3 h) and upper air (every 6 h) observations of temperature, specific humidity, and winds from the NCEP ADP Global Surface/Upper Air Observational Weather Database (10.5065/39C5-Z211, ). This mimicked the effect of meteorological data assimilation and allowed the WRF-simulated meteorology to stay close to the observed states of the atmosphere. IC/BC of chemical species concentrations were identical to those used in the GEOS-Chem Classic nested-China simulation but interpolated to WRF vertical levels and updated every 6 h. The dynamic time step and the external chemistry time step were 2 and 10 min, respectively. Other physical options used in our WRF-GC simulation are summarized in Table .

Our WRF-GC and GEOS-Chem Classic simulations used the exact same GEOS-Chem chemical mechanism for gases and aerosols, including a total of 241 chemical species. Emissions in the two simulations were both calculated by HEMCO and were identical for anthropogenic and biomass burning sources. Monthly mean anthropogenic emissions from China were from the Multi-resolution Emission Inventory for China (MEIC; ) at 0.25∘×0.25∘ horizontal resolution. The MEIC inventory was updated for the year 2015 and included emissions from power generation, industry, transportation, and residential activities. Sector-specific weekly and diurnal variations from the MEIC inventory were applied . Agricultural emissions of ammonia were from . Anthropogenic emissions from the rest of the Asia were from , developed for the year 2010. Monthly mean biomass burning emissions were taken from Global Fire Emissions Database version 4 (GFED4) . Emissions of biogenic species , soil NOx , sea salt , and dust in the two simulations were coupled to model meteorology in HEMCO and thus different between the two models. PM2.5 mass concentrations were diagnosed for both simulations as the sum of masses of sulfate, nitrate, ammonium, EC, primary organic aerosol (assumed to be 2.1 times the primary organic carbon mass), SOA, fine dust (100 % of dust between 0 and 0.7 µm and 38 % of dust between 0.7 and 1.4 µm), and accumulation-mode sea salt, taking into consideration the hygroscopic growth for each species at 35 % relative humidity.

Figure  compares the 6 d average surface PM2.5 concentrations during 22 to 27 January 2015 as in the WRF-GC and the GEOS-Chem Classic nested-China simulations, respectively. Also shown are the PM2.5 concentrations measured at 578 surface sites, managed by the Ministry of Ecology and Environment of China (http://www.cnemc.cn, last access: 24 April 2020). We removed invalid hourly surface PM2.5 observations following the protocol in . The 578 sites were selected by (1) removing surface sites with less than 80 % valid hourly measurements during our simulation period, and (2) sampling the site closest to the model grid center, if that model grid contained multiple surface sites. Both models reproduced the general spatial distributions of the observed PM2.5 concentrations, including the high concentrations over eastern China, as well as the hotspots over the North China Plan, central China, and the Sichuan Basin. However, both models overestimated the PM2.5 concentrations over eastern China. The mean 6 d PM2.5 concentrations averaged for the 578 sites as simulated by WRF-GC and by GEOS-Chem Classic nested-China were 117±68 and 120±76 µgm-3, respectively. In comparison, the observed mean 6 d PM2.5 concentration averaged for the 578 sites was 98±43 µgm-3.

Figure shows the scatter plots of the simulated and observed daily average PM2.5 concentrations over eastern China (eastward of 103∘ E, 507 sites) during 22 to 27 January 2015. We focused on eastern China, because the spatiotemporal variability of PM2.5 concentrations was higher over this region. Again, both models overestimated the daily PM2.5 concentrations over eastern China. The daily PM2.5 concentrations simulated by WRF-GC were 29 % higher than the observations (quantified by the reduced major-axis regression slope between the simulated and observed daily PM2.5 concentration), with a correlation coefficient of r=0.68. The daily PM2.5 concentrations in the GEOS-Chem Classic nested-China simulation were 55 % higher than the observations, with a correlation coefficient of r=0.72.

Figure shows the Taylor diagrams of the hourly PM2.5 concentrations simulated by the two models at 48 major eastern Chinese cities, including 13 cities in the Beijing–Tianjin–Hebei (BTH) area, 22 cities in the Yangtze River Delta (YRD) area, and 13 other major cities. The Taylor diagram evaluates the simulated time series of PM2.5 against the observations, using the Pearson correlation coefficients, the ratio between the simulated and observed standard deviations (σsimσobs), and the normalized root mean square differences (RMSDs) as metrics. Proximity to the point “1” on the x axis in Fig.  indicates that the simulation accurately reproduced both the mean concentration and the temporal variability of the observations. For most cities in the BTH area and for most of the other 13 major cities, the hourly PM2.5 concentrations simulated by WRF-GC showed smaller RMSDs and higher correlation coefficients against observations, compared to those in the GEOS-Chem Classic nested-China simulation. In the YRD area, the performance of the two models was similar.

Our analyses above show that the hourly and daily surface PM2.5 concentrations simulated by the WRF-GC model were in better agreement with observations than those simulated by the GEOS-Chem Classic nested-China model over eastern China during 22 to 27 January 2015. We found that this was partially because the WRF-GC model, nudged with surface and upper-air meteorological observations, better represented the pollution meteorology, compared to the GEOS-FP dataset that was used to drive the GEOS-Chem Classic nested-China simulation. Figure S1 shows the average surface air temperature, relative humidity, and 10 m wind speed as simulated by the WRF-GC model and as provided by the GEOS-FP dataset against the observations during 22 and 27 January 2015 at 367 sites over China. The surface air temperature simulated by WRF-GC and those in the GEOS-FP dataset were both in good agreement with the observations over China. However, the relative humidity and wind speeds simulated by WRF-GC were more consistent with the observations, compared to those in the GEOS-FP dataset. Figure S2 assesses the hourly surface air temperature, relative humidity, and near-surface winds simulated by the WRF-GC model and those in the GEOS-FP assimilated meteorological dataset, against hourly surface measurements over China during 22–27 January 2015. For the 34 sites with publicly available hourly measurements, the meteorological fields simulated by the WRF-GC were generally more consistent with the measurements.

Figure shows the mean planetary boundary layer height (PBLH) at 08:00 LT (00:00 UTC) and 20:00 LT (12:00 UTC) during 22 to 27 January 2015 in the GEOS-Chem Classic nested-China and the WRF-GC simulations, respectively, and compares them with the rawinsonde observations during this period . The PBLH in the GEOS-Chem Classic model was taken from the GEOS-FP dataset, whereas the boundary layer height was simulated by WRF in WRF-GC. Compared to the observations, the PBLH in the GEOS-FP dataset were generally biased low over eastern China and biased high over the mountainous areas in southwestern China and western China. This likely was a major reason for the severe overestimation of surface PM2.5 concentrations in the GEOS-Chem Classic nested-China simulation over eastern China. In comparison, the WRF-GC model correctly represented the PBLH over most regions in China, which was critical to its more accurate simulation of surface PM2.5 concentrations.

We evaluated the computational performance of a WRF-GC simulation and compared it with that of the GEOS-Chem Classic nested-grid simulation of a similar configuration. We configured the WRF-GC and GEOS-Chem Classic nested-grid simulations for the exact same domain (as shown in Fig. a), with the exact same projection and horizontal resolution (0.25∘ latitude ×0.3125∘ longitude resolution, 225×161 atmospheric columns). The GEOS-Chem Classic nested-grid simulation had 47 vertical levels, and the WRF-GC simulation comparably had 50 levels. Both simulations used the same emissions and chemical configurations with a total of 241 chemical species and 10 min external chemical time steps. WRF-GC calculated meteorology online with a 2 min dynamic time step. The GEOS-Chem Classic nested-grid simulation calculated transport (5 min time step) driven by archived GEOS-FP meteorological data read from an ethernet-connected hard disk array. WRF-GC used MPI parallelization, while GEOS-Chem Classic used OpenMP. Both simulations were conducted for 48 h with scheduled output for every 1 h. Both simulations were executed on the same single-node hardware with 32 Intel Broadwell physical cores and an ethernet-connected hard disk array.

Figure compares the computational wall times for the WRF-GC and the GEOS-Chem Classic nested-grid simulations. The total wall time for the WRF-GC simulation was 5127 s, only 31 % of the wall time for the GEOS-Chem Classic nested-grid simulation (16 391 s). We found that the difference in computational efficiency was mainly due to the much faster dynamic and transport calculations in the WRF-GC model relative to the transport calculation in the GEOS-Chem Classic nested-grid model. In WRF-GC, the wall time taken up by the entire WRF (including transport, physics, I/O, and model initiation) was 2462.5 s. In the GEOS-Chem Classic nested-grid simulation, 50 % (8192 s) of the total wall time was used for the transport of tracers, including large-scale advection (6355.7 s), convective transport (694.2 s), and boundary layer mixing (1142.5 s). As a CTM, the GEOS-Chem Classic read archived meteorological data for the entire domain at 3-model-hour intervals from hard drives using a single computational core, which becomes increasingly burdensome for simulations with more grid boxes. In comparison, WRF-GC calculated meteorology online in node memory and updated the model boundary conditions from hard drives every 6 model hours. Finally, the OpenMP parallelization in GEOS-Chem Classic was only at the loop level. In contrast, WRF-GC used MPI parallelization to decompose domain at the model level, thus parallelizing all code within the GEOS-Chem module. As a result, the same chemistry routines actually ran faster in WRF-GC than they did in GEOS-Chem Classic. In WRF-GC, the chemistry routines (deposition, emissions computation, convection, boundary layer mixing, and chemistry) took up 2462.5 s in total; the same routines took up 5263.5 s in GEOS-Chem Classic. The WRF-GC coupler consumed negligible wall time (39 s) in this test simulation.

A side-by-side wall time comparison between WRF-GC and WRF-Chem is difficult to do, because (1) the chemical routines in the two models are very different, and (2) WRF-Chem has many possible configurations for chemistry. Nevertheless, we conducted one test simulation with WRF-Chem using a typical chemistry option (CBMZ-MOSAIC gas-phase chemistry, four-bin aerosol microphysics and chemistry, and aqueous reactions; chem_opt = 9 in registry.chem) with a total of 133 chemical species. This WRF-Chem simulation was configured for the same domain, at the same horizontal and vertical resolutions, and used the same physical and dynamical options as those in the WRF-GC simulation described above. The total wall time for the WRF-Chem simulation was 9985.8 s, which was almost twice as long as the wall time of WRF-GC (5127 s). Chemical routines in this WRF-Chem test simulation took up 61 % of the total wall time (6134.3 s), despite the WRF-Chem having much fewer chemical species than the WRF-GC (241 chemical species). This may be partially due to the computationally intensive bin-resolved aerosol microphysics calculation in the WRF-Chem simulation.

We analyzed the scalability of the WRF-GC model using the 48 h simulation described in Sect. . The model was compiled using the Intel C and Fortran compilers (v16.0.3) and the mvapich2 (v2.3) MPI library. The computing environment (Tianhe-1A) had 28 Intel Broadwell physical cores with 125 GB of RAM per node. Input and output used a networked Lustre high-performance file system.

Figure shows the scalability of the WRF-GC simulation in terms of the total WRF-GC wall time, as well as the wall times of its three components: (1) the WRF model (including input/output), (2) the GEOS-Chem model, and (3) the WRF-GC coupler. For the domain of this test simulation, the total wall time and the WRF wall time both scaled well up to 136 cores. This was because the simulation domain was too fragmented above 136 cores, such that the MPI communication time took up much of the runtime, resulting in performance degradation. Chemical calculations in the GEOS-Chem model were perfectly scalable up to the largest number of cores tested (250 cores), consistent with previous GCHP performance analyses . Figure  also shows that the WRF-GC coupler scaled nearly perfectly and consumed less than 1 % of the total WRF-GC wall time. At above 200 cores there was a slight degradation of the scalability due to cross-core communications at the subdomain boundaries. However, the degradation had negligible impact on the total WRF-GC wall time as the WRF-GC coupler was computationally efficient.

WRF-GC also scales to massively parallel architectures and can be deployed on the cloud, because both the WRF and GEOS-Chem model are already operational on the cloud with their input data readily available . We conducted a test simulation running WRF-GC on the Amazon Web Services (AWS) cloud with up to 128 nodes and 4608 cores. The simulation domain was over the continental US at 5km×5km resolution (950×650 atmospheric columns), with a 10 s dynamical time step and 5 min chemical time step. The scalability test results are shown in Fig. S3. In this massively parallel environment, WRF-GC scaled well up to 1728 cores, with the chemical module scaling well up to 2304 cores. The WRF-GC coupler took less than 0.2 % of the total computational time in this simulation and scaled perfectly up to 4608 cores. The deployability of WRF-GC on the cloud will enhance WRF-GC's accessibility to new users by saving them the investment in hardware purchases and the effort in downloading and hosting large input datasets locally.