Articles | Volume 13, issue 7
https://doi.org/10.5194/gmd-13-3241-2020
https://doi.org/10.5194/gmd-13-3241-2020
Model description paper
 | 
16 Jul 2020
Model description paper |  | 16 Jul 2020

WRF-GC (v1.0): online coupling of WRF (v3.9.1.1) and GEOS-Chem (v12.2.1) for regional atmospheric chemistry modeling – Part 1: Description of the one-way model

Haipeng Lin, Xu Feng, Tzung-May Fu, Heng Tian, Yaping Ma, Lijuan Zhang, Daniel J. Jacob, Robert M. Yantosca, Melissa P. Sulprizio, Elizabeth W. Lundgren, Jiawei Zhuang, Qiang Zhang, Xiao Lu, Lin Zhang, Lu Shen, Jianping Guo, Sebastian D. Eastham, and Christoph A. Keller
Abstract

We developed the WRF-GC model, an online coupling of the Weather Research and Forecasting (WRF) mesoscale meteorological model and the GEOS-Chem atmospheric chemistry model, for regional atmospheric chemistry and air quality modeling. WRF and GEOS-Chem are both open-source community models. WRF-GC offers regional modellers access to the latest GEOS-Chem chemical module, which is state of the science, well documented, traceable, benchmarked, actively developed by a large international user base, and centrally managed by a dedicated support team. At the same time, WRF-GC enables GEOS-Chem users to perform high-resolution forecasts and hindcasts for any region and time of interest. WRF-GC uses unmodified copies of WRF and GEOS-Chem from their respective sources; the coupling structure allows future versions of either one of the two parent models to be integrated into WRF-GC with relative ease. Within WRF-GC, the physical and chemical state variables are managed in distributed memory and translated between WRF and GEOS-Chem by the WRF-GC coupler at runtime. We used the WRF-GC model to simulate surface PM2.5 concentrations over China during 22 to 27 January 2015 and compared the results to surface observations and the outcomes from a GEOS-Chem Classic nested-China simulation. Both models were able to reproduce the observed spatiotemporal variations of regional PM2.5, but the WRF-GC model (r=0.68, bias =29 %) reproduced the observed daily PM2.5 concentrations over eastern China better than the GEOS-Chem Classic model did (r=0.72, bias =55 %). This was because the WRF-GC simulation, nudged with surface and upper-level meteorological observations, was able to better represent the pollution meteorology during the study period. The WRF-GC model is parallelized across computational cores and scales well on massively parallel architectures. In our tests where the two models were similarly configured, the WRF-GC simulation was 3 times more efficient than the GEOS-Chem Classic nested-grid simulation due to the efficient transport algorithm and the Message Passing Interface (MPI)-based parallelization provided by the WRF software framework. WRF-GC v1.0 supports one-way coupling only, using WRF-simulated meteorological fields to drive GEOS-Chem with no chemical feedbacks. The development of two-way coupling capabilities, i.e., the ability to simulate radiative and microphysical feedbacks of chemistry to meteorology, is under way. The WRF-GC model is open source and freely available from http://wrf.geos-chem.org (last access: 10 July 2020).

Dates
1 Introduction

Regional models of atmospheric chemistry simulate the emission, transport, chemical evolution, and removal of atmospheric constituents over a given domain. These models are widely useful for forecasts of air quality, for impact assessment associated with polluting activities, and for theory validation by comparisons against observations. It is thus crucial that regional models be frequently updated to reflect the latest scientific understandings of atmospheric processes. At the same time, the increasing demand for fine-resolution simulations requires models to adapt to massively parallel computation architectures with high scalability. We present here the development of a new regional atmospheric chemistry model: WRF-GC, an online coupling of the Weather Research and Forecasting (WRF) mesoscale meteorological model and the GEOS-Chem atmospheric chemistry model, specifically designed to allow easy updates and be computationally efficient, for use in research and operation applications.

Regional atmospheric chemistry models fall into two categories: offline models and online models. Offline models (also called chemical transport models; CTMs) use archived meteorological fields, either those simulated by models alone or those assimilated with observations, to drive the transport and chemical evolution of atmospheric constituents (Baklanov et al.2014). By eliminating the need to solve dynamical processes online, the developers of offline models can focus their efforts to solving more complex chemical processes. For example, one popular regional CTM is the GEOS-Chem model in its nested-grid configuration (Bey et al.2001; Wang et al.2004; Chen et al.2009; Zhang et al.2015), which is driven by high-resolution assimilated meteorological data from the GEOS model of the NASA Global Modeling and Assimilation Office (GMAO). GEOS-Chem has undergone three major chemical updates in the last year. Its latest standard chemical mechanism includes state-of-the-science OxNOx–volatile organic compound (VOC)–halogen–aerosol reactions. In addition, GEOS-Chem offers a number of specialty simulations to address a variety of scientific questions, such as simulations of CO2 (Nassar et al.2010), CO (Fisher et al.2017), methane (Maasakkers et al.2019), mercury (Horowitz et al.2017; Soerensen et al.2010), persistent organic pollutants (Friedman et al.2013), and dicarbonyls (Fu et al.2008, 2009; Cao et al.2018).

Despite their updated representation of chemical processes and relative ease of use, offline models have several key shortcomings. First, the applications of some offline models are limited by the time span and resolution of the available meteorological data. In the case of the GEOS-Chem nested-grid model, its application is currently limited to 0.5 latitude ×0.625 longitude or coarser resolution between 1979 and the present day when using the Modern-Era Retrospective analysis for Research and Applications, version 2 (MERRA-2) dataset, or to 0.25 latitude ×0.3125 longitude or coarser resolution between 2013 and the present day when using the GEOS Forward Processing (GEOS-FP) dataset. The temporal interpolation of sparsely archived meteorological data can also cause significant errors in the CTM simulations (Yu et al.2018). Most importantly, offline models cannot simulate meteorology–chemistry interactions due to the lack of chemical feedback to meteorology.

In contrast, online regional atmospheric chemistry models perform integrated meteorological and chemical calculations, managed through operator splitting (Baklanov et al.2014). In this way, online models can simulate regional atmospheric chemistry at any location and time of interest, without the need for temporal interpolation of the meteorological variables. Moreover, online models have the option to include “two-way coupling” processes, i.e., the response of meteorology to gases and aerosols via interactions with radiation and cloud processes. Many studies have demonstrated the importance of two-way interactions in accurate air quality simulations (e.g., Li et al. (2011); Ding et al. (2013); J. Wang et al. (2014)). One of the most widely used online regional models is WRF-Chem, with options for either one-way or two-way coupling (Grell et al.2005; Fast et al.2006). The latest version of WRF-Chem (v4.1) includes many options for OxNOx–VOC–aerosol chemistry. WRF-Chem simulates the two-way interactions between chemistry and meteorology by taking into account the scattering and absorption of radiation by gases and aerosols, as well as the activation of aerosols as cloud condensation nuclei and ice nuclei (Fast et al.2006; Gustafson et al.2007; Chapman et al.2009).

However, keeping the representation of atmospheric processes up to date is potentially more challenging for online models than it is for offline models. One of the reasons for this is that the interactions between the chemical and meteorological modules are hard wired in some online models, such that updating either module requires considerable effort. For the same reason, if users make improvements to the chemical or meteorological processes in the online model, those improvements may be relatively difficult to propagate to the broader community. This may lead to the model diverging into different branches, and users may be forced to work with stale, branched versions of the code.

In this work, we developed a new online regional atmospheric chemistry model, WRF-GC, by coupling the WRF meteorology model with the GEOS-Chem chemistry model. Both WRF and GEOS-Chem are open source and actively developed by the community. We constructed WRF-GC with the following guidelines, in order to best take advantage of new developments in the two parent models and to enhance usability:

  1. The coupling structure of WRF-GC should be abstracted from the parent models, and both parent models remain unmodified from their respective sources. In this way, future updates of the parent models can be quickly incorporated into WRF-GC with ease, such that WRF-GC can stay cutting edge. It also enables WRF-GC users to more easily contribute their developments back to the parent models.

  2. The WRF-GC coupled model should scale from conventional computation hardware to massively parallel computation architectures.

  3. The WRF-GC coupled model should be easy to install and use, open source, version controlled, and well documented.

WRF-GC offers users of WRF-Chem or other regional models the option to use the latest GEOS-Chem chemical module, which is actively developed by a large international user base, well documented, traceable, benchmarked, and centrally managed. Through WRF-GC, regional modellers also gain access to the specialty simulations in GEOS-Chem, such as the simulations of mercury (Horowitz et al.2017; Soerensen et al.2010) and persistent organic pollutants (Friedman et al.2013). WRF-GC drives GEOS-Chem with online meteorological fields simulated by WRF, which in turn can be driven by initial and boundary meteorological conditions from many different assimilated datasets or climate model outputs (Skamarock et al.2008, 2019). As such, WRF-GC allows GEOS-Chem users to perform high-resolution simulations in both forecast and hindcast modes at any location and time of interest.

In this Part 1 paper, we describe the development of the WRF-GC model (v1.0, https://doi.org/10.5281/zenodo.3550330, Lin et al.2019) for simulation over a single domain with one-way coupling capability. The nested domain and two-way coupling capabilities are under development and will be described in a forthcoming paper (Feng et al.2020).

2 The parent models: WRF and GEOS-Chem

2.1 The WRF model

Meteorological processes and advection of atmospheric constituents in the WRF-GC coupled model are simulated by the WRF model (v3.9.1.1 or later versions), a mesoscale numerical weather model for research and operational applications (Skamarock et al.2008, 2019). WRF offers its users many options for model configurations and physical schemes. WRF uses the Advanced Research WRF (ARW) dynamical solver, which solves fully compressible, Eulerian non-hydrostatic equations on either hybrid sigma–eta (default) or terrain-following vertical coordinates defined by the user. Horizontal grids in WRF are staggered Arakawa C grids, which can be configured by the user using four map projections: latitude–longitude, Lambert conformal, Mercator, and polar stereographic. WRF supports the use of multiple nested domains to simulate the interactions between large-scale dynamics and mesoscale meteorology. WRF supports grid, spectral, and observational nudging (Liu et al.2005, 2006; Stauffer and Seaman1990, 1994). This allows the WRF model to produce meteorological outputs that mimic assimilated meteorological fields for use in air quality hindcasts. The WRF model offers many options for land surface physics, planetary boundary layer physics, radiative transfer, cloud microphysics, and cumulus parameterization, for use in meteorological studies, real-time numerical weather prediction, idealized simulations, and data assimilation on meso- to regional scales (Skamarock et al.2008, 2019). Table 3 lists the configuration and physical options supported by WRF-GC v1.0. In particular, only the hybrid sigma–eta vertical coordinate is currently supported in WRF-GC.

Table 1Meteorological variables needed for GEOS-Chem.

Download XLSX

The WRF model incorporates a highly modular software framework that is portable across a range of computing platforms. WRF supports two-level domain decomposition for distributed-memory (Message Passing Interface; MPI) and shared-memory (OpenMP) parallel computation. Distributed parallelism is implemented through the Runtime System Library lite (RSL-lite) module, which supports irregular domain decomposition, automatic index translation, distributed input/output, and low-level interfacing with MPI libraries (Michalakes et al.1999).

2.2 The GEOS-Chem model

Our development of WRF-GC was made possible by a recent structural overhaul of GEOS-Chem (Long et al.2015; Eastham et al.2018), which enabled the use of GEOS-Chem as a self-contained chemical module within the WRF-GC model. The original GEOS-Chem CTM (prior to v11.01) was structured specifically for several static sets of global or regional 3-D grids at prescribed horizontal and vertical resolutions (Bey et al.2001). Parallelism for the original GEOS-Chem was implemented through OpenMP, such that the original GEOS-Chem can only operate on single-node hardware with large shared memory. Long et al. (2015) restructured the core processes in GEOS-Chem, including emission, chemistry, convective mixing, planetary boundary layer mixing, and deposition processes, to work in modular units of atmospheric vertical columns. Information about the horizontal grids, formerly fixed at compile time, is now passed to the GEOS-Chem chemical module at runtime. This development enabled the use of the GEOS-Chem chemical module with any horizontal grid structure and horizontal resolution.

The new, modularized structure of the GEOS-Chem has been implemented in two types of configurations. The first type of configuration uses GEOS-Chem as the core of offline CTMs. For example, in the GEOS-Chem “Classic” implementation (GCC), the GEOS-Chem chemical module is driven by the GEOS assimilated meteorological data and is parallelized using OpenMP. This implementation treats the prescribed global or regional model domains as contiguous sets of atmospheric columns placed at regular latitude/longitude intervals, with vertical layers pre-defined to match those of the GEOS model. In essence, the GCC implementation mimics the “original” GEOS-Chem model before the structural overhaul by Long et al. (2015). Other grid systems can also be used with the GEOS-Chem chemical module. For example, the GEOS-Chem High Performance (GCHP) implementation (Eastham et al.2018) calls the GEOS-Chem chemical module on the native cubed-sphere coordinates of the NASA GEOS model via a column interface in GEOS-Chem (GIGC_Chunk_Run). This column interface was built on the Earth System Modeling Framework (ESMF) (Eastham et al.2018) and permits runtime specification of the horizontal grid parameters. The GCHP implementation uses MPI to parallelize GEOS-Chem across nodes through the Model Analysis and Prediction Layer framework (MAPL) (Suarez et al.2007), which is a wrapper on top of ESMF specifically designed for the NASA GEOS model system.

Alternatively, GEOS-Chem can be used as a module coupled to weather models or Earth system models to perform online chemical calculations. Using this capability, Hu et al. (2018) developed an online implementation of GEOS-Chem by coupling it to the NASA GEOS-5 model to simulate global atmospheric chemistry. Lu et al. (2019) coupled GEOS-Chem to the Beijing Climate Center Atmospheric General Circulation Model (BCC-AGCM). However, both the NASA GEOS-5 model and the BCC-AGCM are proprietary.

WRF-GC is the first implementation that couples the GEOS-Chem chemical module to an open-access high-resolution meteorological model. We developed a modular coupler between WRF and GEOS-Chem that draws from the technology of GCHP but does not rely on ESMF (described in Sect. 3.2). We also made changes to GEOS-Chem to accept arbitrary vertical discretization from WRF at runtime and to improve physical compatibility with WRF (described in Sect. 3.2.1). These changes have been incorporated into the mainline GEOS-Chem code. Our coupler and code modifications can be adapted in the future to couple GEOS-Chem to other non-ESMF Earth system models.

Chemical calculations in WRF-GC v1.0 use GEOS-Chem version 12.2.1 (https://doi.org/10.5281/zenodo.2580198, The International GEOS-Chem Community2019). The standard chemical mechanism in GEOS-Chem v12.2.1, used by default in WRF-GC, includes detailed OxNOx–VOC–ozone–halogen–aerosol in the troposphere, as well as the unified tropospheric–stratospheric chemistry extension (UCX) (Eastham et al.2014) for stratospheric chemistry and stratosphere–troposphere exchange. The standard gas-phase mechanism includes 208 chemical species and 981 reactions. Reactions and rates follow the latest recommendations from the Jet Propulsion Laboratory and the International Union of Pure and Applied Chemistry. In addition, GEOS-Chem uses the FlexChem pre-processor (a wrapper for the Kinetic PreProcessor, KPP; Damian et al.2002; Sandu and Sander2006) to configure chemical kinetics (Long et al.2015). This allows users to add or modify gaseous species and reactions to develop custom mechanisms and diagnostic quantities in GEOS-Chem. GEOS-Chem also supports the optional “Tropchem” (troposphere-only chemistry) mechanism, where UCX is disabled and replaced by a parameterized linear chemistry in the stratosphere (McLinden et al.2000). GEOS-Chem uses the Rosenbrock Rodas3 solver (Sandu et al.1997) for gas-phase chemistry by default, but users may choose other solvers through KPP.

Aerosol species in GEOS-Chem include secondary inorganic aerosols (sulfate, nitrate, ammonium), elemental carbon aerosol (EC), primary organic carbon (POC), secondary organic aerosol (SOA), dust, and sea salt. By default, secondary inorganic aerosols, EC, POC, and SOA are simulated as speciated bulk masses. Dust aerosols are represented in 4 size bins (0.1–1.0, 1.0–1.8, 1.8–3.0, and 3.0–6.0 µm) (Fairlie et al.2007), while sea salt aerosols are represented in two size bins (0.1–0.5 and 0.5–4.0 µm) (Jaeglé et al.2011). The thermodynamics of secondary inorganic aerosol are coupled to gas-phase chemistry and computed by the ISORROPIA II module (Park et al.2004; Fountoukis and Nenes2007; Pye et al.2009). EC and POC are represented in GEOS-Chem as partially hydrophobic and partially hydrophilic, with a conversion timescale from hydrophobic to hydrophilic of 1.2 d (Q. Wang et al.2014). The organic matter to organic carbon (OM  OC) mass ratio is assumed to be 2.1 for POC by default, with an option to use seasonally and spatially varying OM  OC ratios (Philip et al.2014). GEOS-Chem has two options to describe the production of SOA, and both options are supported in WRF-GC. By default, SOA is produced irreversibly using simple yields from anthropogenic and biogenic volatile organic precursors (Kim et al.2015). Alternatively, GEOS-Chem can simulate SOA production via the aging of semi-volatile and intermediate volatility organic precursors using a volatility basis set (VBS) scheme (Robinson et al.2007; Pye et al.2010), as well as via the aqueous reactions of the oxidation products from isoprene (Marais et al.2016). The GEOS-Chem model also has the option of simulating detailed, size-dependent aerosol microphysics using the TwO-Moment Aerosol Sectional microphysics (TOMAS) module (Kodros and Pierce2017) or the Advanced Particle Microphysics (APM) module (Yu and Luo2009), but these two modules are not yet supported in WRF-GC.

Emissions of chemical species in WRF-GC are calculated using the Harvard-NASA Emissions Component (HEMCO) in GEOS-Chem (Keller et al.2014). HEMCO allows users to select emission inventories from the HEMCO data directory or add their own inventories, and interpolate the emission fluxes to the model domain and resolution at runtime. The HEMCO data directory currently includes more than 20 global and regional emission inventories, mostly at their respective native resolutions (http://wiki.seas.harvard.edu/geos-chem/index.php/HEMCO_data_directories, last access: 24 April 2020). By default, the Community Emissions Data System (CEDS) inventory (0.5×0.5 resolution, monthly) (Hoesly et al.2018) is used for most of the world; over Asia and the US, the CEDS is superseded by the MIX inventory (0.25×0.25 native resolution, monthly; Li et al.2017b) and the 2011 National Emission Inventory (NEI 2011) (0.1 km×0.1 km native resolution, hourly; U.S. Environmental Protection Agency2014), respectively. HEMCO also has extensions to compute emissions with meteorological dependencies, such as the emissions of biogenic species (Guenther et al.2012), soil NOx (Hudman et al.2012), lightning NOx, sea salt (Gong2003), and dust (Zender et al.2003). With the exception of lightning NOx, these meteorology-dependent emissions are supported in WRF-GC v1.0. Further details about the use of HEMCO in WRF-GC are given in Sect. 3.3.1.

Other physical calculations in GEOS-Chem are coupled to WRF meteorological fields in WRF-GC; we describe the coupling in detail in Sect. 3.3. Convective transport of chemical species is calculated using a single-plume parameterization (Allen et al.1996; Wu et al.2007), which is in turn driven by the cumulus parameterization in WRF. Boundary layer mixing is calculated using a non-local scheme, driven by the WRF-simulated atmospheric instability and boundary layer height (Lin and McElroy2010). Dry deposition is based on a resistance-in-series scheme (Wesely1989; Wang et al.1998). Aerosol deposition is as described in Zhang et al. (2001), with updates to account for size dependency for dust (Fairlie et al.2007) and sea salt (Alexander et al.2005; Jaeglé et al.2011). Wet scavenging of gases and water-soluble aerosols is as described in Liu et al. (2001) and Amos et al. (2012) and driven by WRF-simulated precipitation rates.

3 Description of the WRF-GC coupled model

3.1 Overview of the WRF-GC model architecture

Figure 1 gives an architectural overview of the WRF-GC coupled model. Our development of WRF-GC uses many of the existing infrastructure in the WRF-Chem model that couples WRF to its chemistry module (Grell et al.2005). The subroutine calls between WRF and the chemistry components are exactly the same in WRF-GC and in WRF-Chem. Within WRF-GC, the WRF model and the GEOS-Chem model remain entirely intact. The WRF-GC coupler consists of interfaces with the two parent models, as well as a state conversion module and a state management module. The coupler is separate from both parent models and is written in a manner similar to an application programming interface, enabling independent updates of the parent models within WRF-GC.

https://gmd.copernicus.org/articles/13/3241/2020/gmd-13-3241-2020-f01

Figure 1Architectural overview of the WRF-GC model (v1.0). The WRF-GC coupler (all parts shown in red) includes interfaces to the two parent models, as well as the state conversion and state management modules. The parent models (shown in grey) are standard codes downloaded from their sources, without any modifications.

Download

The WRF-GC model is initialized and driven by WRF, which sets up the simulation domain, establishes the global clock, handles the input and output of data, sets the initial and boundary conditions (IC/BC) for meteorological and chemical variables, and manages cross-processor communication for parallelization. Users configure the WRF-GC model structure in the WRF configuration file (namelist.input), including the domain, projection, horizontal resolution, vertical coordinates, simulation time, dynamic time step, external chemical time step, and other physical and dynamical options. Users also “turn on” GEOS-Chem in WRF-GC by specifying chem_opt = 233 in namelist.input, similar to the way that users specify the chemical mechanism in WRF-Chem. GEOS-Chem is initialized by the WRF model using the WRF-to-chemistry interface described in Sect. 3.2.3. Chemical options within GEOS-Chem, including the choice of standard or custom chemical mechanisms, emission inventories in HEMCO, and diagnostic quantities to be output, are defined by users in the GEOS-Chem configuration files (input.geos, HEMCO_Config.rc, and HISTORY.rc). IC/BC for meteorological and chemical variables are prepared by the user in NetCDF format and read by WRF. Meteorological IC/BC can be prepared using the WRF pre-processor system (WPS) from datasets available from NCAR's Research Data Archive (https://rda.ucar.edu, last access: 24 April 2020). IC/BC of chemical species concentrations are taken from GEOS-Chem Classic global model outputs and interpolated to the WRF-GC model grids using a modified version of the WRF-Chem pre-processor tool mozbc (available along with the WRF-GC code).

Dynamical and physical calculations are performed in WRF-GC exactly as they are in the WRF model. WRF also calculates the grid-scale advection of chemical species. At the beginning of each chemical time step, WRF calls the WRF-GC chemistry component through the WRF-to-chemistry interface, chem_driver, which is a chemical driver similar to that in WRF-Chem. Spatial parameters and the internal state of WRF are translated at runtime to GEOS-Chem by the state conversion and management modules. The GEOS-Chem chemical module then performs convective transport, dry deposition, wet scavenging, emission calculations, boundary layer mixing, and chemistry calculations. This operator splitting between WRF and GEOS-Chem is identical to that between WRF and the chemistry module in WRF-Chem. Then, the GEOS-Chem internal state is translated back to WRF, and the WRF time stepping continues. At the end of the WRF-GC simulation, WRF outputs all meteorological and chemical diagnostic quantities in WRF's standard format.

WRF-GC v1.0 supports all the existing input and output functionality of the WRF model, including serial/parallel reading and writing of NetCDF, HDF5, and GRIB2 datasets. WRF and WRF-Chem users can use existing data pre- and post-processing tools to prepare input data and analyze results from WRF-GC.

3.2 Details about the WRF-GC coupler technology

3.2.1 Further modularization of GEOS-Chem for WRF-GC coupling

Long et al. (2015) restructured the GEOS-Chem model into modular units of atmospheric columns. However, there were limitations in that column structure and its interface which prohibit the coupling with WRF. First, the GEOS-Chem module developed by Long et al. (2015) was hard-coded to operate on prescribed configurations of either 72 or 47 vertical levels. The former configuration was designed to match the native vertical levels of the GEOS model. The latter configuration was designed to match the lumped vertical levels often used by the GEOS-Chem Classic model. Second, the column interface to the GEOS-Chem module as implemented in GCHP depends on the ESMF and MAPL frameworks, which WRF does not support.

We modified the GEOS-Chem module and interface to facilitate more flexible coupling with WRF and other dynamical models. We allowed GEOS-Chem to accept the Ap and Bp parameters for the hybrid sigma–eta vertical grids and the local tropopause level from WRF at runtime. Stratospheric chemistry is only calculated in GEOS-Chem above the tropopause level passed from WRF. Also, 3-D emissions (such as the injection of biomass burning plumes into the free troposphere) are interpolated in HEMCO to the WRF-GC vertical levels.

In addition, we modified the existing GCHP interface GIGC_Chunk_Run to remove its dependencies on ESMF and MAPL when running in WRF-GC. We added a set of compatible error-handling and state management components to GEOS-Chem that interacts with the WRF through the chemistry driver. These new components replace the functionalities originally provided by ESMF. This removes all dependency of the WRF-GC coupler and the GEOS-Chem column interface on external frameworks.

All of our changes adhere to the GEOS-Chem coding and documentation standards and have been fully merged into the GEOS-Chem standard source code as of version 12.0.0 (https://doi.org/10.5281/zenodo.1343547, The International GEOS-Chem Community2019) and are controlled with the pre-processor switch MODEL_WRF at compile time. In the future, these changes will be maintained as part of the standard GEOS-Chem model.

3.2.2 Installation and compilation process

From the user's standpoint, the installation and compilation of WRF-GC is very similar to the procedures for WRF-Chem. WRF-GC is installed by downloading the parent models, WRF and GEOS-Chem, and the WRF-GC coupler, directly from their respective software repositories. The WRF model is installed in a top-level directory, while the WRF-GC coupler and GEOS-Chem are installed under the chem/ subdirectory, where the chemistry routines for WRF-Chem originally reside. An unmodified copy of the GEOS-Chem code is installed in the chem/gc/ subdirectory, and a set of sample GEOS-Chem configuration files is in chem/config/. The WRF meteorology model remains unmodified in WRF-GC, but at present the chemical routines of WRF-Chem cannot work alongside GEOS-Chem under a single WRF-GC top directory.

The standard WRF model includes built-in compile routines for coupling with the WRF-Chem chemistry module. WRF-GC uses these existing compile routines by substituting the parts pertinent to WRF-Chem with a generic chemistry interface. This substitution process is self-contained in the WRF-GC coupler and requires no manual changes to the WRF code. As such, the installation and compilation of WRF-GC require no extra maintenance effort from the WRF developers, and WRF-GC operates as a drop-in chemical module to WRF.

When the user sets the environment variable WRF_CHEM to 1 in the WRF compile script, WRF reads a registry file (registry.chem) containing the GEOS-Chem chemical species information (duplicated from input.geos) and builds these species into the WRF model framework. The WRF compile script then calls the Makefile in the chem/ subdirectory to compile an unmodified copy of GEOS-Chem (located in chem/gc/) with the pre-processor switch MODEL_WRF. This compiles GEOS-Chem into two libraries, which can be called by WRF. The first GEOS-Chem library (libGeosCore.a) contains all GEOS-Chem core routines. The second GEOS-Chem library (libGIGC.a) contains the GEOS-Chem column interface (GIGC_Chunk_Mod). The subsequent compilation process links these GEOS-Chem libraries and the WRF-to-chemistry interface to the rest of the WRF code, creating a single WRF-GC executable (wrf.exe).

3.2.3 Runtime processes

In WRF-Chem, WRF calls its interface to chemistry, chem_driver, which then calls each individual chemical processes. We abstracted this chem_driver interface by removing direct calls to chemical processes. Instead, our chem_driver calls the WRF-GC state conversion module (WRFGC_Convert_State_Mod) and the GEOS-Chem column interface (GIGC_Chunk_Run) to perform chemical calculations. We also modified chemics_init to initializes GEOS-Chem through the column interface GIGC_Chunk_Init.

The WRF-GC state conversion module includes two subroutines. The WRFGC_Get_WRF subroutine receives meteorological data and spatial information from WRF and translates them into GEOS-Chem formats and units. Table 1 summarizes the WRF meteorological variables required to drive GEOS-Chem. Many meteorological variables in WRF only require a conversion of units before they are passed to GEOS-Chem. Some meteorological variables require physics-based diagnosis in the WRFGC_Get_WRF subroutine before being passed to GEOS-Chem. Horizontal grid coordinates and resolutions are passed to GEOS-Chem in the form of latitudes and longitudes at the center and edges of each grid. Vertical coordinates are passed from WRF to GEOS-Chem at runtime as described in Sect. 3.2.1. A second subroutine, WRFGC_Set_WRF, receives chemical species concentrations from GEOS-Chem, converts the units, and saves them in the chemical variable array in WRF.

The WRF-GC state management module (GC_Stateful_Mod) manages the GEOS-Chem internal state in distributed memory, such that GEOS-Chem can run in the MPI-parallel architecture. When running WRF-GC in the distributed-memory configuration, WRF decomposes the horizontal computational domain evenly across the available computational cores at the beginning of runtime. Each computational core has access only to its allocated subset of the full domain as a set of atmospheric columns, plus a halo of columns around that subset domain. The halo columns are used for inter-core communication of grid-aware processes, such as horizontal transport (Skamarock et al.2008). The internal states of GEOS-Chem for each core are managed by the state management module; they are distributed at initialization and independent from each other. The WRF-GC state management module is also critical to the development of nested-grid simulations in the future.

3.3 Treatment of key processes in the WRF-GC coupled model

Below, we describe the operator splitting between WRF and GEOS-Chem within WRF-GC, as well as the treatments of some of the key processes. The general Eulerian form of the continuity equation for m chemical species with number density vector n=(n1,,nm)T is

(1) n i t = - ( n i U ) + P i ( n ) + L i ( n ) i [ 1 , m ] .

U is the wind vector, which is provided by the WRF model in WRF-GC. The first term on the right-hand side of Eq. (1) indicates the transport of species i, which includes grid-scale advection, as well as subgrid turbulent mixing and convective transport. Pi(n) and Li(n) are the local production and loss rates of species i, respectively (Long et al.2015).

In WRF-GC, WRF simulates the meteorological variables using the dynamic equations and meteorological IC/BC. These meteorological variables are then passed to the GEOS-Chem chemical module (Table 1) to solve the local production and loss terms of the continuity equation. Large-scale (grid-scale) advection of chemical species is grid-aware and is calculated by the WRF dynamical core. Local (subgrid) vertical transport processes, including turbulent mixing within the boundary layer and convective transport from the surface to the convective cloud top, are calculated in GEOS-Chem using WRF-simulated meteorology. Dry deposition and wet scavenging of chemical species are also calculated in GEOS-Chem and driven by WRF. This operator-splitting arrangement is identical to that in the WRF-Chem model.

The dynamic and chemical time steps are specified by the user in the WRF configuration file namelist.input. The dynamic time step is constrained by the Courant–Friedrichs–Lewy stability criterion and should be short for high-resolution simulations. WRF-Chem recommends that the chemical time step be set the same as the dynamic time step as best practice (Peckham et al.2017). Because GEOS-Chem uses a Rosenbrock solver, which adapts its internal chemical time step to the stiffness of the chemical mechanism, a larger chemical time step may be used. However, it is recommended that the results be compared to a control simulation with the chemical time step set to the dynamic time step (Peckham et al.2017).

3.3.1 Emission of chemical species

Chemical emissions in WRF-GC are calculated online by the HEMCO module in GEOS-Chem (Keller et al.2014) and configured in HEMCO_Config.rc. HEMCO and its data directory are updated as part of the GEOS-Chem model and remain unmodified in WRF-GC. Users can choose to use one or combine several of the emission inventories already in the HEMCO data directory (Sect. 2.2). Some of the inventories currently available in the HEMCO data directory may not be of sufficiently fine resolution to support the high-resolution WRF-GC simulations. In that case, users can prepare their own emission input files in NetCDF format at arbitrary spatiotemporal resolutions, and HEMCO will interpolate them to the WRF-GC model domain and resolution at runtime. HEMCO also allow users to specify scale factors and diurnal/weekly/monthly variation profiles in HEMCO_Config.rc to be applied to the emission fluxes at runtime. WRF-GC calls HEMCO to compute meteorology-dependent emissions online using WRF-simulated meteorology. These currently include the emissions of biogenic species (Guenther et al.2012), soil NOx (Hudman et al.2012), sea salt (Gong2003), and dust (Zender et al.2003). Lightning NOx emissions are not yet supported in WRF-GC v1.0 but will be added in a future version.

3.3.2 Subgrid vertical transport of chemical species

Subgrid vertical transport of chemical species in WRF-GC, including convective transport and boundary layer mixing, are calculated by GEOS-Chem using WRF meteorology. GEOS-Chem uses the convective mass flux variable to drive convective transport (Allen et al.1996; Wu et al.2007). This variable is calculated in WRF using the user's choice of cumulus parameterization, but its value is not stored in the WRF meteorological variable array. The WRF-GC state conversion module re-diagnoses convective mass fluxes using the user's choice of parameterization in WRF and then pass the values to GEOS-Chem. The WRF-GC state conversion module currently supports convective mass flux calculations using the new Tiedtke scheme (Tiedtke1989; Zhang et al.2011; Zhang and Wang2017) and the Zhang–McFarlane scheme (Zhang and McFarlane1995) (Table 1), because these two cumulus parameterization schemes are more physically compatible with the convective transport algorithm currently in GEOS-Chem. In addition, the users should consider the horizontal resolution of the model when choosing which cumulus parameterization to use. The new Tiedtke scheme and the Zhang–McFarlane schemes are generally recommended for use in simulations at horizontal resolutions larger than 10 km (Skamarock et al.2008; Arakawa and Jung2011). At horizontal resolutions between 2 and 10 km, the so-called “convective grey zone” (Jeworrek et al.2019), the use of the Grell–Freitas scheme is recommended for the WRF model (Grell and Freitas2014), as it allows subsidence to spread to neighboring columns; this option will be implemented in a future WRF-GC version. At horizontal resolutions finer than 2 km, it is assumed that convections are resolved and cumulus parameterizations should not be used (Grell and Freitas2014; Jeworrek et al.2019). The scale dependence of cumulus parameterizations and their impacts on convective mixing of chemical species are an active area of research, which we will explore in the future using WRF-GC.

Boundary layer mixing is calculated in GEOS-Chem using a non-local scheme (Holtslag and Boville1993; Lin and McElroy2010). The boundary layer height, thermodynamic variables, and the vertical level and pressure information are calculated by WRF and passed to GEOS-Chem through the state conversion module. Again, this methodology is the same as that in the WRF-Chem model.

3.3.3 Dry deposition and wet scavenging of chemical species

Dry deposition is calculated in GEOS-Chem using a resistance-in-series scheme (Wesely1989; Wang et al.1998). The land cover data for the simulated domain are read by and used in WRF, but for now WRF-GC only supports the use of the US Geological Survey (USGS) classification. The land cover information is passed to GEOS-Chem, where it is mapped to the land cover classifications of Olson et al. (2001) to assign values of surface roughness and canopy resistance (Wang et al.1998). The dry deposition velocities are calculated locally using WRF-simulated surface air momentum, sensible heat fluxes, temperature, and solar radiation.

To calculate the wet scavenging of chemical species, the WRF-GC coupler diagnoses the WRF-simulated precipitation variables using the microphysical scheme and cumulus parameterization scheme selected by the user (Table 1). The precipitation variables passed to GEOS-Chem include large-scale/convective precipitation production rates, large-scale/convective precipitation evaporation rates, and the downward fluxes of large-scale and convective ice/liquid precipitation. The microphysical schemes currently supported in WRF-GC include the Morrison two-moment scheme (Morrison et al.2009), the CAM5.1 scheme (Neale et al.2012), the WSM6 scheme (Hong and Lim2006), and the Thompson scheme (Thompson et al.2008).

4 Application: surface PM2.5 over China during 22 to 27 January 2015

We simulated surface PM2.5 concentrations over China during a severe haze event in January 2015 using both the WRF-GC model v1.0 (WRF v3.9.1.1 and GEOS-Chem v12.2.1) and the GEOS-Chem Classic model (v12.2.1) in its nested-grid configuration for China. We compared the simulated results from the two models against each other, as well as against surface measurements. Both the WRF-GC and the GEOS-Chem Classic nested-China simulations were conducted from 18 to 27 January 2015; the first 4 d initialized the models. Results from 22 to 27 January 2015 were analyzed. Our goal was to compare the performance of the two models in simulating Chinese surface PM2.5 under their normal mode of operation. To this end, the two simulations were configured as similarly as possible, but there are important innate differences between the two models, as described below.

4.1 Setup of the WRF-GC model and the GEOS-Chem model

Figure 2a shows the domain of the GEOS-Chem Classic nested-China simulation, which was driven by the GEOS-FP assimilated meteorological dataset at its native horizontal resolution of 0.25 latitude ×0.3125 longitude. The vertical resolution of the GEOS-FP dataset was reduced from its native 72 levels to 47 levels by lumping levels in the stratosphere. This treatment is standard in GEOS-Chem Classic simulations. The 47 vertical layers extended from the surface to 0.01 hPa, with seven levels in the bottom 1 km. The top edge of the bottom layer was at approximately 120 m over eastern China. Meteorological variables were updated every 3 h (every hour for surface variables). IC/BC of chemical species concentrations were taken from the outputs of a global GEOS-Chem Classic simulation and updated at the boundaries of the nested-China domain every 3 h. The dynamic time step and the external chemistry time step were 5 and 10 min, respectively.

https://gmd.copernicus.org/articles/13/3241/2020/gmd-13-3241-2020-f02

Figure 2Comparison of the simulated (filled contours) 6 d average PM2.5 concentrations during 22 to 27 January 2015 from (a) the GEOS-Chem Classic nested-China simulation and (b) the WRF-GC nudged simulation. Also shown are the observed 6 d average PM2.5 concentrations during this period at 578 surface sites managed by the Ministry of Ecology and Environment of China.

Figure 2b shows the WRF-GC simulation domain, with a horizontal resolution of 27 km×27 km. We chose this domain and horizontal resolution to be comparable to those of the GEOS-Chem Classic nested-grid simulation. There were 50 vertical levels in our WRF-GC simulation, which extended from the surface up to 10 hPa with 7 levels below 1 km. The top edge of the bottom layer was at approximately 60 m over eastern China. The WRF model does not have the option of using the GEOS-FP dataset for meteorological IC/BC. Instead, we used the NCEP FNL dataset (https://doi.org/10.5065/D6M043C6, National Centers for Environmental Prediction et al.2000) at 1×1 resolution as IC/BC for WRF-GC; the FNL dataset was interpolated to WRF vertical levels and updated every 6 h. In addition, we nudged the WRF-simulated meteorological fields with surface (every 3 h) and upper air (every 6 h) observations of temperature, specific humidity, and winds from the NCEP ADP Global Surface/Upper Air Observational Weather Database (https://doi.org/10.5065/39C5-Z211, Satellite Services Division et al.2004). This mimicked the effect of meteorological data assimilation and allowed the WRF-simulated meteorology to stay close to the observed states of the atmosphere. IC/BC of chemical species concentrations were identical to those used in the GEOS-Chem Classic nested-China simulation but interpolated to WRF vertical levels and updated every 6 h. The dynamic time step and the external chemistry time step were 2 and 10 min, respectively. Other physical options used in our WRF-GC simulation are summarized in Table 2.

(Morrison et al.2009)(Iacono et al.2008)(Iacono et al.2008)(Jimenez et al.2012)(Chen and Dudhia2001a, b)(Nakanishi and Niino2006)(Tiedtke1989; Zhang et al.2011; Zhang and Wang2017)

Table 2WRF-GC physics configuration.

Download Print Version | Download XLSX

Our WRF-GC and GEOS-Chem Classic simulations used the exact same GEOS-Chem chemical mechanism for gases and aerosols, including a total of 241 chemical species. Emissions in the two simulations were both calculated by HEMCO and were identical for anthropogenic and biomass burning sources. Monthly mean anthropogenic emissions from China were from the Multi-resolution Emission Inventory for China (MEIC; Li et al.2014) at 0.25×0.25 horizontal resolution. The MEIC inventory was updated for the year 2015 and included emissions from power generation, industry, transportation, and residential activities. Sector-specific weekly and diurnal variations from the MEIC inventory were applied (Li et al.2017a). Agricultural emissions of ammonia were from Huang et al. (2012). Anthropogenic emissions from the rest of the Asia were from Li et al. (2017b), developed for the year 2010. Monthly mean biomass burning emissions were taken from Global Fire Emissions Database version 4 (GFED4) (Randerson et al.2018). Emissions of biogenic species (Guenther et al.2012), soil NOx (Hudman et al.2012), sea salt (Gong2003), and dust (Zender et al.2003) in the two simulations were coupled to model meteorology in HEMCO and thus different between the two models. PM2.5 mass concentrations were diagnosed for both simulations as the sum of masses of sulfate, nitrate, ammonium, EC, primary organic aerosol (assumed to be 2.1 times the primary organic carbon mass), SOA, fine dust (100 % of dust between 0 and 0.7 µm and 38 % of dust between 0.7 and 1.4 µm), and accumulation-mode sea salt, taking into consideration the hygroscopic growth for each species at 35 % relative humidity.

Table 3List of WRF configuration and physical options supported in WRF-GC v1.0.

Download Print Version | Download XLSX

4.2 Validation of the WRF-GC simulation against surface measurements and comparison with the GEOS-Chem Classic simulation

Figure 2 compares the 6 d average surface PM2.5 concentrations during 22 to 27 January 2015 as in the WRF-GC and the GEOS-Chem Classic nested-China simulations, respectively. Also shown are the PM2.5 concentrations measured at 578 surface sites, managed by the Ministry of Ecology and Environment of China (http://www.cnemc.cn, last access: 24 April 2020). We removed invalid hourly surface PM2.5 observations following the protocol in Jiang et al. (2020). The 578 sites were selected by (1) removing surface sites with less than 80 % valid hourly measurements during our simulation period, and (2) sampling the site closest to the model grid center, if that model grid contained multiple surface sites. Both models reproduced the general spatial distributions of the observed PM2.5 concentrations, including the high concentrations over eastern China, as well as the hotspots over the North China Plan, central China, and the Sichuan Basin. However, both models overestimated the PM2.5 concentrations over eastern China. The mean 6 d PM2.5 concentrations averaged for the 578 sites as simulated by WRF-GC and by GEOS-Chem Classic nested-China were 117±68 and 120±76µg m−3, respectively. In comparison, the observed mean 6 d PM2.5 concentration averaged for the 578 sites was 98±43µg m−3.

https://gmd.copernicus.org/articles/13/3241/2020/gmd-13-3241-2020-f03

Figure 3Scatter plots of observed and simulated daily mean PM2.5 during 22 to 27 January 2015 at 507 surface sites over eastern China for (a) the GEOS-Chem Classic nested-China simulation and (b) the WRF-GC nudged simulation. The solid lines indicate the reduced major-axis regression lines, with slopes, intercepts, and correlation coefficients (r) shown in the inset. The dotted lines indicate the 1:1 lines.

Download

Figure 3 shows the scatter plots of the simulated and observed daily average PM2.5 concentrations over eastern China (eastward of 103 E, 507 sites) during 22 to 27 January 2015. We focused on eastern China, because the spatiotemporal variability of PM2.5 concentrations was higher over this region. Again, both models overestimated the daily PM2.5 concentrations over eastern China. The daily PM2.5 concentrations simulated by WRF-GC were 29 % higher than the observations (quantified by the reduced major-axis regression slope between the simulated and observed daily PM2.5 concentration), with a correlation coefficient of r=0.68. The daily PM2.5 concentrations in the GEOS-Chem Classic nested-China simulation were 55 % higher than the observations, with a correlation coefficient of r=0.72.

https://gmd.copernicus.org/articles/13/3241/2020/gmd-13-3241-2020-f04

Figure 4Taylor diagrams of hourly PM2.5 concentrations during 22 to 27 January 2015 from the GEOS-Chem Classic nested-China simulation (black symbols) and the WRF-GC simulation (red symbols) for (a) 13 cities in the Beijing–Tianjin–Hebei area, (b) 22 cities in the Yangtze River Delta area, and (c) 13 other major Chinese cities. Dashed green, black, and blue lines indicate contours of the normalized centered root mean square differences (RMSDs), the ratios of simulated versus observed standard deviation, and the Pearson correlation coefficients, respectively.

Download

Figure 4 shows the Taylor diagrams of the hourly PM2.5 concentrations simulated by the two models at 48 major eastern Chinese cities, including 13 cities in the Beijing–Tianjin–Hebei (BTH) area, 22 cities in the Yangtze River Delta (YRD) area, and 13 other major cities. The Taylor diagram (Taylor2001) evaluates the simulated time series of PM2.5 against the observations, using the Pearson correlation coefficients, the ratio between the simulated and observed standard deviations (σsimσobs), and the normalized root mean square differences (RMSDs) as metrics. Proximity to the point “1” on the x axis in Fig. 4 indicates that the simulation accurately reproduced both the mean concentration and the temporal variability of the observations. For most cities in the BTH area and for most of the other 13 major cities, the hourly PM2.5 concentrations simulated by WRF-GC showed smaller RMSDs and higher correlation coefficients against observations, compared to those in the GEOS-Chem Classic nested-China simulation. In the YRD area, the performance of the two models was similar.

Our analyses above show that the hourly and daily surface PM2.5 concentrations simulated by the WRF-GC model were in better agreement with observations than those simulated by the GEOS-Chem Classic nested-China model over eastern China during 22 to 27 January 2015. We found that this was partially because the WRF-GC model, nudged with surface and upper-air meteorological observations, better represented the pollution meteorology, compared to the GEOS-FP dataset that was used to drive the GEOS-Chem Classic nested-China simulation. Figure S1 shows the average surface air temperature, relative humidity, and 10 m wind speed as simulated by the WRF-GC model and as provided by the GEOS-FP dataset against the observations during 22 and 27 January 2015 at 367 sites over China. The surface air temperature simulated by WRF-GC and those in the GEOS-FP dataset were both in good agreement with the observations over China. However, the relative humidity and wind speeds simulated by WRF-GC were more consistent with the observations, compared to those in the GEOS-FP dataset. Figure S2 assesses the hourly surface air temperature, relative humidity, and near-surface winds simulated by the WRF-GC model and those in the GEOS-FP assimilated meteorological dataset, against hourly surface measurements over China during 22–27 January 2015. For the 34 sites with publicly available hourly measurements, the meteorological fields simulated by the WRF-GC were generally more consistent with the measurements.

Figure 5 shows the mean planetary boundary layer height (PBLH) at 08:00 LT (00:00 UTC) and 20:00 LT (12:00 UTC) during 22 to 27 January 2015 in the GEOS-Chem Classic nested-China and the WRF-GC simulations, respectively, and compares them with the rawinsonde observations during this period (Guo et al.2016). The PBLH in the GEOS-Chem Classic model was taken from the GEOS-FP dataset, whereas the boundary layer height was simulated by WRF in WRF-GC. Compared to the observations, the PBLH in the GEOS-FP dataset were generally biased low over eastern China and biased high over the mountainous areas in southwestern China and western China. This likely was a major reason for the severe overestimation of surface PM2.5 concentrations in the GEOS-Chem Classic nested-China simulation over eastern China. In comparison, the WRF-GC model correctly represented the PBLH over most regions in China, which was critical to its more accurate simulation of surface PM2.5 concentrations.

https://gmd.copernicus.org/articles/13/3241/2020/gmd-13-3241-2020-f05

Figure 5Comparison of the simulated (fill contours) and observed (fill symbols) planetary boundary layer heights (PBLHs) at 08:00 LT (a, b) and 20:00 LT (c, d) averaged between 22 and 27 January 2015. (a, c) PBLH from the GEOS-FP dataset, which was used to drive the GEOS-Chem Classic nested-China simulation, and (b, d) PBLH simulated by the WRF-GC model.

5 Computational performance and scalability of WRF-GC

5.1 Computational performance of the WRF-GC model

We evaluated the computational performance of a WRF-GC simulation and compared it with that of the GEOS-Chem Classic nested-grid simulation of a similar configuration. We configured the WRF-GC and GEOS-Chem Classic nested-grid simulations for the exact same domain (as shown in Fig. 2a), with the exact same projection and horizontal resolution (0.25 latitude ×0.3125 longitude resolution, 225×161 atmospheric columns). The GEOS-Chem Classic nested-grid simulation had 47 vertical levels, and the WRF-GC simulation comparably had 50 levels. Both simulations used the same emissions and chemical configurations with a total of 241 chemical species and 10 min external chemical time steps. WRF-GC calculated meteorology online with a 2 min dynamic time step. The GEOS-Chem Classic nested-grid simulation calculated transport (5 min time step) driven by archived GEOS-FP meteorological data read from an ethernet-connected hard disk array. WRF-GC used MPI parallelization, while GEOS-Chem Classic used OpenMP. Both simulations were conducted for 48 h with scheduled output for every 1 h. Both simulations were executed on the same single-node hardware with 32 Intel Broadwell physical cores and an ethernet-connected hard disk array.

Figure 6 compares the computational wall times for the WRF-GC and the GEOS-Chem Classic nested-grid simulations. The total wall time for the WRF-GC simulation was 5127 s, only 31 % of the wall time for the GEOS-Chem Classic nested-grid simulation (16 391 s). We found that the difference in computational efficiency was mainly due to the much faster dynamic and transport calculations in the WRF-GC model relative to the transport calculation in the GEOS-Chem Classic nested-grid model. In WRF-GC, the wall time taken up by the entire WRF (including transport, physics, I/O, and model initiation) was 2462.5 s. In the GEOS-Chem Classic nested-grid simulation, 50 % (8192 s) of the total wall time was used for the transport of tracers, including large-scale advection (6355.7 s), convective transport (694.2 s), and boundary layer mixing (1142.5 s). As a CTM, the GEOS-Chem Classic read archived meteorological data for the entire domain at 3-model-hour intervals from hard drives using a single computational core, which becomes increasingly burdensome for simulations with more grid boxes. In comparison, WRF-GC calculated meteorology online in node memory and updated the model boundary conditions from hard drives every 6 model hours. Finally, the OpenMP parallelization in GEOS-Chem Classic was only at the loop level. In contrast, WRF-GC used MPI parallelization to decompose domain at the model level, thus parallelizing all code within the GEOS-Chem module. As a result, the same chemistry routines actually ran faster in WRF-GC than they did in GEOS-Chem Classic. In WRF-GC, the chemistry routines (deposition, emissions computation, convection, boundary layer mixing, and chemistry) took up 2462.5 s in total; the same routines took up 5263.5 s in GEOS-Chem Classic. The WRF-GC coupler consumed negligible wall time (39 s) in this test simulation.

https://gmd.copernicus.org/articles/13/3241/2020/gmd-13-3241-2020-f06

Figure 6Comparison of wall time for the WRF-GC model (v1.0) and the GEOS-Chem Classic nested-grid model (v12.2.1).

Download

A side-by-side wall time comparison between WRF-GC and WRF-Chem is difficult to do, because (1) the chemical routines in the two models are very different, and (2) WRF-Chem has many possible configurations for chemistry. Nevertheless, we conducted one test simulation with WRF-Chem using a typical chemistry option (CBMZ-MOSAIC gas-phase chemistry, four-bin aerosol microphysics and chemistry, and aqueous reactions; chem_opt = 9 in registry.chem) with a total of 133 chemical species. This WRF-Chem simulation was configured for the same domain, at the same horizontal and vertical resolutions, and used the same physical and dynamical options as those in the WRF-GC simulation described above. The total wall time for the WRF-Chem simulation was 9985.8 s, which was almost twice as long as the wall time of WRF-GC (5127 s). Chemical routines in this WRF-Chem test simulation took up 61 % of the total wall time (6134.3 s), despite the WRF-Chem having much fewer chemical species than the WRF-GC (241 chemical species). This may be partially due to the computationally intensive bin-resolved aerosol microphysics calculation in the WRF-Chem simulation.

5.2 Scalability of the WRF-GC model

We analyzed the scalability of the WRF-GC model using the 48 h simulation described in Sect. 5.1. The model was compiled using the Intel C and Fortran compilers (v16.0.3) and the mvapich2 (v2.3) MPI library. The computing environment (Tianhe-1A) had 28 Intel Broadwell physical cores with 125 GB of RAM per node. Input and output used a networked Lustre high-performance file system.

https://gmd.copernicus.org/articles/13/3241/2020/gmd-13-3241-2020-f07

Figure 7WRF-GC model scalability by processes. Grey lines indicate perfect scalability, i.e. halved computational time for each doubling of processor cores.

Download

Figure 7 shows the scalability of the WRF-GC simulation in terms of the total WRF-GC wall time, as well as the wall times of its three components: (1) the WRF model (including input/output), (2) the GEOS-Chem model, and (3) the WRF-GC coupler. For the domain of this test simulation, the total wall time and the WRF wall time both scaled well up to 136 cores. This was because the simulation domain was too fragmented above 136 cores, such that the MPI communication time took up much of the runtime, resulting in performance degradation. Chemical calculations in the GEOS-Chem model were perfectly scalable up to the largest number of cores tested (250 cores), consistent with previous GCHP performance analyses (Eastham et al.2018). Figure 7 also shows that the WRF-GC coupler scaled nearly perfectly and consumed less than 1 % of the total WRF-GC wall time. At above 200 cores there was a slight degradation of the scalability due to cross-core communications at the subdomain boundaries. However, the degradation had negligible impact on the total WRF-GC wall time as the WRF-GC coupler was computationally efficient.

WRF-GC also scales to massively parallel architectures and can be deployed on the cloud, because both the WRF and GEOS-Chem model are already operational on the cloud with their input data readily available (Hacker et al.2017; Zhuang et al.2019). We conducted a test simulation running WRF-GC on the Amazon Web Services (AWS) cloud with up to 128 nodes and 4608 cores. The simulation domain was over the continental US at 5 km×5 km resolution (950×650 atmospheric columns), with a 10 s dynamical time step and 5 min chemical time step. The scalability test results are shown in Fig. S3. In this massively parallel environment, WRF-GC scaled well up to 1728 cores, with the chemical module scaling well up to 2304 cores. The WRF-GC coupler took less than 0.2 % of the total computational time in this simulation and scaled perfectly up to 4608 cores. The deployability of WRF-GC on the cloud will enhance WRF-GC's accessibility to new users by saving them the investment in hardware purchases and the effort in downloading and hosting large input datasets locally.

6 Conclusions

We developed the WRF-GC model, which is an online coupling of the WRF meteorological model and the GEOS-Chem chemical model, to simulate regional atmospheric chemistry at high resolution, with high computational efficiency, and underpinned by the latest scientific understanding of atmospheric processes. By design, the WRF-GC model is structured to work with native copies of the parent models and involves no hard-wired code to either parent model. This allows the WRF-GC model to integrate future updates of either parent model with relative ease.

WRF-GC provides current users of WRF-Chem and other regional models with access to GEOS-Chem, which is state of the science, well documented, traceable, benchmarked, actively developed by a large international community, and centrally managed. At the same time, WRF-GC enables GEOS-Chem users to perform high-resolution regional chemistry simulations in both forecast and hindcast mode at any location and time of interest, with high performance.

Our first application showed that the WRF-GC model was able to reproduce the spatiotemporal variation of surface PM2.5 concentrations over China in January 2015, with smaller biases compared to the results of the GEOS-Chem Classic nested-China simulation. This was partially because the WRF-GC model better represented the pollution meteorology, including the variability of the planetary boundary layer heights, over the region. In addition, the WRF-GC simulation was 3 times faster than a comparable GEOS-Chem Classic nested-grid simulation.

WRF-GC demonstrated good scalability to massively parallel architectures, with near-perfect scalability of its chemistry component. This enables the WRF-GC model to be used on multiple-node systems or high-performance cloud computing platforms, which is not possible with the GEOS-Chem Classic. The GCHP model also scales to massively parallel architectures (Zhuang et al.2020), but GCHP can only operate as a global model. The deployability of WRF-GC on the cloud will enhance WRF-GC's accessibility to new users.

The WRF-GC coupling structure, including the GEOS-Chem column interface and the state conversion module, are extensible and can be adapted to models other than WRF. This opens up possibilities of coupling GEOS-Chem to other weather and Earth system models in an online, modular manner. Using native, out-of-the-box copies of parent models in coupled models reduces maintenance and avoids branching of the parent model code. It also enables the community to more easily transfer developments in the parent models to the coupled model, and vice versa.

The WRF-GC model is free and open source to all users. The one-way coupled version of WRF-GC (v1.0) is now publicly available at https://wrf.geos-chem.org (last access: 10 July 2020). A two-way coupled version with chemical feedbacks from GEOS-Chem to WRF is under development and will be presented in a forthcoming paper. Further development of WRF-GC will aim to enable nested-domain simulations, support size-dependent aerosol microphysical calculations, as well as further improve the physical compatibility with WRF. We envision WRF-GC to become a powerful tool for research, forecast, and regulatory applications of regional atmospheric chemistry and air quality.

Appendix A: Abbreviations
Abbreviation Description
ARW Advanced Research WRF (dynamical core)
BC Boundary condition
CCN Cloud condensation nuclei
CTM Chemical transport model
EC Elemental carbon
ESMF Earth System Modeling Framework
GCC GEOS-Chem Classic
GCHP GEOS-Chem High Performance
GCM General circulation model
GDAS Global Data Assimilation System
GEOS Goddard Earth Observing System
GEOS-FP GEOS Forward Processing
GMAO NASA Global Modeling and Assimilation Office
HEMCO Harvard-NASA Emissions Component
IC Initial condition
KPP Kinetic PreProcessor
MAPL Model Analysis and Prediction Layer
MERRA-2 Modern-Era Retrospective analysis for Research and Applications, Version 2
MMM Mesoscale and Microscale Meteorology Laboratory, NCAR
MPI Message Passing Interface
NCAR National Center of Atmospheric Research
NCEP National Centers for Environmental Prediction
NWP Numerical weather prediction
PBLH Planetary boundary layer height
POC Primary organic carbon
SOA Secondary organic aerosol
WRF Weather Research and Forecasting model
WRF-Chem Weather Research and Forecasting model coupled with chemistry
UCX Unified Chemistry Extension
VBS Volatility basis set
Code availability

WRF-GC is free and open source and can be obtained at http://wrf.geos-chem.org (Lin et al.2019). The version of WRF-GC (v1.0) described in this paper coupled WRF v3.9.1.1 with GEOS-Chem v12.2.1 and is permanently archived at https://github.com/jimmielin/wrf-gc-pt1-paper-code (last access: 24 April 2020) (https://doi.org/10.5281/zenodo.3550330; Lin et al.2019). The two parent models, WRF and GEOS-Chem, are also open source and can be obtained from their developers at https://www.mmm.ucar.edu/weather-research-and-forecasting-model (Skamarock et al.2008) and http://www.geos-chem.org (The International GEOS-Chem Community2019), respectively.

Supplement

The supplement related to this article is available online at: https://doi.org/10.5194/gmd-13-3241-2020-supplement.

Author contributions

TMF envisioned and oversaw the project. HL designed the WRF-GC coupler. HL, XF, and HT developed the WRF-GC code, with assistance from YM and LJZ. XF, HL, and TMF performed the simulations and wrote the manuscript. HL performed the scalability and performance analysis. RMY, MPS, EWL, JZ, DJJ, XL, SDE, and CAK assisted in the adaptation of the GEOS-Chem model and the HEMCO module to WRF-GC. QZ provided the MEIC emissions inventory for China. XL, LZ, and LS prepared the MEIC emissions for GEOS-Chem. JG provided the boundary layer height observations. All authors contributed to the manuscript.

Competing interests

The authors declare that they have no conflict of interest.

Acknowledgements

This project was supported by the National Natural Sciences Foundation of China (grant no. 41975158). Computational resources used in this work were partially supported by Center for Computational Science and Engineering of Southern University of Science and Technology. GEOS-FP data were provided by the Global Modeling and Assimilation Office (GMAO) at NASA Goddard Space Flight Center. We gratefully acknowledge the developers of WRF for making the model free and in the public domain.

Financial support

This research has been supported by the National Natural Science Foundation of China (grant no. 41975158).

Review statement

This paper was edited by David Topping and reviewed by two anonymous referees.

References

Alexander, B., Park, R. J., Jacob, D. J., Li, Q., Yantosca, R. M., Savarino, J., Lee, C., and Thiemens, M.: Sulfate formation in sea-salt aerosols: Constraints from oxygen isotopes, J. Geophys. Res.-Atmos., 110, D10307, https://doi.org/10.1029/2004JD005659, 2005. a

Allen, D. J., Rood, R. B., Thompson, A. M., and Hudson, R. D.: Three-dimensional radon 222 calculations using assimilated meteorological data and a convective mixing algorithm, J. Geophys. Res.-Atmos., 101, 6871–6881, https://doi.org/10.1029/95JD03408, 1996. a, b

Amos, H. M., Jacob, D. J., Holmes, C. D., Fisher, J. A., Wang, Q., Yantosca, R. M., Corbitt, E. S., Galarneau, E., Rutter, A. P., Gustin, M. S., Steffen, A., Schauer, J. J., Graydon, J. A., Louis, V. L. St., Talbot, R. W., Edgerton, E. S., Zhang, Y., and Sunderland, E. M.: Gas-particle partitioning of atmospheric Hg(II) and its effect on global mercury deposition, Atmos. Chem. Phys., 12, 591–603, https://doi.org/10.5194/acp-12-591-2012, 2012. a

Arakawa, A. and Jung, J. H.: Multiscale modeling of the moist-convective atmosphere – A review, Atmos. Res., 102, 263–285, https://doi.org/10.1016/j.atmosres.2011.08.009, 2011. a

Baklanov, A., Schlünzen, K., Suppan, P., Baldasano, J., Brunner, D., Aksoyoglu, S., Carmichael, G., Douros, J., Flemming, J., Forkel, R., Galmarini, S., Gauss, M., Grell, G., Hirtl, M., Joffre, S., Jorba, O., Kaas, E., Kaasik, M., Kallos, G., Kong, X., Korsholm, U., Kurganskiy, A., Kushta, J., Lohmann, U., Mahura, A., Manders-Groot, A., Maurizi, A., Moussiopoulos, N., Rao, S. T., Savage, N., Seigneur, C., Sokhi, R. S., Solazzo, E., Solomos, S., Sørensen, B., Tsegas, G., Vignati, E., Vogel, B., and Zhang, Y.: Online coupled regional meteorology chemistry models in Europe: current status and prospects, Atmos. Chem. Phys., 14, 317–398, https://doi.org/10.5194/acp-14-317-2014, 2014. a, b

Bey, I., Jacob, D. J., Yantosca, R. M., Logan, J. A., Field, B. D., Fiore, A. M., Li, Q., Liu, H. Y., Mickley, L. J., and Schultz, M. G.: Global modeling of tropospheric chemistry with assimilated meteorology: Model description and evaluation, J. Geophys. Res.-Atmos., 106, 23073–23095, https://doi.org/10.1029/2001JD000807, 2001. a, b

Cao, H., Fu, T.-M., Zhang, L., Henze, D. K., Miller, C. C., Lerot, C., Abad, G. G., De Smedt, I., Zhang, Q., van Roozendael, M., Hendrick, F., Chance, K., Li, J., Zheng, J., and Zhao, Y.: Adjoint inversion of Chinese non-methane volatile organic compound emissions using space-based observations of formaldehyde and glyoxal, Atmos. Chem. Phys., 18, 15017–15046, https://doi.org/10.5194/acp-18-15017-2018, 2018. a

Chapman, E. G., Gustafson Jr., W. I., Easter, R. C., Barnard, J. C., Ghan, S. J., Pekour, M. S., and Fast, J. D.: Coupling aerosol-cloud-radiative processes in the WRF-Chem model: Investigating the radiative impact of elevated point sources, Atmos. Chem. Phys., 9, 945–964, https://doi.org/10.5194/acp-9-945-2009, 2009. a

Chen, D., Wang, Y., McElroy, M. B., He, K., Yantosca, R. M., and Le Sager, P.: Regional CO pollution and export in China simulated by the high-resolution nested-grid GEOS-Chem model, Atmos. Chem. Phys., 9, 3825–3839, https://doi.org/10.5194/acp-9-3825-2009, 2009. a

Chen, F. and Dudhia, J.: Coupling an advanced land surface-hydrology model with the Penn State-NCAR MM5 modeling system. Part I: Model implementation and sensitivity, Mon. Weather Rev., 129, 569–585, https://doi.org/10.1175/1520-0493(2001)129<0569:CAALSH>2.0.CO;2, 2001a. a

Chen, F. and Dudhia, J.: Coupling an advanced land surface-hydrology model with the Penn State-NCAR MM5 modeling system. Part II: Preliminary model validation, Mon. Weather Rev., 129, 587–604, https://doi.org/10.1175/1520-0493(2001)129<0587:CAALSH>2.0.CO;2, 2001b. a

Damian, V., Sandu, A., Damian, M., Potra, F., and Carmichael, G. R.: The kinetic preprocessor KPP-a software environment for solving chemical kinetics, Comput. Chem. Eng., 26, 1567–1579, https://doi.org/10.1016/S0098-1354(02)00128-X, 2002. a

Ding, A. J., Fu, C. B., Yang, X. Q., Sun, J. N., Petäjä, T., Kerminen, V.-M., Wang, T., Xie, Y., Herrmann, E., Zheng, L. F., Nie, W., Liu, Q., Wei, X. L., and Kulmala, M.: Intense atmospheric pollution modifies weather: a case of mixed biomass burning with fossil fuel combustion pollution in eastern China, Atmos. Chem. Phys., 13, 10545–10554, https://doi.org/10.5194/acp-13-10545-2013, 2013. a

Eastham, S. D., Weisenstein, D. K., and Barrett, S. R.: Development and evaluation of the unified tropospheric–stratospheric chemistry extension (UCX) for the global chemistry-transport model GEOS-Chem, Atmos. Environ., 89, 52–63, https://doi.org/10.1016/j.atmosenv.2014.02.001, 2014. a

Eastham, S. D., Long, M. S., Keller, C. A., Lundgren, E., Yantosca, R. M., Zhuang, J., Li, C., Lee, C. J., Yannetti, M., Auer, B. M., Clune, T. L., Kouatchou, J., Putman, W. M., Thompson, M. A., Trayanov, A. L., Molod, A. M., Martin, R. V., and Jacob, D. J.: GEOS-Chem High Performance (GCHP v11-02c): a next-generation implementation of the GEOS-Chem chemical transport model for massively parallel applications, Geosci. Model Dev., 11, 2941–2953, https://doi.org/10.5194/gmd-11-2941-2018, 2018. a, b, c, d

Fairlie, T. D., Jacob, D. J., and Park, R. J.: The impact of transpacific transport of mineral dust in the United States, Atmos. Environ., 41, 1251–1266, https://doi.org/10.1016/j.atmosenv.2006.09.048, 2007. a, b

Fast, J. D., Gustafson Jr., W. I., Easter, R. C., Zaveri, R. A., Barnard, J. C., Chapman, E. G., Grell, G. A., and Peckham, S. E.: Evolution of ozone, particulates, and aerosol direct radiative forcing in the vicinity of Houston using a fully coupled meteorology-chemistry-aerosol model, J. Geophys. Res.-Atmos., 111, D21305, https://doi.org/10.1029/2005JD006721, 2006. a, b

Feng, X., Lin, H., and Fu, T.-M.: WRF-GC: online two-way coupling of WRF and GEOS-Chem for regional atmospheric chemistry modeling, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-5165, https://doi.org/10.5194/egusphere-egu2020-5165, 2020. a

Fisher, J. A., Murray, L. T., Jones, D. B. A., and Deutscher, N. M.: Improved method for linear carbon monoxide simulation and source attribution in atmospheric chemistry models illustrated using GEOS-Chem v9, Geosci. Model Dev., 10, 4129–4144, https://doi.org/10.5194/gmd-10-4129-2017, 2017. a

Fountoukis, C. and Nenes, A.: ISORROPIA II: a computationally efficient thermodynamic equilibrium model for K+Ca2+Mg2+NH4+Na+SO42-NO3-ClH2O aerosols, Atmos. Chem. Phys., 7, 4639–4659, https://doi.org/10.5194/acp-7-4639-2007, 2007. a

Friedman, C. L., Zhang, Y., and Selin, N. E.: Climate change and emissions impacts on atmospheric PAH transport to the Arctic, Environ. Sci. Technol., 48, 429–437, https://doi.org/10.1021/es403098w, 2013. a, b

Fu, T.-M., Jacob, D. J., Wittrock, F., Burrows, J. P., Vrekoussis, M., and Henze, D. K.: Global budgets of atmospheric glyoxal and methylglyoxal, and implications for formation of secondary organic aerosols, J. Geophys. Res-Atmos., 113, D15303, https://doi.org/10.1029/2007JD009505, 2008. a

Fu, T.-M., Jacob, D. J., and Heald, C. L.: Aqueous-phase reactive uptake of dicarbonyls as a source of organic aerosol over eastern North America, Atmos. Environ., 43, 1814–1822, https://doi.org/10.1016/j.atmosenv.2008.12.029, 2009. a

Gong, S. L.: A parameterization of sea-salt aerosol source function for sub-and super-micron particles, Global Biogeochem. Cy., 17, 1097, https://doi.org/10.1029/2003GB002079, 2003. a, b, c

Grell, G. A. and Freitas, S. R.: A scale and aerosol aware stochastic convective parameterization for weather and air quality modeling, Atmos. Chem. Phys., 14, 5233–5250, https://doi.org/10.5194/acp-14-5233-2014, 2014. a, b

Grell, G. A., Peckham, S. E., Schmitz, R., McKeen, S. A., Frost, G., Skamarock, W. C., and Eder, B.: Fully coupled “online” chemistry within the WRF model, Atmos. Environ., 39, 6957–6975, https://doi.org/10.1016/j.atmosenv.2005.04.027, 2005. a, b

Guenther, A. B., Jiang, X., Heald, C. L., Sakulyanontvittaya, T., Duhl, T., Emmons, L. K., and Wang, X.: The Model of Emissions of Gases and Aerosols from Nature version 2.1 (MEGAN2.1): an extended and updated framework for modeling biogenic emissions, Geosci. Model Dev., 5, 1471–1492, https://doi.org/10.5194/gmd-5-1471-2012, 2012. a, b, c

Guo, J., Miao, Y., Zhang, Y., Liu, H., Li, Z., Zhang, W., He, J., Lou, M., Yan, Y., Bian, L., and Zhai, P.: The climatology of planetary boundary layer height in China derived from radiosonde and reanalysis data, Atmos. Chem. Phys., 16, 13309–13319, https://doi.org/10.5194/acp-16-13309-2016, 2016. a

Gustafson, Jr., W. I., Chapman, E. G., Ghan, S. J., Easter, R. C., and Fast, J. D.: Impact on modeled cloud characteristics due to simplified treatment of uniform cloud condensation nuclei during NEAQS 2004, Geophys. Res. Lett., 34, 19809, https://doi.org/10.1029/2007GL030021, 2007. a

Hacker, J. P., Exby, J., Gill, D., Jimenez, I., Maltzahn, C., See, T., Mullendore, G., and Fossell, K.: A containerized mesoscale model and analysis toolkit to accelerate classroom learning, collaborative research, and uncertainty quantification, B. Am. Meteorol. Soc., 98, 1129–1138, https://doi.org/10.1175/BAMS-D-15-00255.1, 2017. a

Hoesly, R. M., Smith, S. J., Feng, L., Klimont, Z., Janssens-Maenhout, G., Pitkanen, T., Seibert, J. J., Vu, L., Andres, R. J., Bolt, R. M., Bond, T. C., Dawidowski, L., Kholod, N., Kurokawa, J.-I., Li, M., Liu, L., Lu, Z., Moura, M. C. P., O'Rourke, P. R., and Zhang, Q.: Historical (1750–2014) anthropogenic emissions of reactive gases and aerosols from the Community Emissions Data System (CEDS), Geosci. Model Dev., 11, 369–408, https://doi.org/10.5194/gmd-11-369-2018, 2018. a

Holtslag, A. and Boville, B.: Local Versus Nonlocal Boundary-Layer Diffusion in a Global Climate Model, J. Climate, 6, 1825–1842, https://doi.org/10.1175/1520-0442(1993)006<1825:LVNBLD>2.0.CO;2, 1993. a

Hong, S.-Y. and Lim, J.-O. J.: The WRF single-moment 6-class microphysics scheme (WSM6), J. Korean Meteor. Soc., 42, 129–151, 2006. a

Horowitz, H. M., Jacob, D. J., Zhang, Y., Dibble, T. S., Slemr, F., Amos, H. M., Schmidt, J. A., Corbitt, E. S., Marais, E. A., and Sunderland, E. M.: A new mechanism for atmospheric mercury redox chemistry: implications for the global mercury budget, Atmos. Chem. Phys., 17, 6353–6371, https://doi.org/10.5194/acp-17-6353-2017, 2017. a, b

Hu, L., Keller, C. A., Long, M. S., Sherwen, T., Auer, B., Da Silva, A., Nielsen, J. E., Pawson, S., Thompson, M. A., Trayanov, A. L., Travis, K. R., Grange, S. K., Evans, M. J., and Jacob, D. J.: Global simulation of tropospheric chemistry at 12.5 km resolution: performance and evaluation of the GEOS-Chem chemical module (v10-1) within the NASA GEOS Earth system model (GEOS-5 ESM), Geosci. Model Dev., 11, 4603–4620, https://doi.org/10.5194/gmd-11-4603-2018, 2018. a

Huang, X., Song, Y., Li, M., Li, J., Huo, Q., Cai, X., Zhu, T., Hu, M., and Zhang, H.: A high-resolution ammonia emission inventory in China, Global Biogeochem. Cy., 26, GB1030, https://doi.org/10.1029/2011GB004161, 2012. a

Hudman, R. C., Moore, N. E., Mebust, A. K., Martin, R. V., Russell, A. R., Valin, L. C., and Cohen, R. C.: Steps towards a mechanistic model of global soil nitric oxide emissions: implementation and space based-constraints, Atmos. Chem. Phys., 12, 7779–7795, https://doi.org/10.5194/acp-12-7779-2012, 2012. a, b, c

Iacono, M. J., Delamere, J. S., Mlawer, E. J., Shephard, M. W., Clough, S. A., and Collins, W. D.: Radiative forcing by long-lived greenhouse gases: Calculations with the AER radiative transfer models, J. Geophys. Res.-Atmos., 113, D13103, https://doi.org/10.1029/2008JD009944, 2008. a, b

Jaeglé, L., Quinn, P. K., Bates, T. S., Alexander, B., and Lin, J.-T.: Global distribution of sea salt aerosols: new constraints from in situ and remote sensing observations, Atmos. Chem. Phys., 11, 3137–3157, https://doi.org/10.5194/acp-11-3137-2011, 2011. a, b

Jeworrek, J., West, G., and Stull, R.: Evaluation of Cumulus and Microphysics Parameterizations in WRF across the Convective Gray Zone, Weather Forecast., 34, 1097–1115, https://doi.org/10.1175/WAF-D-18-0178.1, 2019. a, b

Jiang, Z., Jolleys, M. D., Fu, T.-M., Palmer, P. I., Ma, Y. P., Tian, H., Li, J., and Yang, X.: Spatiotemporal and probability variations of surface PM2.5 over China between 2013 and 2019 and the associated changes in health risks: An integrative observation and model analysis, Sci. Total. Environ., 723, 137896, https://doi.org/10.1016/j.scitotenv.2020.137896, 2020. a

Jimenez, P. A., Dudhia, J., Gonzalez-Rouco, J. F., Navarro, J., Montavez, J. P., and Garcia-Bustamante, E.: A Revised Scheme for the WRF Surface Layer Formulation, Mon. Weather Rev., 140, 898–918, https://doi.org/10.1175/MWR-D-11-00056.1, 2012. a

Keller, C. A., Long, M. S., Yantosca, R. M., Da Silva, A. M., Pawson, S., and Jacob, D. J.: HEMCO v1.0: a versatile, ESMF-compliant component for calculating emissions in atmospheric models, Geosci. Model Dev., 7, 1409–1417, https://doi.org/10.5194/gmd-7-1409-2014, 2014. a, b

Kim, P. S., Jacob, D. J., Fisher, J. A., Travis, K., Yu, K., Zhu, L., Yantosca, R. M., Sulprizio, M. P., Jimenez, J. L., Campuzano-Jost, P., Froyd, K. D., Liao, J., Hair, J. W., Fenn, M. A., Butler, C. F., Wagner, N. L., Gordon, T. D., Welti, A., Wennberg, P. O., Crounse, J. D., St. Clair, J. M., Teng, A. P., Millet, D. B., Schwarz, J. P., Markovic, M. Z., and Perring, A. E.: Sources, seasonality, and trends of southeast US aerosol: an integrated analysis of surface, aircraft, and satellite observations with the GEOS-Chem chemical transport model, Atmos. Chem. Phys., 15, 10411–10433, https://doi.org/10.5194/acp-15-10411-2015, 2015. a

Kodros, J. and Pierce, J.: Important global and regional differences in aerosol cloud-albedo effect estimates between simulations with and without prognostic aerosol microphysics, J. Geophys. Res.-Atmos., 122, 4003–4018, https://doi.org/10.1002/2016JD025886, 2017. a

Li, M., Zhang, Q., Streets, D. G., He, K. B., Cheng, Y. F., Emmons, L. K., Huo, H., Kang, S. C., Lu, Z., Shao, M., Su, H., Yu, X., and Zhang, Y.: Mapping Asian anthropogenic emissions of non-methane volatile organic compounds to multiple chemical mechanisms, Atmos. Chem. Phys., 14, 5617–5638, https://doi.org/10.5194/acp-14-5617-2014, 2014. a

Li, M., Liu, H., Geng, G., Hong, C., Liu, F., Song, Y., Tong, D., Zheng, B., Cui, H., Man, H., Zhang, Q., and He, K.: Anthropogenic emission inventories in China: a review, Natl. Sci. Rev., 4, 834–866, https://doi.org/10.1093/nsr/nwx150, 2017a. a

Li, M., Zhang, Q., Kurokawa, J.-I., Woo, J.-H., He, K., Lu, Z., Ohara, T., Song, Y., Streets, D. G., Carmichael, G. R., Cheng, Y., Hong, C., Huo, H., Jiang, X., Kang, S., Liu, F., Su, H., and Zheng, B.: MIX: a mosaic Asian anthropogenic emission inventory under the international collaboration framework of the MICS-Asia and HTAP, Atmos. Chem. Phys., 17, 935–963, https://doi.org/10.5194/acp-17-935-2017, 2017b. a, b

Li, Z., Niu, F., Fan, J., Liu, Y., Rosenfeld, D., and Ding, Y.: Long-term impacts of aerosols on the vertical development of clouds and precipitation, Nat. Geosci., 4, 888–894, https://doi.org/10.1038/NGEO1313, 2011. a

Lin, H., Feng, X., Fu, T.-M., Tian, H., Ma, Y., Zhang, L., Jacob, D. J., Yantosca, R. M., Sulprizio, M. P., Lundgren, E. W., Zhuang, J., Zhang, Q., Lu, X., Zhang, L., Shen, L., Guo, J., Eastham, S. D., and Keller, C. A.: WRF-GC v1.0, Zenodo, https://doi.org/10.5281/zenodo.3550330, 2019. a, b, c

Lin, J.-T. and McElroy, M. B.: Impacts of boundary layer mixing on pollutant vertical profiles in the lower troposphere: Implications to satellite remote sensing, Atmos. Environ., 44, 1726–1739, 2010. a, b

Liu, H., Jacob, D. J., Bey, I., and Yantosca, R. M.: Constraints from 210Pb and 7Be on wet deposition and transport in a global three-dimensional chemical tracer model driven by assimilated meteorological fields, J. Geophys. Res.-Atmos., 106, 12109–12128, https://doi.org/10.1029/2000JD900839, 2001. a

Liu, Y., Bourgeois, A., Warner, T., Swerdlin, S., and Hacker, J.: Implementation of Observation-Nudging Based FDDA into WRF for Supporting ATEC Test Operations, 2005 WRF user workshop, Boulder, CO, 1–4, 2005. a

Liu, Y., Bourgeois, A., Warner, T., Swerdlin, S., and Yu, W.: An update on “observation nudging”-based FDDA for WRF-ARW: Verification using OSSE and performance of real-time forecasts, 2006 WRF user workshop, Boulder, CO, 1–6, 2006. a

Long, M. S., Yantosca, R., Nielsen, J. E., Keller, C. A., da Silva, A., Sulprizio, M. P., Pawson, S., and Jacob, D. J.: Development of a grid-independent GEOS-Chem chemical transport model (v9-02) as an atmospheric chemistry module for Earth system models, Geosci. Model Dev., 8, 595–602, https://doi.org/10.5194/gmd-8-595-2015, 2015. a, b, c, d, e, f, g

Lu, X., Zhang, L., Wu, T., Long, M. S., Wang, J., Jacob, D. J., Zhang, F., Zhang, J., Eastham, S. D., Hu, L., Zhu, L., Liu, X., and Wei, M.: Development of the global atmospheric general circulation-chemistry model BCC-GEOS-Chem v1.0: model description and evaluation, Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2019-240, in review, 2019. a

Maasakkers, J. D., Jacob, D. J., Sulprizio, M. P., Scarpelli, T. R., Nesser, H., Sheng, J.-X., Zhang, Y., Hersher, M., Bloom, A. A., Bowman, K. W., Worden, J. R., Janssens-Maenhout, G., and Parker, R. J.: Global distribution of methane emissions, emission trends, and OH concentrations and trends inferred from an inversion of GOSAT satellite data for 2010–2015, Atmos. Chem. Phys., 19, 7859–7881, https://doi.org/10.5194/acp-19-7859-2019, 2019. a

Marais, E. A., Jacob, D. J., Jimenez, J. L., Campuzano-Jost, P., Day, D. A., Hu, W., Krechmer, J., Zhu, L., Kim, P. S., Miller, C. C., Fisher, J. A., Travis, K., Yu, K., Hanisco, T. F., Wolfe, G. M., Arkinson, H. L., Pye, H. O. T., Froyd, K. D., Liao, J., and McNeill, V. F.: Aqueous-phase mechanism for secondary organic aerosol formation from isoprene: application to the southeast United States and co-benefit of SO2 emission controls, Atmos. Chem. Phys., 16, 1603–1618, https://doi.org/10.5194/acp-16-1603-2016, 2016. a

McLinden, C., Olsen, S., Hannegan, B., Wild, O., Prather, M., and Sundet, J.: Stratospheric ozone in 3-D models: A simple chemistry and the cross-tropopause flux, J. Geophys. Res. Atmos., 105, 14653–14665, https://doi.org/10.1029/2000JD900124, 2000. a

Michalakes, J., Dudhia, J., Gill, D., Klemp, J., and Skamarock, W.: Design of a next-generation regional weather research and forecast model, Towards Teracomputing: The Use of Parallel Processors in Meteorology, Proceedings of the Eighth ECMWF Workshop on the Use of Parallel Processors in Meteorology, edited by: Zweiflhofer, W. and Kreitz, N. World Scientific, 117–124, 1999. a

Morrison, H., Thompson, G., and Tatarskii, V.: Impact of Cloud Microphysics on the Development of Trailing Stratiform Precipitation in a Simulated Squall Line: Comparison of One- and Two-Moment Schemes, Mon. Weather Rev., 137, 991–1007, https://doi.org/10.1175/2008MWR2556.1, 2009. a, b

Nakanishi, M. and Niino, H.: An improved mellor-yamada level-3 model: Its numerical stability and application to a regional prediction of advection fog, Bound.-Lay. Meteorol., 119, 397–407, https://doi.org/10.1007/s10546-005-9030-8, 2006. a

Nassar, R., Jones, D. B. A., Suntharalingam, P., Chen, J. M., Andres, R. J., Wecht, K. J., Yantosca, R. M., Kulawik, S. S., Bowman, K. W., Worden, J. R., Machida, T., and Matsueda, H.: Modeling global atmospheric CO2 with improved emission inventories and CO2 production from the oxidation of other carbon species, Geosci. Model Dev., 3, 689–716, https://doi.org/10.5194/gmd-3-689-2010, 2010. a

Neale, R. B., Chen, C.-C., Gettelman, A., Lauritzen, P. H., Park, S., Williamson, D. L., Conley, A. J., Garcia, R., Kinnison, D., Lamarque, J.-F., Marsh, D., Mills, M., Smith, A. K., Tilmes, S., Vitt, F., Morrison, H., Cameron-Smith, P., Collins, W. D., Iacono, M. J., Easter, R. C., Ghan, S. J., Liu, X., Rasch, P. J., and Taylor, M. A.: NCAR Tech. Note NCAR/TN-486+STR: Description of the NCAR Community Atmosphere Model (CAM 5.0), 2012. a

National Centers for Environmental Prediction/National Weather Service/NOAA/U.S. Department of Commerce: NCEP FNL Operational Model Global Tropospheric Analyses, continuing from July 1999, Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory, https://doi.org/10.5065/D6M043C6, 2000. a

Olson, D. M., Dinerstein, E., Wikramanayake, E. D., Burgess, N. D., Powell, G. V. N., Underwood, E. C., D'amico, J. A., Itoua, I., Strand, H. E., Morrison, J. C., Loucks, C. J., Allnutt, T. F., Ricketts, T. H., Kura, Y., Lamoreux, J. F., Wettengel, W. W., Hedao, P., and Kassem, K. R.: Terrestrial Ecoregions of the World: A New Map of Life on Earth: A new global map of terrestrial ecoregions provides an innovative tool for conserving biodiversity, BioScience, 51, 933–938, https://doi.org/10.1641/0006-3568(2001)051[0933:TEOTWA]2.0.CO;2, 2001. a

Park, R. J., Jacob, D. J., Field, B. D., Yantosca, R. M., and Chin, M.: Natural and transboundary pollution influences on sulfate-nitrate-ammonium aerosols in the United States: Implications for policy, J. Geophys. Res.-Atmos., 109, D15204, https://doi.org/10.1029/2003JD004473, 2004. a

Peckham, S. E., Grell, G. A., McKeen, S. A., Ahmadov, R., Barth, M., Pfister, G., Wiedinmyer, C., Fast, J. D., Gustafson, W. I., Ghan, S. J., Zaveri, R., Easter, R. C., Barnard, J., Chapman, E., Hewson, M., Schmitz, R., Salzmann, M., B. V., and Freitas, S. R.: WRF-Chem Version 3.9.1.1 User's Guide, 2017. a, b

Philip, S., Martin, R. V., Pierce, J. R., Jimenez, J. L., Zhang, Q., Canagaratna, M. R., Spracklen, D. V., Nowlan, C. R., Lamsal, L. N., Cooper, M. J., and Krotkov, N. A.: Spatially and seasonally resolved estimate of the ratio of organic mass to organic carbon, Atmos. Environ., 87, 34–40, https://doi.org/10.1016/j.atmosenv.2013.11.065, 2014. a

Pye, H. O. T., Liao, H., Wu, S., Mickley, L. J., Jacob, D. J., Henze, D. K., and Seinfeld, J. H.: Effect of changes in climate and emissions on future sulfate-nitrate-ammonium aerosol levels in the United States, J. Geophys. Res.-Atmos., 114, D01205, https://doi.org/10.1029/2008JD010701, 2009. a

Pye, H. O. T., Chan, A. W. H., Barkley, M. P., and Seinfeld, J. H.: Global modeling of organic aerosol: the importance of reactive nitrogen (NOx and NO3), Atmos. Chem. Phys., 10, 11261–11276, https://doi.org/10.5194/acp-10-11261-2010, 2010. a

Randerson, J. T., van der Werf, G. R., Giglio, L., Collatz, G. J., and Kasibhatla, P. S.: Global Fire Emissions Database, Version 4, (GFEDv4), ORNL DAAC, Oak Ridge, Tennessee, USA, https://doi.org/10.3334/ORNLDAAC/1293, 2018. a

Robinson, A. L., Donahue, N. M., Shrivastava, M. K., Weitkamp, E. A., Sage, A. M., Grieshop, A. P., Lane, T. E., Pierce, J. R., and Pandis, S. N.: Rethinking organic aerosols: Semivolatile emissions and photochemical aging, Science, 315, 1259–1262, https://doi.org/10.1126/science.1133061, 2007. a

Sandu, A. and Sander, R.: Technical note: Simulating chemical systems in Fortran90 and Matlab with the Kinetic PreProcessor KPP-2.1, Atmos. Chem. Phys., 6, 187–195, https://doi.org/10.5194/acp-6-187-2006, 2006. a

Sandu, A., Verwer, J., Blom, J., Spee, E., Carmichael, G., and Potra, F.: Benchmarking stiff ode solvers for atmospheric chemistry problems II: Rosenbrock solvers, Atmos. Environ., 31, 3459–3472, https://doi.org/10.1016/S1352-2310(97)83212-8, 1997. a

Satellite Services Division/Office of Satellite Data Processing and Distribution/NESDIS/NOAA/U.S. Department of Commerce, and National Centers for Environmental Prediction/National Weather Service/NOAA/U.S. Department of Commerce, NCEP ADP Global Upper Air Observational Weather Data, October 1999 - continuing, Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory, https://doi.org/10.5065/39C5-Z211, 2004. a

Skamarock, W. C., Klemp, J. B., Dudhia, J., Gill, D. O., Barker, D. M., Duda, M. G., Huang, X.-Y., Wang, W., and Powers, J. G.: NCAR Tech. Note NCAR/TN-475+STR: A Description of the Advanced Research WRF Version 3, https://doi.org/10.5065/D68S4MVH, 2008. a, b, c, d, e, f

Skamarock, W. C., Klemp, J. B., Dudhia, J., Gill, D. O., Liu, Z., Berner, J., and Huang, X.: NCAR Tech. Note NCAR/TN-556+STR: A Description of the Advanced Research WRF Model Version 4, https://doi.org/10.5065/1dfh-6p97, 2019. a, b, c

Soerensen, A. L., Sunderland, E. M., Holmes, C. D., Jacob, D. J., Yantosca, R. M., Skov, H., Christensen, J. H., Strode, S. A., and Mason, R. P.: An improved global model for air-sea exchange of mercury: High concentrations over the North Atlantic, Environ. Sci. Technol., 44, 8574–8580, https://doi.org/10.1021/es102032g, 2010. a, b

Stauffer, D. R. and Seaman, N. L.: Use of Four-Dimensional Data Assimilation in a Limited-Area Mesoscale Model. Part I: Experiments with Synoptic-Scale Data, Mon. Weather. Rev., 118, 1250–1277, https://doi.org/10.1175/1520-0493(1990)118<1250:UOFDDA>2.0.CO;2, 1990. a

Stauffer, D. R. and Seaman, N. L.: Multiscale Four-Dimensional Data Assimilation, J. Appl. Meteorol., 33, 416–434, https://doi.org/10.1175/1520-0450(1994)033<0416:MFDDA>2.0.CO;2, 1994. a

Suarez, M., Trayanov, A., Hill, C., Schopf, P., and Vikhliaev, Y.: MAPL: a high-level programming paradigm to support more rapid and robust encoding of hierarchical trees of interacting high-performance components, in: Proceedings of the 2007 symposium on Component and framework technology in high-performance and scientific computing, ACM, 11–20, https://doi.org/10.1145/1297385.1297388, 2007. a

Taylor, K.: Summarizing multiple aspects of model performance in a single diagram., J. Geophys. Res.-Atmos., 106, 7183–7192, https://doi.org/10.1029/2000JD900719, 2001. a

The International GEOS-Chem Community: geoschem/geos-chem: GEOS-Chem 12.2.1 (Version 12.2.1), Zenodo, https://doi.org/10.5281/zenodo.2580198, 2019. a, b, c

Thompson, G., Field, P. R., Rasmussen, R. M., and Hall, W. D.: Explicit Forecasts of Winter Precipitation Using an Improved Bulk Microphysics Scheme. Part II: Implementation of a New Snow Parameterization, Mon. Weather Rev., 136, 5095–5115, https://doi.org/10.1175/2008MWR2387.1, 2008. a

Tiedtke, M.: A comprehensive mass flux scheme for cumulus parameterization in large-scale models, Mon. Weather. Rev., 117, 1779–1800, https://doi.org/10.1175/1520-0493(1989)117<1779:ACMFSF>2.0.CO;2, 1989. a, b

U.S. Environmental Protection Agency: 2011 National Emissions Inventory, version 1 Technical Support Document, 2014. a

Wang, J., Wang, S., Jiang, J., Ding, A., Zheng, M., Zhao, B., Wong, D. C., Zhou, W., Zheng, G., Wang, L., Pleim, J. E., and Hao, J.: Impact of aerosol-meteorology interactions on fine particle pollution during China's severe haze episode in January 2013, Environ. Res. Lett., 9, 094002, https://doi.org/10.1088/1748-9326/9/9/094002, 2014. a

Wang, Q., Jacob, D. J., Spackman, J. R., Perring, A. E., Schwarz, J. P., Moteki, N., Marais, E. A., Ge, C., Wang, J., and Barrett, S. R. H.: Global budget and radiative forcing of black carbon aerosol: Constraints from pole-to-pole (HIPPO) observations across the Pacific, J. Geophys. Res.-Atmos., 119, 195–206, https://doi.org/10.1002/2013JD020824, 2014. a

Wang, Y., Jacob, D. J., and Logan, J. A.: Global simulation of tropospheric O3−NOx-hydrocarbon chemistry: 1. Model formulation, J. Geophys. Res.-Atmos., 103, 10713–10725, https://doi.org/10.1029/98JD00158, 1998. a, b, c

Wang, Y. X., McElroy, M. B., Jacob, D. J., and Yantosca, R. M.: A nested grid formulation for chemical transport over Asia: Applications to CO, J. Geophys. Res.-Atmos., 109, D22307, https://doi.org/10.1029/2004JD005237, 2004. a

Wesely, M. L.: Parameterization of surface resistances to gaseous dry deposition in regional-scale numerical models, Atmos. Environ., 23, 1293–1304, https://doi.org/10.1016/0004-6981(89)90153-4, 1989. a, b

Wu, S., Mickley, L. J., Jacob, D. J., Logan, J. A., Yantosca, R. M., and Rind, D.: Why are there large differences between models in global budgets of tropospheric ozone?, J. Geophys. Res.-Atmos., 112, D05302, https://doi.org/10.1029/2006JD007801, 2007. a, b

Yu, F. and Luo, G.: Simulation of particle size distribution with a global aerosol model: contribution of nucleation to aerosol and CCN number concentrations, Atmos. Chem. Phys., 9, 7691–7710, https://doi.org/10.5194/acp-9-7691-2009, 2009. a

Yu, K., Keller, C. A., Jacob, D. J., Molod, A. M., Eastham, S. D., and Long, M. S.: Errors and improvements in the use of archived meteorological data for chemical transport modeling: an analysis using GEOS-Chem v11-01 driven by GEOS-5 meteorology, Geosci. Model Dev., 11, 305–319, https://doi.org/10.5194/gmd-11-305-2018, 2018. a

Zender, C. S., Bian, H., and Newman, D.: Mineral Dust Entrainment and Deposition (DEAD) model: Description and 1990s dust climatology, J. Geophys. Res.-Atmos., 108, 4416, https://doi.org/10.1029/2002JD002775, 2003. a, b, c

Zhang, C. and Wang, Y.: Projected future changes of tropical cyclone activity over the western North and South Pacific in a 20-km-Mesh regional climate model, J. Climate, 30, 5923–5941, https://doi.org/10.1175/JCLI-D-16-0597.1, 2017. a, b

Zhang, C., Wang, Y., and Hamilton, K.: Improved representation of boundary layer clouds over the southeast Pacific in ARW-WRF using a modified Tiedtke cumulus parameterization scheme, Mon. Weather Rev., 139, 3489–3513, https://doi.org/10.1175/MWR-D-10-05091.1, 2011. a, b

Zhang, G. J. and McFarlane, N. A.: Sensitivity of climate simulations to the parameterization of cumulus convection in the Canadian Climate Centre general circulation model, Atmos. Ocean., 33, 407–446, https://doi.org/10.1080/07055900.1995.9649539, 1995. a

Zhang, L., Gong, S., Padro, J., and Barrie, L.: A size-segregated particle dry deposition scheme for an atmospheric aerosol module, Atmos. Environ., 35, 549–560, https://doi.org/10.1016/S1352-2310(00)00326-5, 2001. a

Zhang, L., Liu, L., Zhao, Y., Gong, S., Zhang, X., Henze, D. K., Capps, S. L., Fu, T.-M., Zhang, Q., and Wang, Y.: Source attribution of particulate matter pollution over North China with the adjoint method, Environ. Res. Lett., 10, 084011, https://doi.org/10.1088/1748-9326/10/8/084011, 2015. a

Zhuang, J., Jacob, D. J., Gaya, J. F., Yantosca, R. M., Lundgren, E. W., Sulprizio, M. P., and Eastham, S. D.: Enabling immediate access to Earth science models through cloud computing: application to the GEOS-Chem model, B. Am. Meteorol. Soc., 100, 1943–1960, https://doi.org/10.1175/BAMS-D-18-0243.1, 2019. a

Zhuang, J., Jacob, D. J., Lin, H., Lundgren, E. W., Yantosca, R. M., Gaya, J. F., Sulprizio, M. P., and Eastham, S. D.: Enabling high-performance cloud computing for Earth science modeling on over a thousand cores: application to the GEOS-Chem atmospheric chemistry model, J. Adv. Model. Earth. Sy., 12, e2020MS002064, https://doi.org/10.1029/2020MS002064, 2020. a

Short summary
Online coupling of meteorology and chemistry models often presents maintenance issues with hard-wired coding. We present WRF-GC, an one-way online coupling of the WRF meteorological model and GEOS-Chem atmospheric chemistry model for regional atmospheric chemistry and air quality modeling. Our coupling structure allows future versions of either parent model to be immediately integrated into WRF-GC. The WRF-GC model was able to well reproduce regional PM2.5 with greater computational efficiency.