Global simulation of tropospheric chemistry at 12.5 km resolution: performance and evaluation of the GEOS-Chem chemical module (v10-1) within the NASA GEOS Earth system model (GEOS-5 ESM)

We present a full-year online global simulation of tropospheric chemistry (158 coupled species) at cubed-sphere c720 ($\sim \mathrm{12.5}\times \mathrm{12.5}{\mathrm{km}}^{\mathrm{2}}$) resolution in the NASA Goddard Earth Observing System Model version 5 Earth system model (GEOS-5 ESM) with GEOS-Chem as a chemical module (G5NR-chem). The GEOS-Chem module within GEOS uses the exact same code as the offline GEOS-Chem chemical transport model (CTM) developed by a large atmospheric chemistry research community. In this way, continual updates to the GEOS-Chem CTM by that community can be seamlessly passed on to the GEOS chemical module, which remains state of the science and referenceable to the latest version of GEOS-Chem. The 1-year G5NR-chem simulation was conducted to serve as the Nature Run for observing system simulation experiments (OSSEs) in support of the future geostationary satellite constellation for tropospheric chemistry. It required 31 wall-time days on 4707 compute cores with only 24 % of the time spent on the GEOS-Chem chemical module. Results from the GEOS-5 Nature Run with GEOS-Chem chemistry were shown to be consistent to the offline GEOS-Chem CTM and were further compared to global and regional observations. The simulation shows no significant global bias for tropospheric ozone relative to the Ozone Monitoring Instrument (OMI) satellite and is highly correlated with observations spatially and seasonally. It successfully captures the ozone vertical distributions measured by ozonesondes over different regions of the world, as well as observations for ozone and its precursors from the August–September 2013 Studies of Emissions, Atmospheric Composition, Clouds and Climate Coupling by Regional Surveys (SEAC⁴RS) aircraft campaign over the southeast US. It systematically overestimates surface ozone concentrations by 10 ppbv at sites in the US and Europe, a problem currently being addressed by the GEOS-Chem CTM community and from which the GEOS ESM will benefit through the seamless update of the online code.

1 Introduction

Integration of atmospheric chemistry into Earth system models (ESMs) has been identified as a next frontier for ESM development (National Research Council, 2012) and is a priority science area for atmospheric chemistry research (National Academies of Sciences, Engineering, and Medicine, 2016). Atmospheric chemistry drives climate forcing and feedbacks, is an essential component of global biogeochemical cycling, and is key to air quality applications. A growing ensemble of atmospheric chemistry observations from space needs to be integrated into ESM-based data assimilation systems. Models of atmospheric chemistry are rapidly evolving, and an atmospheric chemistry module within an ESM must be able to readily update to the state of the science. We have developed such a capability by integrating the Goddard Earth Observing System with chemistry (GEOS-Chem) chemical transport model (CTM) as a comprehensive and seamlessly updatable atmospheric chemistry module in the NASA GEOS ESM (Long et al., 2015). Here, we present the first application and evaluation of this GEOS-Chem capability within GEOS version 5 (GEOS-5) for a full-year global simulation of tropospheric ozone chemistry at cubed-sphere c720 ($\sim \mathrm{12.5}\times \mathrm{12.5}{\mathrm{km}}^{\mathrm{2}}$) resolution. This simulation is now serving as the Nature Run (pseudo-atmosphere) for observing system simulation experiments (OSSEs) in support of the near-future geostationary satellite constellation for tropospheric chemistry (Zoogman et al., 2017).

GEOS-Chem (http://geos-chem.org) is an open-source global 3-D Eulerian model of atmospheric chemistry driven by GEOS-5 assimilated meteorological data. It includes state-of-the-science capabilities for tropospheric and stratospheric gas–aerosol chemistry (Eastham et al., 2014; Hu et al., 2017), with additional capabilities for aerosol microphysics (Trivitayanurak et al., 2008; Yu and Luo, 2009). It is used by over a hundred active research groups in 25 countries around the world for a wide range of atmospheric chemistry applications, providing a continual stream of innovation (Hu et al., 2017). Strong version control and benchmarking maintain the integrity and referenceability of the model. The code is freely available through an open license (http://acmg.seas.harvard.edu/geos/geos_licensing.html, last access: 14 September 2018).

GEOS-Chem as used by the atmospheric chemistry community operates in an “offline” CTM mode, without explicit simulation of meteorology. Meteorological data are inputted to the model to simulate chemical transport and other processes. The offline approach makes the model simple to use and facilitates community development of the core chemical module that describes local chemical sources and sinks from emissions, reactions, thermodynamics, and deposition. Implementing the GEOS-Chem chemical module into ESMs offers a state-of-the-science and referenceable representation of atmospheric chemistry, but it is essential that the module be able to automatically incorporate new updates as the offline model evolves. Otherwise, it would become quickly dated and unsupported.

Figure 1 Schematic of GEOS-Chem chemical module used either offline as a chemical transport model (CTM) or online in an Earth system model (ESM), with interfaces managed through the Earth system modeling framework (ESMF). The module is grid independent, with individual computations done on atmospheric columns at user-selected grid points. It computes local changes in concentrations with time (dn∕dt) as a result of emissions (E), chemical production (P), chemical loss (L), and deposition (D). Emissions are handled through the Harvard-NASA EMission COmponent (HEMCO) to provide ESM users the option of only integrating emissions.

From a GEOS-Chem atmospheric chemistry user perspective, there are a number of reasons why an online simulation capability is of interest. Users working on both climate and atmospheric chemistry modeling can use the same module for both, thus improving the consistency of their approach. Some atmospheric chemistry problems involving fast coupling between chemistry and dynamics, such as aerosol–cloud interactions, require online coupling. As the resolution of ESMs increases, use of archived meteorological data becomes more difficult and incurs increased error (Yu et al., 2018), so online simulation can become more attractive. Finally, online simulations can take advantage of vast computational resources available at climate modeling centers to achieve very high resolution as illustrated in this paper. Such high-resolution model outputs are particularly important for air quality and human health applications (Cohen et al., 2017) and for OSSEs to support the design of satellite missions (Barré et al., 2016; Claeyman et al., 2011; Fishman et al., 2012; Yumimoto, 2013; Zoogman et al., 2014). A future modus operandi for the GEOS-Chem community might involve development in the offline model at coarse resolution, and online simulations when high resolution is required.

We have developed the capability to efficiently integrate the GEOS-Chem chemical module into ESMs in a way that enables seamless updating to the latest standard version of GEOS-Chem. This involved transformation of GEOS-Chem into a grid-independent and Earth system modeling framework (ESMF)-compliant model (Long et al., 2015). The exact same code is now used in stand-alone offline mode (CTM) by the GEOS-Chem community and as an online chemical module within GEOS-5 (Fig. 1). Using the exact same code for both applications ensures that the chemical module in the ESM keeps current with the latest well-documented standard version of GEOS-Chem. An important component is the Harvard-NASA EMission COmponent (Keller et al., 2014), which allows GEOS-Chem users to build customized layers of emission inventories on any grid and with no editing of the GEOS-Chem source code. HEMCO is now the standard emission module in GEOS-Chem and is used in GEOS-5 as an independent module for applications when full chemistry is not needed.

Here, we apply the GEOS-Chem chemical module within GEOS-5 in a very-high-resolution (VHR) simulation of global tropospheric ozone chemistry. The simulation is conducted for 1 full year (2013–2014) at c720 cubed-sphere resolution ($\sim \mathrm{12.5}\times \mathrm{12.5}{\mathrm{km}}^{\mathrm{2}}$) with 158 coupled chemical species. To the best of our knowledge, such a resolution in a state-of-the-science global simulation of tropospheric chemistry is unprecedented. Resolution not only increases the quality of local information, e.g., for air quality, but it also provides better representation of chemical non-linearities. We compare the model outputs with coarse offline GEOS-Chem CTM results and with independent observations for tropospheric ozone and precursors as a test of fidelity and increased power. We refer to this VHR simulation as the GEOS-5 Nature Run with GEOS-Chem chemistry, or G5NR-chem.

Figure 2 The 500 hPa ozone distribution on 1 August 2013 at 00:00 Z simulated by GEOS-5 with the GEOS-Chem chemical module at cubed-sphere c720 ($\sim \mathrm{12.5}\times \mathrm{12.5}{\mathrm{km}}^{\mathrm{2}}$) resolution.

2 GEOS-Chem as an atmospheric chemistry module in the GEOS ESM

An Eulerian (fixed frame of reference) CTM such as GEOS-Chem solves the system of coupled continuity equations for an ensemble of m species with number density vector $\mathit{n}={\left(n_{\mathrm{1}},\mathrm{\dots },n_m\right)}^T$:

$$\begin{array}{}\text{(1)}& {\displaystyle \frac{\partial n_i}{\partial t}}=-\mathrm{\nabla }\cdot \left(n_i\mathit{U}\right)+(P_i-L_i)\left(\mathit{n}\right)+E_i-D_{i}i\in \left[\mathrm{1},m\right],\end{array}$$

where U is the wind vector including subgrid components to be parameterized as turbulent diffusion or convection. (P_i−L_i)(n), E_i, and D_i are the local chemical production and loss, emission, and deposition rates, respectively, of species i. Coupling across species is through the chemical term (P_i−L_i). In GEOS-Chem, as in all 3-D CTMs, Eq. (1) is solved by operator splitting to separate the transport and local components over finite time steps. The local operator,

$$\begin{array}{}\text{(2)}& {\displaystyle \frac{\text{d}{n}_i}{\text{d}t}}=(P_i-L_i)\left(\mathit{n}\right)+E_i-D_{i}i\in \left[\mathrm{1},m\right],\end{array}$$

includes no transport terms (no spatial coupling) and thus reduces to a system of coupled ordinary differential equations (ODEs). It is commonly called the chemical operator even though emission and deposition terms are included. The transport operator,

$$\begin{array}{}\text{(3)}& {\displaystyle \frac{\partial n_i}{\partial t}}=-\mathrm{\nabla }\cdot \left(n_i\mathit{U}\right)i\in \left[\mathrm{1},m\right],\end{array}$$

does not involve coupling between chemical species.

Use of GEOS-Chem as chemical module in an ESM requires only the code that updates concentrations over a given time step for local production and loss as given by Eq. (2) (Fig. 1). The CTM has its own transport modules to solve Eq. (3) using archived meteorological inputs (offline), but these are not needed in the ESM where transport is computed as part of atmospheric dynamics (online).

Figure 3 Middle tropospheric ozone distribution at 700–400 hPa from the GEOS-5 Nature Run with GEOS-Chem chemistry (a), Ozone Monitoring Instrument (OMI) satellite observations (b), and the offline GEOS-Chem CTM (c) for four seasons covering July 2013–June 2014. The data are on a $\mathrm{2}{}^{\circ }\times \mathrm{2.5}{}^{\circ }$ grid. All data are smoothed by the OMI averaging kernels, using a single fixed a priori profile so that variability is solely driven by observations. The OMI observations have been further corrected for a global mean positive bias of 3.6 ppbv (Hu et al., 2017). Both models are sampled along the OMI tracks. Numbers in the left and right columns are the mean model bias ± standard deviation. Gray shading indicates regions where OMI data are unreliable and not used (poleward of 45^∘ in winter–spring and poleward of 60^∘ year round; see Hu et al., 2017).

The ESM chemical module is tasked with updating chemical concentrations by integration of Eq. (2) on the ESM grid and time step. Exchange of information between the chemical module and other ESM modules can be done by various couplers such as ESMF (Hill et al., 2004). Our guiding principle is that the CTM and the ESM chemical module share the exact same code. This required restructuring GEOS-Chem to a grid-independent form and making the code compliant with the ESMF Modeling Analysis and Prediction Layer (MAPL) coupler used by GEOS (Long et al., 2015). The 1-D vertical columns are the smallest efficient unit of computation for the chemical module because several operations are vertically coupled, including radiative transfer, vertically distributed emissions, wet scavenging, and particle settling. In the now grid-independent GEOS-Chem code, horizontal grid points are selected at runtime through the ESMF interface. The chemical and emission modules proceed to solve Eq. (2) on 1-D columns for the specified horizontal grid points (Fig. 1). We managed to carry out this major software transformation in GEOS-Chem in a way that was completely transparent to CTM users (Long et al., 2015). The exact same ESMF-compliant, grid-independent GEOS-Chem code is now used both in the stand-alone CTM and within GEOS. This enables seamless integration of future new GEOS-Chem scientific developments into the GEOS chemical module, which thus always remains current and referenced to the latest standard version of GEOS-Chem.

An important step in transforming GEOS-Chem to a grid-independent structure was to reconfigure the emission module. The emission module consists of multiple layers of databases and algorithms describing emissions for different species and regions, with scaling factors defining diurnal/weekly/seasonal/secular trends or dependences on environmental variables. The databases are on different grids and time stamps, and may add to or supersede each other as controlled by the user. The HEMCO module allows users to choose any combination of emission inventories in NetCDF format, on any grid and with any scaling factors, and apply them to any model grid specified at runtime (Keller et al., 2014). HEMCO provides a complete functional separation of emissions from transport, deposition, and chemistry in GEOS-Chem. The purpose of this separation is that the ESM may need emissions independently from atmospheric chemistry, for example, to simulate species such as CO₂ and methane. HEMCO is thus configured as a stand-alone component in the ESM, accessed separately through the ESMF interface (Fig. 1).

Some care is needed when interfacing the GEOS-Chem chemical module with fast vertical transport processes in GEOS-5 involving boundary layer mixing (Lock et al., 2000; Louis et al., 1982) and deep convection with wet scavenging. Here, we apply boundary layer mixing at every time step in GEOS-5 with emissions and dry deposition updates from GEOS-Chem but before applying chemistry. This avoids anomalies in the lowest model layer when the timescale for boundary layer mixing is shorter than the time step for emissions. Deep convective transport of chemical species including scavenging in the updrafts is performed by the GEOS-Chem convection scheme driven by instantaneous diagnostic variables from the GEOS-5 convection component (Molod et al., 2015). This takes advantage of the gas and aerosol scavenging capability of the GEOS-Chem scheme (Amos et al., 2012; Liu et al., 2001). Radon-222 tracer simulation tests within GEOS-5 show that the GEOS-Chem convection scheme closely reproduces the GEOS-5 convective transport (Yu et al., 2018). As convection becomes increasingly resolved at higher model resolution, the GEOS-5 subgrid convection parameterization (Moorthi and Suarez, 1992) is invoked less frequently. As a consequence, an increasing fraction of the washout in GEOS-Chem becomes characterized as large scale, as opposed to convective. No attempts were made to offset the possible increase in washout efficiency that may arise from this.

3 GEOS-5 Nature Run with GEOS-Chem chemistry

We perform a year-long (1 July 2013 to 1 July 2014) GEOS-5 simulation with GEOS-Chem at cubed-sphere c720 ($\sim \mathrm{12.5}\times \mathrm{12.5}{\mathrm{km}}^{\mathrm{2}}$) horizontal resolution and 72 vertical levels extending up to 0.01 hPa. For initialization, we use 12 months at c48 resolution ($\sim \mathrm{200}\times \mathrm{200}{\mathrm{km}}^{\mathrm{2}}$) followed by 6 months at c720 resolution. Figure 2 shows a snapshot of the simulated 500 hPa ozone field, illustrating the fine detail enabled by the very high resolution.

3.1 General description

The GEOS-5 Nature Run with GEOS-Chem chemistry is performed with the Heracles version of GEOS-5 (tag “M2R12K-3_0_GCC”). The finite-volume dynamics is run in a non-hydrostatic mode with a heartbeat time of 300 s applied to the physics, chemical, and dynamics components. The simulation is forced by downscaled meteorological data from the lower-resolution Modern-Era Retrospective Analysis for Research and Applications version 2 (MERRA-2) reanalysis (Gelaro et al., 2017; Molod et al., 2015). The downscaling is performed by using the replay capability in GEOS, which adds a forcing term to the model equations, constraining them to a specific trajectory to simulate the 2013–2014 meteorological year (Orbe et al., 2017). Downscaling applications filter the replay increments so that only the larger scales of the flow are constrained, allowing scales finer than the analysis to evolve freely. In this simulation, a wave number of 60 was chosen as the cutoff. The simulation is performed in two segments, with the first with GEOS-Chem turned off (“regular replay”). The analysis increment produced during the first run segment is saved and reused in the subsequent run with GEOS-Chem turned on (“exact increment”). This two-segment process is computationally more efficient, as it avoids rewinding and checkpointing the model with full chemistry during the regular replay stage.

The chemical module is GEOS-Chem version v10-01 in tropospheric-only mode. It is run “passively” in G5NR-chem; thus, aerosols and trace gases do not influence the meteorology. It includes detailed HO_x−NO_x–VOC–ozone–BrO_x–aerosol tropospheric chemistry with 158 species and 412 reactions, following Jet Propulsion Laboratory (JPL) and International Union of Pure and Applied Chemistry (IUPAC) recommendations for chemical kinetics (Sander et al., 2011) and updates for BrO_x and isoprene chemistry (Mao et al., 2013; Parrella et al., 2012). The default GEOS-Chem bulk aerosol scheme is used to simulate major components for dust, sea salt, black carbon, organic carbon, sulfate, nitrate, and ammonium aerosols (Fairlie et al., 2007; Jaeglé et al., 2011; Kim et al., 2015; Park et al., 2004; Wang et al., 2014). The Fast-JX scheme with approximate randomized cloud overlap method and taking aerosol loading into account is used to calculate photolysis frequencies (Bian and Prather, 2002), as implemented by Mao et al. (2010). Linearized stratospheric chemistry is used (McLinden et al., 2000; Murray et al., 2012). The dry deposition calculation is based on a resistance-in-series model (Wesely, 1989), as implemented by Wang et al. (1998). Wet scavenging of aerosols and gases is as described by Liu et al. (2001) and Amos et al. (2012).

Emissions are calculated through HEMCO v1.1.008 (Keller et al., 2014). They are the default 2013–2014 emissions for GEOS-Chem (Hu et al., 2017), with a few modifications. Open fire emissions are from the Quick Fire Emissions Dataset (QFED) version 2.4-r6 (Darmenov and Da Silva, 2015). US NO_x emissions follow Travis et al. (2016). Parameterizations for lightning NO_x (Murray et al., 2012) and mineral dust aerosol emissions (Zender et al., 2003) have large dependencies on grid resolution and are scaled globally following general GEOS-Chem practice to annual totals of 6.5 TgN for lightning and 840 TgC for dust. No adjustments are made to emission of biogenic volatile organic compounds (VOCs) (Guenther et al., 2012; Hu et al., 2015b) and sea salt aerosol (Jaeglé et al., 2011), both of which agree with GEOS-Chem emissions within 15 %. GEOS-Chem uses in-plume chemistry of ship emissions (PARANOx) to account for the excessive dilution of ship exhaust plumes at coarse model resolution (Vinken et al., 2011); this was disabled in the VHR simulation given that the Nature Run resolution is fine enough to resolve the non-linear chemistry associated with ship plume emissions. A summary of the various emission sources used for the simulation is given in Table A2.

Table 1 Global tropospheric burdens.

^a Hu et al. (2017). ^b Calculated with a chemical tropopause as the 150 ppbv ozone isopleth. ^c Interquartile range of 50 models summarized in Young et al. (2018); limited observational estimates fall within that range. ^d Gaubert et al. (2016). ^e Global mean air-mass-weighted OH concentration. ^f From 16 model results summarized in Naik et al. (2013).

Figure 4 Comparison of GEOS-5 Nature Run with GEOS-Chem chemistry to OMI 700–400 hPa ozone measurements for four seasons in July 2013–June 2014, colored by the latitude of the observations. Each point represents the seasonal mean for a $\mathrm{2}{}^{\circ }\times \mathrm{2.5}{}^{\circ }$ grid cell. Black dashed lines show the best fit (reduced major axis regression) with regression parameters given in the inset. Numbers on the bottom right are the global mean model bias ± standard deviation. The 1:1 line is shown in red.

Figure 5 Annual mean ozonesonde profiles in July 2013–June 2014 for representative global regions (Tilmes et al., 2012). Results from the GEOS-5 Nature Run with GEOS-Chem chemistry (red) are compared to observations (black) and to the GEOS-Chem CTM (blue; $\mathrm{2}{}^{\circ }\times \mathrm{2.5}{}^{\circ }$ version in Hu et al., 2017). The models are sampled at the ozonesonde launch times and locations.

Figure 6 Mean vertical profiles of trace gas concentrations over the southeast US during the NASA Studies of Emissions, Atmospheric Composition, Clouds and Climate Coupling by Regional Surveys (SEAC⁴RS) aircraft campaign (August–September 2013; Toon et al., 2016). Results from the GEOS-5 Nature Run with GEOS-Chem chemistry are compared to observations for ozone and NO_x, CO, peroxyacetyl nitrate (PAN), and formaldehyde (HCHO), and to the GEOS-Chem CTM (nested $\mathrm{0.25}{}^{\circ }\times \mathrm{0.3125}{}^{\circ }$ version in Travis et al., 2016). Model results are sampled along the flight tracks at the time of flights.

3.2 Computational environment and cost

The computations were carried out on the Discover supercomputing cluster of the NASA Center for Climate Simulation. Overall, 1 day of simulation took approximately 2 wall-time hours, using 4705 compute cores with 45 % spent on dynamics and physics, 24 % on chemistry, and 31 % on input/output (I/O) (Table A1). The large I/O wall time is due to bottlenecks in the MAPL software version used for the simulation, with excessive reading and remapping of the hourly high-resolution emission fields. This issue has been addressed in newer versions of MAPL. As first shown by Long et al. (2015), the chemical module has excellent scalability even when running with thousands of cores. The percentage of the wall time spent on chemistry in G5NR-chem (24 %) is much lower than in coarse-resolution simulations that are typically done with only a small number of cores (Eastham et al., 2018). The computational cost of chemistry relative to dynamics/transport decreases as grid resolution increases; thus, it is no longer the computing bottleneck in ESM simulations.

4 Model evaluation

4.1 Observational datasets and offline CTM

The GEOS-5 Nature Run with GEOS-Chem chemistry simulation is intended to support geostationary constellation OSSEs focused on tropospheric ozone and related satellite measurements (Zoogman et al., 2017), and ozone is therefore our evaluation focus. We use 2013–2014 observations that were previously compared to the GEOS-Chem CTM including (1) global ozonesondes and Ozone Monitoring Instrument (OMI) satellite data (Hu et al., 2017), (2) aircraft data for ozone and precursors from the NASA Studies of Emissions, Atmospheric Composition, Clouds and Climate Coupling by Regional Surveys (SEAC⁴RS) campaign over the southeast US (Travis et al., 2016), and (3) surface ozone monitoring data over Europe and the US (Grange, 2017; Yan et al., 2016). An important goal of the evaluation here is to examine consistency between the GEOS-Chem chemical module within the GEOS ESM c720 environment and the offline GEOS-Chem CTM. Although the GEOS-Chem simulation is at coarser resolution and offline transport may incur errors (Yu et al., 2018), it is extensively diagnosed by the GEOS-Chem user community, including recently by Hu et al. (2017) for global tropospheric ozone. Two GEOS-Chem CTM v10-01 simulations are used for comparison to G5NR-chem: a global simulation with $\mathrm{2}{}^{\circ }\times \mathrm{2.5}{}^{\circ }$ resolution, and a nested simulation for North America with $\mathrm{0.25}{}^{\circ }\times \mathrm{0.3125}{}^{\circ }$ resolution. Both are driven by GEOS-5 Forward Processing (GEOS-FP) (GEOS-5.7.2 and later versions) assimilated meteorological data. Some differences with G5NR-chem are to be expected because of differences in the transport modules, resolution, distribution of natural sources computed online such as lightning NO_x, and meteorological data from different versions of the GEOS system (MERRA-2 vs. GEOS-FP). All comparisons to observations use model output sampled at the location and time of observations.

Figure 7 Monthly median afternoon (12:00–16:00 LT) ozone concentrations (ppbv) for 2013–2014 at surface sites in the US and Europe, with interquartile range shaded. Surface monitoring sites are grouped according to Fig. A2. Also shown are the GEOS-5 Nature Run with GEOS-Chem chemistry (red) and the offline GEOS-Chem CTM (blue). Hourly model outputs are sampled for the locations and time of observations at the surface (lowest) level.

Observational datasets are described in the above references. Briefly, global ozonesonde observations are extracted from the WOUDC (World Ozone and Ultraviolet Radiation Data Centre; http://www.woudc.org, last access: 14 September 2018) and NOAA ESRL-GMD (Earth System Research Laboratory – Global Monitoring Division; ftp://ftp.cmdl.noaa.gov/ozwv/Ozonesonde/, last access: 14 September 2018). Ozonesonde stations are grouped into coherent regions for model evaluation (Tilmes et al., 2012). OMI middle tropospheric ozone data at 700–400 hPa are from the Smithsonian Astrophysical Observatory (SAO TROPOZ) retrieval (Huang et al., 2017; Liu et al., 2010) and are regridded to $\mathrm{2}{}^{\circ }\times \mathrm{2.5}{}^{\circ }$ resolution to reduce retrieval noise. The NASA SEAC⁴RS dataset for southeast US described by Toon et al. (2016) is filtered following Travis et al. (2016) to remove open fire plumes (CH₃CN > 200 pptv), stratospheric air (O₃∕CO > 1.25 mol mol⁻¹), and urban plumes (NO₂ > 4 ppbv). Hourly surface observations for ozone are taken from the European Environment Agency database (complied by Grange, 2017) and the US Environmental Protection Agency Air Quality System (http://aqsdr1.epa.gov/aqsweb/aqstmp/airdata/download_files.html#Raw, last access: 14 September 2018). Only “background” sites are considered in the analysis: for the US, this includes sites defined by the EPA as “suburban” and “rural”; for Europe, this includes sites categorized as “urban background”, “background”, and “rural” (see Fig. A2).

4.2 Global burdens

Standard global metrics for evaluation of tropospheric chemistry models include the global burdens of tropospheric ozone, CO, and OH (Table 1). The global annual average ozone burden in G5NR-chem amounts to 348 Tg, consistent with the GEOS-Chem CTM and the Tropospheric Ozone Assessment Report (Hu et al., 2017; Young et al., 2018). The global burden of tropospheric CO of 294 Tg is consistent with the GEOS-Chem CTM and on the low end of the observationally based estimate of Gaubert et al. (2016). The global mean OH concentration is lower than in the GEOS-Chem CTM and more consistent with observational constraints (Prather et al., 2012; Prinn et al., 2005). The differences appear to be mainly driven by differences in the meteorological data.

4.3 Free tropospheric evaluation: OMI, ozonesonde, SEAC⁴RS

Figure 3 compares the GEOS-5 Nature Run with GEOS-Chem chemistry to OMI midtropospheric ozone. GEOS-Chem CTM results from Hu et al. (2017) are also shown. OMI data have been reprocessed with a single fixed a priori profile (so that variability is solely due to observations), corrected for their global mean bias relative to ozonesondes, and filtered for high latitudes because of large biases (Hu et al., 2017). Model outputs are sampled along the OMI tracks and smoothed with the OMI averaging kernels. G5NR-chem captures well-known major features of the ozone distribution, such as ozone enhancements at northern midlatitudes during MAM and JJA, and downwind of South America and Africa during SON. It shows no significant global bias relative to OMI and relative to the offline GEOS-Chem CTM. The global mean seasonal biases are less than 2.7±3.2 ppbv. Spatial correlations for the four seasons on the $\mathrm{2}{}^{\circ }\times \mathrm{2.5}{}^{\circ }$ grid scale are high and show no significant latitudinal bias ($R=\mathrm{0.81}-\mathrm{0.93}$; Fig. 4).

Figure 5 further evaluates the simulated vertical distribution of ozone in comparison to ozonesonde data. There are differences between G5NR-chem and the GEOS-Chem CTM that could be due to a number of factors including differences in tropopause altitude and the distribution of lightning. For example, although the annual total lightning NO_x emission in G5NR-chem is scaled to that in GEOS-Chem CTM, the lightning location is not constrained by satellite lightning flash data as the CTM is. Annual mean ozone biases in G5NR-chem are generally less than 6 ppbv in the lower troposphere. There are some larger biases in the upper troposphere including differences with the GEOS-Chem CTM that could be due to the spatial distribution of lightning. Overall, G5NR-chem tends to improve the simulation of ozone vertical profiles compared to the GEOS-Chem CTM, most dramatically at high southern latitudes.

Figure 6 compares the model to mean vertical profiles of ozone and precursors measured over the southeast US during the SEAC⁴RS aircraft campaign. Here, the GEOS-Chem CTM results are from a nested $\mathrm{0.25}{}^{\circ }\times \mathrm{0.3125}{}^{\circ }$ simulation by Travis et al. (2016). The lower ozone in the northern midlatitude upper troposphere in G5NR-chem appears to be due to a weaker lightning NO_x source. G5NR-chem overestimates ozone in the lower troposphere by 10 ppbv, while such bias is reduced in the GEOS-Chem CTM, even through both show almost identical NO_x levels. The lower HCHO over the southeast US is due to weaker isoprene emission because of lower temperatures. Both models underestimate CO in the free troposphere but the bias is more apparent in the CTM, likely due to differences in fire emissions, global OH fields, or transport error in offline simulations (Yu et al., 2018).

4.4 Evaluation with surface observations over the US and Europe

Figure 7 shows monthly median surface ozone concentrations grouped by regions in the US and Europe. Here, hourly data between 12:00 and 16:00 LT are used to remove the known issue that models typically underestimate the ozone nighttime depletion at surface sites (Millet et al., 2015). G5NR-chem systematically overestimates surface ozone in almost all months by about 10 ppbv in all regions, while the GEOS-Chem CTM has no or small bias in winter and spring, but shows similar overestimates as in G5NR-chem during summer and fall. In general, the GEOS-5 Nature Run with GEOS-Chem chemistry better captures the observed seasonality. Expanding the analysis to all hourly data does not affect the systematic bias in G5NR-chem significantly but tends to increases the summer–fall bias in the CTM (Fig. A5). Part of the systematic bias is due to the subgrid vertical gradient between the lowest model level and the measurement altitude (Travis et al., 2017). Recent model developments such as improved halogen chemistry, aromatic chemistry, and ozone dry deposition are expected to reduce the surface high bias (Schmidt et al., 2016; Sherwen et al., 2016a, b, 2017; Silva and Heald, 2018; Yan et al., 2018). These updates are being incorporated into the GEOS-Chem version currently under development and will be passed on to the GEOS-5 simulation as they become available.

5 Conclusions

We presented a 1-year global simulation of tropospheric chemistry within the NASA GEOS ESM version 5 (GEOS-5) at cubed-sphere c720 ($\sim \mathrm{12.5}\times \mathrm{12.5}{\mathrm{km}}^{\mathrm{2}}$) resolution. This demonstrated the success of implementing the GEOS-Chem chemical module within an ESM for online simulations with detailed chemistry. The GEOS-Chem chemical module online within GEOS and offline as the GEOS-Chem CTM uses exactly the same code. In this way, the continual stream of chemical updates from the large GEOS-Chem CTM community can be seamlessly incorporated as updates to the online model, which always remains state of the science and referenceable to the latest version of the GEOS-Chem. This 1-year simulation addressed an immediate need to generate the Nature Run for OSSEs in support of the geostationary satellite constellation for tropospheric chemistry. More broadly, implementation of the GEOS-Chem capability opens up a new capability for GEOS to address aerosol–chemistry–climate interactions, to assimilate satellite data of atmospheric composition, and to develop global air quality forecasts.

The 1-year GEOS-5 simulation at c720 resolution required 31 days of wall time on 4705 cores. Overall, 45 % of the wall time was spent on model dynamics and physics, 31 % on input/output, and 24 % on chemistry. Chemistry has near-perfect scalability in massively parallel architectures because it operates on individual grid columns; thus, it is no longer a computing bottleneck in ESM simulations. Transporting the large number of species involved in atmospheric chemistry simulations may be a greater challenge.

We evaluated the GEOS-5 Nature Run with GEOS-Chem chemistry for consistency with the offline GEOS-Chem CTM at coarser resolution ($\mathrm{2}{}^{\circ }\times \mathrm{2.5}{}^{\circ }$ global and $\mathrm{0.25}{}^{\circ }\times \mathrm{0.3125}{}^{\circ }$ nested over North America) as well as an ensemble of global observations for tropospheric ozone and aircraft observations of ozone precursors over the southeast US. The model shows no significant global bias relative to OMI midtropospheric ozone data and the offline GEOS-Chem CTM. Evaluations with ozonesondes show reduced model biases for high-latitude ozone. The GEOS-5 Nature Run with GEOS-Chem chemistry systematically overestimates surface ozone concentrations by 10 ppbv all year round in the US and Europe but is able to capture the observed seasonality, while the offline GEOS-Chem CTM reproduces observed surface ozone levels in winter and spring but has similar biases in summer and fall in all regions. Resolving this surface bias is presently a focus of attention in the GEOS-Chem CTM community and future model updates to address that bias can then be readily implemented into GEOS-5.

Appendix A

Figure A1 Same as Fig. 5 but for western US, equatorial South America, and Atlantic/equatorial Africa.

Figure A2 Surface monitoring sites in the US and Europe grouped by subregions as analyzed in the text. Background image © Natural Earth (public domain license).

Figure A3 Probability distribution of afternoon (12:00–16:00 LT) ozone concentrations (ppbv) for 2013–2014 at surface sites in the US and Europe.

Figure A4 Monthly median ozone concentrations (ppbv; all 24 h data) for 2013–2014 at surface sites in the US and Europe, with interquartile range shaded. Surface monitoring sites are grouped according to Fig. A2. Also shown are the GEOS-5 Nature Run with GEOS-Chem chemistry (red) and the offline GEOS-Chem CTM (blue). Hourly model outputs are sampled for the locations and time of observations at the surface (lowest) level.

Figure A5 Probability distribution of ozone concentrations (ppbv; all 24 h data) for 2013–2014 at surface sites in the US and Europe.

Table A1 Computing resources for the GEOS-5 Nature Run with GEOS-Chem chemistry^a.

^a The simulation is from 1 July 2013 to 1 July 2014 in GEOS ESM with GEOS-Chem as a chemical module at 12.5×12.5 km². The computation was carried out at NASA Discover supercomputing cluster. ^b The simulation uses 337 14-core 2.6 GHz Intel Xeon Haswell compute nodes with 128 GB of memory per node and an Infiniband FDR interconnect using the Intel compiler suite (v. 15.0.0.090) and MPT v. 2.11. ^c 158 species are simulated and transported by the GEOS ESM; among them, 29 species are saved out as hourly outputs. Data are available at https://portal.nccs.nasa.gov/datashare/G5NR-Chem/Heracles/12.5km/DATA/ (last access: 14 September 2018).

2008

Table A2 Emissions in the GEOS-5 Nature Run with GEOS-Chem chemistry.

^a Excluding aircraft and ships, which are listed separately. ^b http://aerocom.met.no/download/emissions/AEROCOM_HC/volc/ (last access: 14 September 2018). ^c Hudman et al. (2012). ^d Murray et al. (2012). ^e Jaeglé et al. (2011) and Fischer et al. (2012). ^f Zender et al. (2003). ^g TgC yr⁻¹ for ≥C₄ alkanes, acetone, methyl ethyl ketone (MEK), CH₃CHO, C₃H₆, C₃H₈, isoprene, and C₂H₆. Tg yr⁻¹ for the rest of the species. ^h http://edgar.jrc.ec.europa.eu/htap_v2/index.php?SECURE=123 (last access: 14 September 2018). ⁱ RETRO monthly global inventory for the year 2000 (Schultz et al., 2007) implemented as described by Hu et al. (2015a). ^j US EPA National Emission Inventory 2008 (http://www.epa.gov/ttn/chief/net/2008report.pdf; last access: 14 September 2018) and scaled to 2013 (https://www3.epa.gov/airtrends/; last access: 14 September 2018). ^k Asian anthropogenic emissions (Li et al., 2014). ^l Stettler et al. (2011). ^m Quick Fire Emissions Dataset (QFED) version 2.4-r6 (Darmenov and Da Silva, 2015). ⁿ MEGANv2.1 (Guenther et al., 2012) implemented in GEOS-Chem as described by Hu et al. (2015b).