TROPOMI retrieves atmospheric methane columns from backscattered sunlight in the 2.3 µm methane absorption band, with daily global coverage at 5.5 km × 7 km nadir pixel resolution (7 km × 7 km prior to August 2019). Measurements are made at ∼ 13:30 local solar time. The methane retrieval is produced by the Netherlands Institute for Space Research (SRON). It is based on the RemoTeC full-physics algorithm (Butz et al., 2009, 2010, 2011) and retrieves methane data as column-average dry-air mixing ratios XCH4 (ppb) along with surface reflectivity and scattering properties of the atmosphere (Butz et al., 2012; Hu et al., 2016). The TROPOMI data are posted operationally on the AWS cloud and updated daily by MEEO with a latency of a few days (https://registry.opendata.aws/sentinel5p, last access: 8 June 2022). The methane product provides information on numerous retrieval parameters together with XCH4, including the center and boundaries of the pixel, the surface pressure, the 12-layer pressure grid of the retrieval, the vertical averaging kernel vector and prior vertical profile of methane dry-air mixing ratio, a quality assurance value, and the retrieved surface albedo in the near-infrared (NIR) and SWIR spectral ranges.

The operational TROPOMI record begins in May 2018. The methane retrieval is presently Version 1 (Hasekamp et al., 2019) until July 2021 and Version 2 (Lorente et al., 2021) afterward. Validation of Version 1.3.0 showed a global mean bias of -2.7 ppb relative to ground-based measurements from 19 sites in the Total Column Carbon Observing Network (TCCON; Wunch et al., 2011a; Qu et al., 2021), but global bias is of no consequence for regional inversions because it is effectively corrected through the boundary conditions. Of more concern are spatially variable biases (regional biases), caused mainly by aliasing of surface albedo errors into the methane retrieval (Lorente et al., 2021) but also by scattering-induced surface reflectance errors (Barré et al., 2021) and errors in surface altitude (Hachmeister et al., 2022). Qu et al. (2021) quantified a nominal TROPOMI regional bias of 6.7 ppb in Version 1.3.0 as the standard deviation of station-to-station biases between TROPOMI and the 19 TCCON sites, and a similar analysis for Version 2.2.0 shows a regional bias of 5.6 ppb (Lorente et al., 2021). This is sufficiently small to enable successful regional inversions, for which Buchwitz et al. (2015) estimated a regional bias threshold of 10 ppb. In the IMI we only use recommended high-quality retrievals over land, with quality assurance value ≥ 0.5 (Hu et al., 2016). We further remove observations with low SWIR albedo (< 0.05; de Gouw et al., 2020) and high “blended albedo” (> 0.85), a linear combination of NIR and SWIR albedo, to avoid biases from dark and snow-covered scenes (Wunch et al., 2011b; Lorente et al., 2021). The quantity of data removed by these additional filters depends on the region and period for the inversion; we find for example that they remove roughly 25 % (summer) to 40 % (winter) of otherwise high-quality observations across North America in 2019.

GEOS-Chem is a three-dimensional CTM that simulates methane concentrations on the basis of prescribed emissions either globally or for user-selected nested domains (Wecht et al., 2014). It is driven by Goddard Earth Observation System (GEOS) meteorological data from the NASA Global Modelling and Assimilation Office (GMAO). The IMI uses as default the GEOS Fast Processing (GEOS-FP) meteorological data product at 0.25∘ × 0.3125∘ resolution, with an option to use the GEOS Modern-Era Retrospective Analysis for Research and Applications, version 2 (MERRA-2) at 0.5∘ × 0.625∘ resolution. The GEOS data have 72 vertical levels from the surface to the mesopause, and these are condensed to 47 levels in our GEOS-Chem simulations by merging levels in the upper stratosphere and mesosphere.

We use the nested capability of GEOS-Chem to simulate methane concentrations over the inversion domain, with dynamic boundary conditions outside the inversion domain updated every 3 h from a global archive of TROPOMI data smoothed spatially over a rolling ± 10∘ window and temporally over a 1-month period centered on each grid square and day and distributed vertically following a GEOS-Chem simulation at 4∘ × 5∘ resolution (Shen et al., 2021). This smoothed TROPOMI 3-D archive is provided as part of the IMI. Using smoothed TROPOMI data as boundary conditions minimizes bias from boundary conditions advected over the user-selected region. Smoothing of the TROPOMI data is necessary because of the sparsity of successful retrievals and the noise therein. To further reduce the bias associated with boundary conditions, we expand the inversion domain beyond the user-selected region of interest to include a buffer area, and coarse buffer elements are added to the state vector of methane emissions to be optimized (Fig. 1, Sect. 2.3).

The user-specified period of interest defines the time window for the GEOS-Chem simulation. Starting from the smoothed TROPOMI fields as initial conditions, we apply a 1-month spin-up with prior emission estimates to properly initialize the model concentration fields within the inversion domain; 1 month is sufficient to fully ventilate any practical regional domain. This spin-up only needs to be done once.

The GEOS-Chem simulation includes chemical methane sinks from archived (offline) tropospheric concentrations of oxidants (OH, Cl) and stratospheric loss frequencies (Maasakkers et al., 2019), as well as soil uptake (Murguia-Flores et al., 2018), but these are inconsequential for nested-domain simulations and are not optimized by the IMI. Ventilation of the inversion domain takes place on much shorter timescales than the methane atmospheric lifetime, and the sinks are relatively spatially smooth, so no information on methane sinks is to be gained from a regional inversion. The effect of methane sinks is implicitly included in the specification of boundary conditions.

The state vector x is the ensemble of variables (“state variables”) to be optimized in the inversion. In the IMI, these are the gridded methane emissions (temporal mean) at 0.25∘ × 0.3125∘ resolution for the region and period of interest, plus buffer elements at coarser resolution bordering the region of interest and filling out the inversion domain (eight elements by default). Users specify a region and time period of interest in the IMI configuration file. The region of interest can have any irregular shape, as illustrated in Fig. 1. In that example case, the region of interest is an assemblage of 235 0.25∘ × 0.3125∘ grid cells covering the geological extent of the Permian Basin, and the eight buffer elements expand to a rectangular inversion domain 24–39∘ N, 95–111∘ W. The state vector in this example has length n=235+8=243.

The simplest (default) option for the user is to select a rectangular region of interest as latitude and longitude bounds. The IMI then infers emissions for the 0.25∘ × 0.3125∘ grid cells within that region, excluding any grid cells less than 25 % over land (adjustable default), and selects eight additional buffer elements with a k-means algorithm to pad out the rectangular inversion domain. The k-means algorithm sorts grid cells by latitude–longitude coordinates, and the number of buffer elements can be adjusted in the configuration file. Users also have the option to select an irregular region of interest, as in the Permian example of Fig. 1, by providing a previously defined state vector file or a shapefile for the region boundaries. Offshore emissions can be included in the state vector by lowering the default 25 % land cover requirement or by directly modifying the state vector file. TROPOMI does not observe over water except in the glint mode, but information on offshore emissions can still be gained from the plumes transported over nearby land (Shen et al., 2021).

The prior emission estimates xa​​​​​​​ should represent the best knowledge of methane emissions prior to performing the inversion. They need to be available in gridded format to match the resolution of the inversion. Table 1 compiles the bottom-up emission inventories used as default prior estimates in the IMI. The North American anthropogenic emissions are gridded versions of the national sector-resolved inventories reported by the individual countries to the United Nations Framework Convention on Climate Change (UNFCCC) as given by Maasakkers et al. (2016) for the United States, Scarpelli et al. (2020a) for Mexico, and Scarpelli et al. (2022a) for Canada. The emissions from fuel exploitation (oil, gas, coal) in the rest of the world similarly grid the national emissions reported annually to the UNFCCC (Scarpelli et al., 2022b). The Emission Database for Global Atmospheric Research (EDGAR) v6 is otherwise used as the global default. Natural emissions include contributions from wetlands with monthly resolution (Bloom et al., 2017), open fires with daily resolution (Randerson et al., 2018), and small sources from geological seeps and termites. These default inventories can be superseded by users with their own prior estimates, and we give an example of this in Sect. 4.

The inversion infers emissions on the 0.25∘ × 0.3125∘ grid, and this may include contributions from different sectors. Users can attribute the corrections to individual sectors based on the sectoral distribution of the emissions in the prior inventories and estimates of prior errors for each sector (Shen et al., 2021; Cusworth et al., 2021a). This needs to be done in post-processing of the inversion results.

The forward model y=F(x) for the inversion involves successive application of a GEOS-Chem operator C=G(x) that relates the emission state vector x to the resulting 3-D simulated dry-air mixing ratio field C and a TROPOMI operator y=T(C) that relates the vertical profile of simulated dry-air mixing ratios to the corresponding column-average dry-air mixing ratio (XCH4) that would be observed by TROPOMI. The TROPOMI retrieval provides information on the operator T as the dependence of XCH4 on the local vertical profile vector of dry-air mixing ratios c (with prior estimate ca) for 12 sub-column pressure layers extending from the local surface to the top of the atmosphere, with vertical sensitivity described by a column-averaging kernel vector η for those 12 layers: 1XCH4=ηTc+(1-η)Tca, where 1 denotes a 12-dimensional unit vector.

Figure 2 summarizes the operations involved in simulating TROPOMI observations of the GEOS-Chem atmosphere. The first step is to geo-locate the TROPOMI pixel (nadir resolution 5.5 km × 7 km, but coarser off-nadir) on the GEOS-Chem 0.25∘ × 0.3125∘ grid, including the region of interest and the surrounding buffer elements. If the pixel overlaps two or more GEOS-Chem grid cells then the calculation is done for each grid cell column followed by area-weighted averaging. We remap the sub-column mixing ratios from the GEOS-Chem vertical grid (47 layers) to the TROPOMI vertical grid (12 layers) with total or partial allocation of GEOS-Chem layers to TROPOMI layers on the basis of pressure edges (Fig. 2). We then apply the TROPOMI column-averaging kernel vector η with Eq. () to obtain the column-average dry-air mixing ratio XCH4 as would be observed by TROPOMI in the GEOS-Chem atmosphere. When remapping GEOS-Chem to the TROPOMI vertical grid, we address differences in surface pressure between GEOS-Chem and TROPOMI by adjusting the lowest GEOS-Chem pressure edge to match that of TROPOMI, as illustrated in Fig. 2; this applies the lowest-level sub-column mixing ratio in GEOS-Chem down to the lowest TROPOMI pressure edge.

The column-averaging kernel sensitivities in TROPOMI are generally within 2 % of unity in the troposphere and drop off slowly in the stratosphere (Hu et al., 2016). Thus the pressure remapping has relatively little effect except in regions with strong topography, where high-elevation pixels have greater stratospheric contribution to XCH4. Stanevich et al. (2020) reported that stratospheric methane in GEOS-Chem exhibits a high bias relative to ACE-FTS satellite observations, but Zhang et al. (2021) found that this bias is largely restricted to polar vortex conditions where TROPOMI does not have observations.

Our Bayesian inversion to infer methane emissions fits the GEOS-Chem simulation to the TROPOMI observations, weighing prior and observational uncertainties and assuming normal error pdf's. This involves minimization of the scalar cost function (Brasseur and Jacob, 2017) 2J(x)=(x-xa)TSa-1(x-xa)+γ(y-Kx)TSo-1(y-Kx), where K=∂y/∂x is the Jacobian matrix, Sa is the prior error covariance matrix, So is the observational error covariance matrix including contributions from instrument and forward model errors, and γ is an additional regularization parameter. K describes the sensitivity of observations y to the state vector x as described by the forward model F(x). It is computed column by column from an ensemble of perturbation simulations in the forward model, each perturbing a single element of the state vector from the reference simulation. Because the model is strictly linear, K defines GEOS-Chem for the purpose of the inversion.

The default Sa is constructed in the IMI by assuming 50 % error standard deviation on emissions, with no error correlations (diagonal matrix). The default So assumes a uniform observational error standard deviation of 15 ppb, based on previous estimates of 13–15 ppb for TROPOMI by the residual error method (Qu et al., 2021; Shen et al., 2021), again with no error correlation. These default values are adjustable by the user through the configuration file. The assumption of uncorrelated prior errors may lead to underestimation of the aggregated error in total regional emissions.

The regularization parameter γ is used to prevent overfitting and underfitting that would result from inexact specifications of Sa and So and because the observations are not perfectly independent and identically distributed (IID condition). The best value for γ can be selected on the basis of the L curve (Hansen, 1999) or the expected Chi-square distribution of the cost function's prior terms (Lu et al., 2021). These two methods yield consistent results (Qu et al., 2021). Shen et al. (2021) used the L curve to select γ=0.25 for a regional inversion of TROPOMI observations over eastern Mexico at 0.25∘ × 0.3125∘ resolution. We adopt that value in the IMI as default, but it can be adjusted in configuration.

The posterior state vector x^ minimizing J(x) is obtained by an analytical solution of dJ/dx=0 as 3x^=xa+γKTSo-1K+Sa-1-1γKTSo-1(y-Kxa), with posterior error covariance matrix (characterizing uncertainty in x^) given by 4S^=γKTSo-1K+Sa-1-1. S^ provides full closed-form characterization of the error in x^ assuming that the inverse problem has been well posed through the formulation of the cost function. Errors in the formulation of the cost function can be evaluated through an inversion ensemble varying inversion parameters (e.g., γ), prior emission estimates, and satellite observation sampling. The averaging kernel matrix 5A=In-S^Sa-1 describes the sensitivity of x^ to the truth (i.e., A=∂x^/∂x). The trace of A, referred to as the degrees of freedom for signal (DOFS; Rodgers, 2000), measures the information content of the observations towards optimizing the state vector. It represents the number of independent pieces of information on the state vector that the observations can quantify. The diagonal entries of A are referred to as averaging kernel sensitivities, and they give an estimate of how much the posterior solution for a given state vector element is informed by the observations as opposed to the prior estimates (Cui et al., 2014; Brasseur and Jacob, 2017). An emission element with averaging kernel sensitivity 0 is not quantified by the observations at all, and the inversion results for that grid cell return the prior value. An emission element with averaging kernel sensitivity 1 is fully quantified by the observations, and the inversion results for that grid cell are independent of the prior estimate. We use sparse matrix algebra for the matrix operations in Eqs. ()–() so that the computational cost of the optimization procedure is small relative to the cost of constructing the Jacobian. Sample performance statistics are given in Sect. 3.7.

The IMI includes a preview feature designed to help users avoid spending resources on inversions with insufficient information content. Lack of information could come from low TROPOMI data density (e.g., from cloud cover) and/or from seriously biased prior emission estimates for the region and period of interest. The preview can be run after configuring the IMI and before initiating the inversion, and it performs several tasks. First, it maps the TROPOMI data and prior emission estimates for the selected region and period of interest, so the user can assess spatial correspondence between the two datasets. Second, it maps observation density and counts the total number of observations available for the selected region and period. Third, it maps the SWIR albedo retrieved by TROPOMI to help users identify potential artifacts if the SWIR albedo and methane retrievals show similar features (Barré et al., 2021). Fourth, it estimates the USD financial cost of performing the inversion by scaling the cost of our illustrative Permian Basin inversion (Sect. 4.3) according to the number of state variables, grid resolution, and inversion period length. Finally, it makes a rough estimate of the expected DOFS for the user's inversion using the procedure outlined below. A detailed example of the IMI preview feature is presented in Sect. 4.2.

The rough estimate of the expected DOFS is done as follows. Ignoring error correlations, assuming uniform observational errors, and further assuming uniform transport, the calculation of the averaging kernel matrix reduces to a scalar problem (Brasseur and Jacob, 2017). The averaging kernel sensitivity A for a given emission element in the state vector is computed as 6A=σa2σa2+(σo/k)2m, where σa (kgm-2s-1) is the prior error standard deviation of the emission element, σo (molmol-1) is the observational error standard deviation, m is the number of satellite observations relevant to that emission element, and the transport model is defined by the parameter k (m2skg-1) as a summary representation of the Jacobian. With default 50 % prior error standard deviation, we have σa=0.5Qa/(nL2), where Qa (kgs-1) is the total prior emission for the region of interest, n is the number of emission elements in that region of interest, and L (m) is the grid cell side length (25 km in the GEOS-FP default). For our guiding Permian Basin example using the default IMI emission inventories, Qa = 1.1 Tga-1, and n=235, which yields σa = 1.2 × 10-10 kgm-2s-1. The mean number of observations m per emission element is the total number of observations for the region and period of interest, divided by n; for the May 2018 Permian example we obtain m=86 from 19 978 observations (see Sect. 4.2). σo is by default 15 × 10-9 molmol-1.

To estimate k we use the approximation proposed by Nesser et al. (2021) for simple mass balance ventilation of local emissions in the grid cell by a constant wind: 7k=αMairMCH4LgUp, where Mair is the molar mass of dry air, MCH4 is the molar mass of methane, g is gravitational acceleration, U is a uniform wind speed ventilating the emission element (assumed 5 kmh-1), and p is the surface pressure (assumed 1010 hPa). The parameter α serves as a simple representation of turbulent diffusion, and here we take α=0.4 following Nesser et al. (2021) so that k = 1.26 m2skg-1. After computing A in this way, the expected information content for the inversion can be obtained as 8DOFS=nA=nσa2σa2+(σo/k)2m.

Equation () gives a quick estimate of the information content to be expected from the inversion without actually performing the inversion. Although very rough, it is based on the same principles as the actual inversion, and we find that it gives a good approximation of the actual DOFS as demonstrated in Sect. 4.2. It further has the advantage of being transparent in that n and m are defined by the user choice of region and period of interest, σa and σo are set by default in the IMI but are configurable by the user, and k has direct physical meaning. In fact, k can be used for a very rough estimate of emissions corresponding to a local column enhancement (Jacob et al., 2016).

The user may decide on the basis of the DOFS estimated from Eq. () whether or not to carry out the inversion. DOFS ∼ 1 would be a minimum requirement to achieve any solid information on emissions in the region of interest, and more may be desirable if multiple pieces of information are desired on the emission fields within the region. Shen et al. (2022) required DOFS > 2 to reliably estimate basin-wide emissions from oil and gas basins in North America. If the user deems the DOFS to be insufficient, a cure is to increase the number of observations by lengthening the observation period. The user may also revisit the information on the prior emission estimate and whether a larger value of σa may be appropriate, which will increase the DOFS.

Beyond inspection of the DOFS, the user should inspect the preview plots to guard against large artifacts in the observations or large bias in the spatial distribution of prior estimates. Artifacts in the observations can be diagnosed by similarity of patterns between XCH4 and SWIR albedo, implying that spectral dependence of the albedo is propagating into the XCH4 retrieval. If so the observations should not be used. Large bias in the spatial distribution of prior estimates can be diagnosed by comparison to the TROPOMI observations and would be problematic in the inversion by misallocating the corrections (Yu et al., 2021); this can be addressed by increasing the error in the prior estimate (including very large values to mimic a non-informative uniform prior) or switching to a different prior emission inventory, as is illustrated in Sect. 4 in the context of the Permian example.

We perform a 1-month inversion for the Permian Basin (currently the most prolific US oil-producing basin) as an illustrative application of the IMI. We choose 1–31 May 2018 as the period of interest for the inversion. The region of interest is defined from a shapefile for the Permian Basin and comprises 235 state vector elements to describe emissions within the region at 0.25∘ × 0.3125∘ resolution, plus 8 buffer elements to pad out the inversion domain, for a total of 243 state vector elements (Fig. 1).

We perform the inversion using the default IMI settings laid out in Tables 1 and 2 but with the custom state vector of Fig. 1. The steps prior to initiating the inversion are as follows:

Create an AWS instance with the IMI workflow AMI.

Figure 5 shows the IMI preview results including the mean TROPOMI XCH4 data for the selected region and period, the observation density, the TROPOMI SWIR albedo, and the default prior emission estimates (here the EPA GHGI). The TROPOMI XCH4 data (Fig. 5a) include N = 19 978 individual observations for the region of interest, and these are used for the DOFS estimate in the preview. There are more than 100 000 additional observations in the inversion domain outside the region of interest and covering the buffer grid cells (Fig. 1). The two methane hotspots at the center of Fig. 5a correspond to the Permian's Delaware and Midland sub-basins. TROPOMI provides relatively uniform sampling across the region of interest (Fig. 5c), and visual comparison of Fig. 5a and d shows no indication of albedo-related regional XCH4 biases. However, we see that the gridded GHGI inventory (Fig. 5b) severely misrepresents the spatial distribution of emissions in the Permian by failing to capture the sub-basin structure apparent in Fig. 5a. Furthermore, the inversion preview indicates an expected DOFS value of 2.0, which is marginal for quantifying emissions on that regional scale (Shen et al., 2021b).

At this point it would be sensible to reconfigure the IMI before performing the inversion, and we explain how to do so in what follows. If we proceed and conduct the inversion with these default settings, we find a DOFS of 1.9 (close to the preview). The posterior emission integrated over the region of interest is 1.8 Tga-1, much higher than the default GHGI prior emission of 1.1 Tga-1, and with scale factors (posterior over prior ratios) ranging from 1.0 to 3.3. These results are consistent with independent observations that the GHGI emissions for the Permian Basin are far too low (Omara et al., 2018; Robertson et al., 2020; Y. Chen et al., 2022; Cusworth et al., 2021b; Irakulis-Loitxate et al., 2021; Lyon et al., 2021), but the low DOFS and biased spatial distribution in the prior emissions do not inspire confidence in the results.

One can increase the DOFS simply by increasing the length of the inversion period, thus accumulating more observations, but the incorrect spatial distribution of the prior estimate will make it harder for the inversion to converge to the correct solution (Yu et al., 2021). An alternative is to increase the magnitude of the prior error estimate, but this may result in unphysical solutions if the problem is underconstrained in part of the domain. The user can judge from the output if these issues are severe.

A better alternative is to investigate whether an improved bottom-up inventory would enable a more accurate inversion. In the case of the Permian Basin, an alternative gridded bottom-up inventory is available from the Environmental Defense Fund (EDF) with more accurate accounting of oil and gas infrastructure and larger total emissions of 2.7 Tga-1 (Zhang et al., 2020). IMI results using the EDF inventory as a custom bottom-up prior estimate are shown in Fig. 6. Starting with the IMI preview, we find that the spatial distribution of prior emissions is much more consistent with the TROPOMI data (Fig. 6a, compare to Fig. 5b), with a much higher expected DOFS value of 11.7 that reflects the higher prior emissions (and hence the larger absolute prior error standard deviations). Proceeding to run the IMI workflow, we find that the posterior emissions now total 3.9 Tga-1, up 45 % from the prior estimate of 2.7 Tga-1 and with clear demarcation of the two sub-basins. The new scale factors range from 0.68 to 2.55, reflecting a need for both increased and decreased emissions in different parts of the basin to better match the satellite data. The averaging kernel sensitivities yield a DOFS value of 10.8 (consistent with the IMI preview), which gives us confidence in the inversion results both on the basin scale and in the spatial allocation within the basin. In particular, we see the need for more systematic increase in emissions in the Midland than the Delaware sub-basin.

Figure 7 shows the GEOS-Chem simulations for the inversion period with the prior and posterior emissions. The posterior simulation produces much higher methane concentrations over the Midland sub-basin, better matching the TROPOMI observations of Fig. 5. The mean GEOS-Chem-TROPOMI bias across the region of interest improves from -9.6 ppb in the reference simulation to -2.1 ppb in the posterior simulation, and the root mean square error (RMSE) improves from 14.1 to 11.2 ppb. A longer inversion would further decrease the bias and improve the RMSE.

We conducted the illustrative inversion presented here on an AWS EC2 c5.9xlarge instance with 36 CPUs and 500 GB of EBS storage. Table 3 shows the runtime for different components of the IMI workflow. Compute wall time was 10.7 h, with > 85 % of that time spent constructing the Jacobian matrix K. Our cost was USD 17 for an “on-demand” instance, in which the requested resources are made available almost immediately. A 1-year inversion would cost roughly USD 300 (12 × USD 17 = USD 204, plus the cost of additional EBS storage to accommodate the longer inversion period), and wall time could be reduced by requesting more CPUs at no additional cost since the charge is per CPU hour. Costs scale linearly with the area of the inversion domain and (for a fixed domain size) the number of state vector elements, again subject to changes in EBS storage needs. Performing additional inversions with different parameters and prior inventories (Table 2, Sect. 4.2) adds little cost because there is no need to reconstruct K. Data download and transfer between AWS services may incur some cost, but this is also minimal. A cheaper alternative to on-demand instances are “spot instances”, which tap unused EC2 capacity and can reduce costs by a factor of 3–4 or more (Zhuang et al., 2019). Spot instances can be reclaimed by AWS at any time, which would cause the IMI to crash, but in practice this is rare, and users can generally expect to retain a spot instance for up to a month of wall time (Pary, 2018).