TROPOMI observations are available from May 2018 to present. The current TROPOMI operational retrieval is posted with 2–3 d latency on the AWS cloud (presently version 02.06.00; Sentinel-5P Level 2 – Registry of Open Data on AWS, 2024; Lorente et al., 2023) and is accessed there by the IMI as default. The retrieval is regularly updated to resolve artifacts from surface albedo, aerosols, clouds, and cross-track detector differences (Lorente et al., 2023). We provide an option to use the blended TROPOMI+GOSAT dataset of Balasus et al. (2023), which applies machine learning to correct the TROPOMI version 02.06.00 retrieval with the more accurate but much sparser retrieval from the GOSAT satellite instrument (Parker et al., 2020). Glint observations over water were previously filtered out due to high biases and artifacts in the TROPOMI retrieval, but with the higher fidelity of the blended TROPOMI+GOSAT dataset we include an option to include these observations in the inversion. This can help to quantify offshore emissions, adding to the information from when the offshore emission plume is transported over land. The blended TROPOMI+GOSAT data are available on the AWS cloud for the full duration of the TROPOMI record and we keep them current for use in the IMI. Comparison of the blended TROPOMI+GOSAT and operational TROPOMI datasets through the IMI preview can be insightful for identifying retrieval artifacts.

Construction of the Jacobian matrix, K, is the most computationally expensive component of the IMI and has previously limited the size of the state vector to ∼2000 elements. K is constructed column by column by conducting GEOS-Chem simulations over the inversion time period, perturbing individual state vector elements, and collecting the resulting changes in concentrations. This was done in IMI 1.0 with separate GEOS-Chem simulations for each state vector element, and additional overhead was incurred by compiling methane emissions in GEOS-Chem with the Harmonized Emission Component (HEMCO; Lin et al., 2021). For IMI 2.0 we made several improvements to our Jacobian construction practices. To avoid the effect of small nonlinearities in the advection code (Lin and Rood, 1996), we construct our Jacobian columns by specifying a low methane background and initial conditions (1 ppb), applying a high relative perturbation such that the median emission is 10-8kgm-2s-1 for each grid cell in the perturbed state vector elements, and setting all other emission state vector elements to zero. To improve computational performance, we modified GEOS-Chem to represent methane emissions from multiple state vector elements in a single simulation as independently transported tracers so that several columns of the Jacobian can be constructed at once with no additional overhead. Additionally, we apply HEMCO to precompute total emissions for individual grid cells before running GEOS-Chem, reducing the time to read in emissions. Running GEOS-Chem with a large number of tracers can increase wall time because GEOS-Chem Classic simulations are limited to a single node (shared-memory parallelization), while multiple GEOS-Chem simulations for Jacobian construction can be spread across nodes. We optimized the number of tracers per GEOS-Chem simulation to balance the total CPU time (which decreases with the number of tracers) and the wall time (which increases with the number of tracers because the IMI then uses fewer compute nodes). Tests on the Harvard supercomputing cluster using 32 CPUs per GEOS-Chem simulation indicate an optimum of five tracers per simulation (Fig. 2). This yields a 5-fold speed-up in the construction of the Jacobian matrix relative to single-tracer simulations in total compute hours, traded against a 60 % increase in wall time, as shown in Fig. 2. Precompiling emissions with HEMCO yields an additional 2-fold speed-up, for an overall 10-fold decrease in CPU cost and a net decrease in wall time. Users with a large number of available nodes can reduce wall time at the expense of CPU time by choosing fewer tracers per simulation as specified in the configuration file (Table 2).

IMI 2.0 uses super-observations as the TROPOMI observation vector y in the inversion, following the work of Chen et al. (2023). Super-observations average all individual TROPOMI observations for a given GEOS-Chem grid cell and TROPOMI orbit. We call them super-observations following Eskes et al. (2003) because they have lower error than individual observations. Loss of information in this averaging of individual observations is negligible because GEOS-Chem model values are the same for all observations being averaged, and retrieval averaging kernels are similar. Using super-observations reduces the storage size and computational time associated with performing the inversion step of the IMI by reducing the dimension of y. Additionally, it provides a better characterization of observational error correlations to avoid overfit in the inversion. Using individual observations in the inversion with a diagonal observational error correlation matrix assumes that errors in individual observations are uncorrelated. In fact, transport errors for individual TROPOMI observations within a GEOS-Chem grid cell are perfectly correlated, and retrieval errors may be correlated as well. Using the residual error method (Heald et al., 2004), Chen et al. (2023) derived the observational error variance σsuper2 for a super-observation by averaging P individual TROPOMI observations: 5σsuper2=σretrieval21-rretrievalP+rretrieval+σtransport2, where σtransport2 is the error variance associated with GEOS-Chem transport, σretrieval2 is the single retrieval error variance, and rretrieval is the error correlation coefficient for the P observations being averaged. We use rretrieval=0.55 and σtransport=4.5ppb following Chen et al. (2023) for an inversion at 0.25°×0.3125° resolution. σretrieval is set by the user, with a default value of 15 ppb from previous applications of the residual error method to TROPOMI data (Qu et al., 2021; Shen et al., 2021; Chen et al., 2023). The improved observational error characterization from using super-observations reduces our reliance on γ to account for overfit, allowing γ to be set closer to 1.

It is critical for the IMI to use unbiased initial and boundary conditions (IC/BCs) because bias will otherwise propagate to the emission correction. IMI 1.0 used an archive of spatially and temporally smoothed TROPOMI fields as IC/BCs to avoid systematic bias and further optimized buffer clusters around the region of interest to correct for BC errors (Sect. 2). In IMI 2.0 we make four updates to the treatment of IC/BCs. First, we improve the process for generating smoothed TROPOMI fields to include a more accurate and synchronous stratosphere (Mooring et al., 2024). Second, we produce an additional parallel archive of smoothed fields for the blended TROPOMI+GOSAT product (Balasus et al., 2023). Third, we allow for the optimization of BCs as well as buffer clusters. Fourth, we conduct the smoothing backward in time (rather than centered in time) to allow for near-real-time (low-latency) applications. The archives of smoothed TROPOMI fields for both the operational product and the blended product are kept current with ∼1 month latency.

To produce the smoothed TROPOMI archives, we start from a global GEOS-Chem simulation (version 14.3.1) at a horizontal resolution of 2°×2.5°, improving upon the 4°×5° resolution of IMI 1.0, with 47 vertical layers extending to 0.01 hPa. The simulation uses the default prior emission estimates for the IMI (Table 3). It starts on 1 April 2018 (1 month before the start of the TROPOMI record) with ICs from a separate GEOS-Chem simulation initialized in 1985 (Mooring et al., 2024). This multidecadal spin-up simulation uses monthly interpolated global surface observations of methane concentrations from the NOAA GLOBALVIEW flask dataset as BCs (Murray, 2016). GEOS-Chem transports these surface BCs throughout the atmosphere, a process that takes years for the stratosphere (Chabrillat et al., 2018). In this manner, we create an 1 April 2018 IC that is consistent with both long-term trends in tropospheric methane and stratospheric transport (Mooring et al., 2024).

The 3-D methane concentration fields from our emissions-driven global simulation starting on 1 April 2018 from this unbiased synchronized atmosphere are archived every 3 h from 1 April 2018 until the present. We correct these fields to the TROPOMI satellite observations to generate the smoothed TROPOMI fields. For each observation, we apply the TROPOMI operator, which describes the sensitivity of the observation to different vertical levels, to the co-located GEOS-Chem vertical profile, giving us pairs of TROPOMI XCH4 and simulated GEOS-Chem XCH4. We average these XCH4 values on a 2°×2.5° daily grid and subtract them to get a GEOS-Chem column bias: ΔXCH4=XCH4,GEOS-Chem-XCH4,TROPOMI. The column-bias fields are then smoothed with a rolling average spatially to 10°×12.5° and temporally to 15 d back in time. If there are no observations for the past 15 d, we extend the temporal smoothing to 30 d. For grid cells with no column-bias information after smoothing (such as over open oceans or high latitudes in the winter), we use a zonal average of the column bias for that latitude band or for the closest latitude band where information is available. The resulting 2°×2.5° daily smoothed column-bias fields are removed from each of the 47 layers to yield the bias-corrected 3-D GEOS-Chem fields (called smoothed TROPOMI fields) used as IC/BCs in the inversions.

Although our smoothed TROPOMI fields are intended to avoid systematic IC/BC bias in inversions of TROPOMI data, there may still be error in the BCs not captured by the smoothing, particularly in areas with few observations (oceans, high latitudes). In IMI 1.0 this was corrected by optimizing emissions in buffer clusters surrounding the region of interest. In IMI 2.0 we implement a new option to allow optimization of the BCs as part of the inversion. When enabled, BC optimization adds four additional elements to the state vector, one for each edge of the GEOS-Chem domain (when using a custom shapefile the domain is padded to be rectilinear). Each edge is then optimized as a constant correction as part of the inversion. Optimization of BCs can be done in place of or in addition to optimization of buffer clusters. Recent work by Nesser et al. (2024b) finds that optimization of BCs is preferable to optimization of buffer clusters and has the advantage of being more physically based.

The maximum resolution of the IMI is set by the native 0.25°×0.3125° grid cell resolution of the GEOS-Chem forward model. Optimizing emissions at such a high resolution may be computationally burdensome for large domains and may not be justified by the information content of the observations. We introduce in IMI 2.0 an adaptive k-means clustering algorithm following Nesser et al. (2021) that clusters individual grid cells on the basis of proximity and information content. This maintains high resolution in areas of high emissions and dense observations while smoothing the solution in areas with weak emissions or insufficient observations. The algorithm is adaptive in determining the best clustering to apply given user specification of a desired number of state vector elements and minimum resolution, as well as in adjusting to changing observing conditions for temporally resolved inversions (Sect. 3.9). It is interactive with the user through the IMI preview. Users have options to exclude selected grid cells from the clustering (force them to remain at native resolution) on the basis of, for example, ancillary observations of large point sources (Sect. 3.6). To improve national emission accounting, users also have the option to ensure that clusters respect country boundaries.

Figure 3 illustrates our clustering algorithm with the example of the CONUS demo in Sect. 4. The user configures their desired number of state vector elements and maximum cluster size (2°×2.5° by default). The clustering algorithm first applies the IMI preview to estimate averaging kernel sensitivities, aii, (Eq. 4), for each native resolution grid cell in the domain on the basis of the number of super-observations and the prior emission estimates. Cells that have an aii greater than a clustering threshold are kept at native resolution (p=1) in the state vector. They may also be kept at native resolution if flagged by the user or by a point source dataset (Sect. 3.6). The clustering threshold has a default value of the DOFS divided by the desired number of state vector elements (DOFS / n), but this can be configured by the user. The algorithm then performs a k-means clustering of proximate grid cell pairs (p≈2) for the remaining lower-information grid cells on the basis of latitude, longitude, and aii and retains pairs in the state vector that exceed the clustering threshold for the sum of their estimated averaging kernel sensitivities. The procedure is repeated for p≈3, and so on. The iteration stops when either the desired number of state vector elements is reached, the maximum cluster size is reached, or continuing iteration would assign greater than the desired number of clusters. In the latter case, the remainder of the state vector is filled with elements of the maximum cluster size to achieve the desired number of state vector elements. At that point the final state vector is specified.

Users can determine a suitable number of state vector elements to balance information content and computational cost by running the IMI preview for varying state vector sizes and comparing the estimated DOFS (Fig. 4). The IMI preview visualizes the clustered state vector and the gridded aii of each state vector element for user inspection. By adjusting the clustering threshold and maximum cluster size with feedback from the IMI preview, users can effectively control the size distribution of state vector elements. For example, in the CONUS inversion (Sect. 4) the estimated information content over CONUS is bimodal, with many grid cells of relatively high information content and many with very low information content. This bimodal distribution causes too many grid cell clusters to be above the default clustering threshold (estimated DOFS/ n), leading to a premature filling of the grid with background clusters of the maximum cluster size to achieve a 600-element state vector. If left unchecked, this would lead to low-resolution background state vector elements having an outsized sensitivity to the observations. To fix this issue we apply a clustering threshold of 2.0. Alternatively, the user could increase the desired number of state vector elements, but the information content may not justify the added cost as illustrated in Fig. 4 with the DOFS asymptote.

We refer to point sources as emissions over 100 kgh-1 from individual facilities. Point source satellite imagers with high spatial resolution are variably able to detect plumes from individual point sources and quantify emissions, but integrating this growing dataset into inversions is challenging due to uncertainties in point source observability, variability, and persistence (Cardoso-Saldaña and Allen, 2020; Cusworth et al., 2020; Watine-Guiu et al., 2023). Possible methods include enforcing high state vector resolution where point sources have been observed (Chen et al., 2023), introducing additional point source state vector elements (Naus et al., 2023), and using point source data to evaluate posterior results in a TROPOMI-only inversion (Cusworth et al., 2020). Here, we integrate point source information into the IMI when constructing a reduced state vector (Sect. 3.5) by enforcing native resolution for state vector elements with point sources above a specified threshold (default of>2500kgh-1) for a minimum number of repeated observations (default of 50) in the continually updated global datasets from SRON (Schuit et al., 2023; Methane Plume Maps, 2024), IMEO (International Methane Emissions Observatory, 2024), and Carbon Mapper (Cusworth et al., 2024). These datasets include satellite point source information from TROPOMI, Sentinel-2/3/5, EMIT, PRISMA, EnMAP, GOES, and Landsat-8/9. Point source locations are included as an overlay on the prior emission maps in the IMI preview.

IMI 1.0 only optimized methane emissions, but sinks also affect the methane concentrations. Global inversions commonly optimize tropospheric OH (the main methane sink) in the state vector as a methane loss frequency separately from emissions (Maasakkers et al., 2019; Yin et al., 2021). A common misconception is that regional inversions (either Eulerian or Lagrangian) do not need to optimize OH because the ventilation timescale is much shorter than the methane lifetime (Sheng et al., 2018), but the exact same consideration would apply to the effect of emissions. In IMI 2.0 we provide the option to optimize the tropospheric OH concentration (as the methane loss frequency) averaged over the regional domain or as hemispheric quantities for global inversions (Sect. 3.8). Prior estimates are the 4°×5° global 3-D monthly fields of OH concentrations used in GEOS-Chem (Wecht et al., 2014). These fields yield a global methane lifetime of 10.7 years against tropospheric OH, within the observationally constrained range of 11.2±1.3 years from proxy observations (Prather et al., 2012). Users can modify the default GEOS-Chem configuration with alternative OH prior estimates if desired. The default prior error standard deviation for the inversion is 10 % (Zhang et al., 2018) but can be modified by the user in the configuration file. We find in our default regional inversions that optimization of tropospheric OH is effectively done by optimization of the BCs, rather than by optimization of OH itself, so that the methane loss frequency does not need to be in the state vector. This is because the default OH concentration field is relatively smooth. Users may want to include OH in their regional inversions if they replace the default OH concentration field or if they modify the default prior error variances to have lower errors in BCs and/or higher errors in OH.

IMI 2.0 supports global inversions with optimization of both methane emissions at up to 2°×2.5° resolution and hemispheric OH concentrations (two additional state vector elements), following Qu et al. (2021). The same analytical inversion method used in regional inversions is applied globally. The global inversion is necessarily coarser than regional inversions. The Kalman filter approach (Sect. 3.9) for time-dependent optimization of emissions can be used.

A critical requirement for global inversions is unbiased initial conditions. The smoothed TROPOMI archive described in Sect. 3.4 is effective for this purpose. It provides an unbiased synchronous representation of the stratosphere at the initial time that can then evolve with the forward simulation.

IMI 2.0 enables temporal emission updates and continuous emission monitoring at low latency with a Kalman filter approach as described by Varon et al. (2023). This workflow is depicted in Fig. 5. The IMI conducts successive methane inversions over user-specified time intervals (such as weekly or monthly) using observations for that interval. Prior emissions are taken as the posterior emissions of the preceding interval (Kalman filter), the original bottom-up estimates, or a combination of the two through a nudge factor α (α=0 for the preceding interval, α=1 for the original bottom-up estimate). Relative error standard deviations for the prior emissions are not updated from their original values (50 % in the default) because of unresolved prior error in the temporal variability of emissions. Varon et al. (2023) used α=0.1 in their weekly inversions for the Permian Basin to retain some information from the original prior distribution and to prevent state vector elements from getting locked at low values when a fixed relative error in the prior emission is assumed. The state vector can be updated from one interval to the next with the adaptive k-means clustering scheme (Sect. 3.5) to take advantage of changing spatial patterns in information content. The latency of temporal emission updates is dependent on the availability of the IMI input data. At present the latency is about 1 month behind real time, limited by availability of the GEOS meteorological data to drive GEOS-Chem.

Table 3 lists the default bottom-up inventories used as prior estimates in IMI 2.0. Users may add or substitute their own prior estimates as NetCDF files to be read through HEMCO (Lin et al., 2021). New bottom-up inventories introduced in IMI 2.0 include worldwide emissions from hydroelectric reservoirs (Delwiche et al., 2022), an updated gridded version of US emissions from the EPA Greenhouse Gas Inventory (Maasakkers et al., 2023), and updated global anthropogenic emissions from EDGAR v8 (Crippa et al., 2023). Bottom-up anthropogenic inventories are reported with lag times of a few years or more and we use as default the most recent reported year, but this is of little consequence because most reported anthropogenic emissions and their distributions generally vary little from year to year. Wetland emissions can have large year-to-year variability but also large uncertainties, so we use as default the monthly emissions from the mean of the WetCHARTs inventory ensemble for individual years (Bloom et al., 2021), which users can replace by monthly WetCHARTs or LPJ-wsl climatologies. GFED4 open fire emissions have daily temporal resolution and are for individual years.

IMI 2.0 includes an option to use lognormal error PDFs for the prior emission estimates. The frequency distribution of methane emissions often exhibits heavy tails that can be better characterized using lognormal errors rather than normal errors (Yuan et al., 2015; Cui et al., 2019). An added benefit of using lognormal errors is that it enforces positivity in the posterior solution (Miller et al., 2014), considering that we do not optimize the soil sink in the IMI. Following Maasakkers et al. (2019), we implement lognormal error PDFs for the prior emission estimates by optimizing for the logarithm of the emissions and otherwise using the same equations. The forward model relating methane concentrations to the logarithm of emissions is nonlinear, requiring an iterative approach to find the solution through repeated update of the Jacobian matrix K (Rodgers, 2000). Updates to K in log space are readily computed through simple scaling without having to re-run GEOS-Chem (Maasakkers et al., 2019). The iterative nature of the solution increases the computational cost dependent on the number of iterations needed to reach convergence. Only the prior emission elements of the state vector in the region of interest are optimized in log space; buffer elements, boundary conditions, and OH concentrations continue to be optimized with normal errors (Maasakkers et al., 2019; Chen et al., 2022).

When using the lognormal error PDF option, the inversion optimizes the median of the posterior PDF, starting from the median of the prior PDF. But the prior estimates reported in Table 3 should be viewed as the means of their PDFs, with a geometric error standard deviation σg (for example, σg=2 states a factor of 2 uncertainty). The median of a lognormal PDF is related to its mean by xmedian=xmeanexp⁡[-(ln⁡σg2)/2], and we apply this correction to the prior estimates for input to the inversion. The inversion then returns the median of the posterior PDF with posterior error covariance matrix in log space. We convert the median to the mean of the posterior PDF with geometric error standard deviations by following the reverse of the above procedure. See Hancock et al. (2025) for further details on the method.

IMI 1.0 output included gridded posterior emission estimates with error standard deviations, averaging kernel sensitivities, and tables of emission totals. It also compared the GEOS-Chem simulations with posterior versus prior emissions to the TROPOMI observations as a diagnostic of the improved fit resulting from the inversion, with means and spatial standard deviations of the time-averaged GEOS-Chem–TROPOMI differences as indicators. Here we add tabulated and gridded posterior emissions by source sectors, using the prior sectoral contributions within individual grid cells to assign corrections from the inversion to individual sectors (Wecht et al., 2014). This assumes that the relative contributions from different sectors within individual state vector elements are correct, which is a better assumption at high resolution because emissions within a given state vector element are then more likely to be dominated by a single sector. This also assumes that the prior error is the same for all sectors, which could be improved by using sector-dependent prior error variances (Shen et al., 2021). Sectoral output can be further partitioned by individual countries within the inversion domain. We also provide time series of emissions when using temporal emission updates (Sect. 3.9). Output for the IMI preview now includes gridded visualization of both TROPOMI datasets (operational and blended), overlay of point source locations (Sect. 3.6) on the map of prior emissions, and visualization of estimated averaging kernel sensitivities. The averaging kernel sensitivities are estimated in the preview using the density of super-observations (Sect. 3.3) with a modified version of the IMI 1.0 formula in Eq. (4): 6aii=σai2σai2+(σsuper,i/k)2msuper,i, where the variables are the same as in Eq. (4) except σsuper,i and msuper,i. σsuper,i is the observational error standard deviation for state vector element i calculated as in Eq. (5) using the average number of grid cell observations, P, within that element over the inversion time period. msuper,i is the number of super-observations for state vector element i over the inversion time period. The IMI configuration file is also written to the output directory for user reference.

IMI 2.0 includes a software container to facilitate installation of the IMI to local systems and support scheduled inversion workflows (e.g., once a week or once a month). A container is a lightweight, stand-alone, and executable software package that encapsulates an application and all its dependencies, including libraries, frameworks, and system tools. Creating an IMI container provides a stable and reproducible environment. It ensures that the IMI can run consistently across different systems, such as local clusters, cloud servers, and even local computers regardless of operating system. Once the container is downloaded, the only dependency needed to run the software is Docker^® (a container engine). The necessary input data are automatically downloaded from the AWS cloud upon running the IMI container.

The IMI container is built in two stages: a base container and an operational container. The base container builds the environment and dependencies needed to run the IMI (Python packages, forward model dependencies, system tools). The build of the base environment is performed with two commonly used scientific package managers, Spack™ and Micromamba™. The operational container build simply downloads and configures the IMI source code into the container. The operational container is built automatically via GitHub Actions upon new IMI version releases and archived on a publicly accessible cloud repository with download instructions on the IMI documentation site.

An application of the containerized IMI is to support the US GHG Center, a multi-agency initiative to provide a trusted repository of greenhouse gas data from models and observations on the AWS cloud (U.S. Greenhouse Gas Center, 2024). The US GHG Center uses a Multi-Mission Algorithm and Analysis Platform (MAAP; Earthdata, 2024) cloud environment designed to ingest containers. The IMI operates within MAAP to provide an inversion tool as part of the US GHG Center capabilities.