Metrics for assessing Linear Inverse Problems: a case study of a Trace Gas Inversion
Abstract. Multiple metrics have been proposed and utilized to assess the performance of linear Bayesian and geostatistical inverse problems. These metrics are mostly related to assessing reduction in prior uncertainties, comparing modeled observations to true observations, and checking distributional assumptions. These metrics though important should be augmented with sensitivity analysis to obtain a comprehensive understanding of the performance of inversions and critically improve confidence in the estimated fluxes. With this motivation, we derive analytical forms of the local sensitivities with respect to the number of inputs such as measurements, covariance parameters, covariates, and forward operator or jacobian. In addition to local sensitivity, we develop a framework for global sensitivity analysis that shows the apportionment of the uncertainty of different inputs to an inverse problem. The proposed framework is applicable to any other domain that employs linear Bayesian and geostatistical inverse methods. We show the application of our methodology in the context of an atmospheric inverse problem for estimating urban GHG emissions in Los Angeles. Within its context, we also propose a mathematical framework to construct correlation functions and components of uncertainty matrices from a pre-computed jacobian that encompasses non-stationary structures.
Vineet Yadav et al.
Vineet Yadav et al.
Vineet Yadav et al.
Viewed (geographical distribution)
This paper presents a series of quantities that can be derived from linear inverse theory. Put roughly they are the similarity of footprints (nearly the independence of rows of the Jacobian), the local sensitivity of the result to various inputs and finally a global sensitivity using a first-order Taylor expansion with respect to all inputs. The metrics are potentially useful and some are, to my knowledge, novel. The paper is potentially in scope for AMT though I think it needs more work to make it more relevant to likely readers.
My first problem with the paper is its title. the word "assessment" suggests some comment on the quality or robustness of an inversion. The authors don't do that and it's not clear from the paper that the generated metrics can do it. I'm unclear, for example, what new information is provided by the overlap of footprints. It might well mean that parts of the control space are under-sampled by the observations but the posterior uncertainty already tells us this. For the linear Gaussian case the posterior uncertainty can be calculated without measurements.
Likewise the sensitivity of the posterior estimate to the value of a given measurement is potentially useful as a warning flag for measurements that might have undue control on the outcome but it's not really developed. the global sensitivity analysis, which allows consideration of all inputs to the linear inverse problem, is potentially more interesting but again is not developed beyond generation of the first-order expansion. The example presents a good opportunity to demonstrate application of these methods but this is not taken far beyond calculation of the diagnostics.
I see two possibilities for the paper:
1) Repackage it as a technical note focusing on the calculation of the diagnostics
2) Extend the work to generate diagnostics of overall inversion performance, probably focusing on robustness.
L130: What is a footprint-induced probability distribution?
L190: The definition of the averaging kernel is true but this is an odd motivation for it, much better below when contemplating sensitivity of result to prior
Eq. 19: it's worth noting that this sensitivity is very close to the proportional uncertainty reduction A P^-1 and hence the averaging kernel. By the way I thank the authors for making me think hard enough about the relationship between AK, DOFS and uncertainty reduction to finally get an intuitive sense of it
L275: When commenting on covariances between H, Q etc we should also note that constraints like conservation of mass introduce strong covariances within the parameters of H. Covariances can only occur on physically plausible manifolds. This is a profoundly under-studied problem in transport modelling and there is probably great insight to be borrowed from Numerical Weather Prediction.
L431: not sure what the authors mean by aggregation error here. If they're truly commenting on temporal aggregation error they should cite DOI:10.5194/acp-11-3443-2011.