Statistical inverse problems and uncertainty quantification

Statistical inverse problems refer to situations where we can get data in one domain, but our interest lies in another domain where measurements cannot be taken. This subject has a high-tech flavour and is the focus of much current research. Typical kinds of statistical inverse problems include nonparametric curve estimation like regression functions and densities. Often statistical inversion reformulates inverse problems as problems of statistical inference by means of Bayesian statistics. Uncertainty quantification is concerned with quantitative characterization and estimation of uncertainties in both computational and real world applications. It tries to determine how likely certain outcomes are if some aspects of the system are not exactly known.

Researchers

Prof.dr.ir. G. (Geurt) Jongbloed

Dr.ir. G.N.J.C. (Joris) Bierkens

Dr. H.N. (Hanne) Kekkonen

Prof.dr. A.W. (Aad) van der Vaart

Key Publications

2023 - Paul Dobson, Joris Bierkens - Stochastic Processes and their Applications

Infinite dimensional Piecewise Deterministic Markov Processes

In this paper we aim to construct infinite dimensional versions of well established Piecewise Deterministic Monte Carlo methods, such as the Bouncy Particle Sampler, the Zig-Zag Sampler and the Boomerang Sampler. In order to do so we provide an abstract infinite dimensional framework for Piecewise Deterministic Markov Processes (PDMPs) with unbounded event intensities. We further develop exponential convergence to equilibrium of the infinite dimensional Boomerang Sampler, using hypocoercivity techniques. Furthermore we establish how the infinite dimensional Boomerang Sampler admits a finite dimensional approximation, rendering it suitable for computer simulation.

2022 - Hanne Kekkonen - Inverse Problems

Consistency of Bayesian inference with Gaussian process priors for a parabolic inverse problem

We consider the statistical non-linear inverse problem of recovering the absorption term f > 0 in the heat equation

\(\begin{equation*}\begin{cases}{\partial }_{t}u-\frac{1}{2}{\Delta}u+fu=0\quad \hfill & \quad \text{on}\;\mathcal{O}\times (0,\mathbf{T})\hfill \\ u=g\quad \hfill & \quad \text{on}\;\partial \mathcal{O}\times (0,\mathbf{T})\hfill \\ u(\cdot ,0)={u}_{0}\quad \hfill & \quad \text{on}\;\mathcal{O},\hfill \end{cases}\end{equation*}\)

where \(\mathcal{O}\in {\mathbb{R}}^{d}\) is a bounded domain, \(T < ∞\) is a fixed time, and \(g, u_0\) are given sufficiently smooth functions describing boundary and initial values respectively. The data consists of N discrete noisy point evaluations of the solution uf on \(\mathcal{O}\times (0,\mathbf{T})\). We study the statistical performance of Bayesian nonparametric procedures based on a large class of Gaussian process priors. We show that, as the number of measurements increases, the resulting posterior distributions concentrate around the true parameter generating the data, and derive a convergence rate for the reconstruction error of the associated posterior means. We also consider the optimality of the contraction rates and prove a lower bound for the minimax convergence rate for inferring \(f\) from the data, and show that optimal rates can be achieved with truncated Gaussian priors.

2020 - Shota Gugushvili, Aad van der Vaart, Dong Yan - Annales de l'Institut Henri Poincaré, Probabilités et Statistiques

Bayesian linear inverse problems in regularity scales

We obtain rates of contraction of posterior distributions in inverse problems defined by scales of smoothness classes. We derive abstract results for general priors, with contraction rates determined by Galerkin approximation. The rate depends on the amount of prior concentration near the true function and the prior mass of functions with inferior Galerkin approximation. We apply the general result to non-conjugate series priors, showing that these priors give near optimal and adaptive recovery in some generality, Gaussian priors, and mixtures of Gaussian priors, where the latter are also shown to be near optimal and adaptive. The proofs are based on general testing and approximation arguments, without explicit calculations on the posterior distribution. We are thus not restricted to priors based on the singular value decomposition of the operator. We illustrate the results with examples of inverse problems resulting from differential equations.

2020 - Matteo Giordano, Hanne Kekkonen - SIAM/ASA Journal on Uncertainty Quantification

Bernstein--von Mises Theorems and Uncertainty Quantification for Linear Inverse Problems

We consider the statistical inverse problem of recovering an unknown function \(f\) from a linear measurement corrupted by additive Gaussian white noise. We employ a nonparametric Bayesian approach with standard Gaussian priors, for which the posterior-based reconstruction of \(f\) corresponds to a Tikhonov regularizer \(\bar f\) with a reproducing kernel Hilbert space norm penalty. We prove a semiparametric Bernstein--von Mises theorem for a large collection of linear functionals of \(f\), implying that semiparametric posterior estimation and uncertainty quantification are valid and optimal from a frequentist point of view. The result is applied to study three concrete examples that cover both the mildly and severely ill-posed cases: specifically, an elliptic inverse problem, an elliptic boundary value problem, and the heat equation. For the elliptic boundary value problem, we also obtain a nonparametric version of the theorem that entails the convergence of the posterior distribution to a prior-independent infinite-dimensional Gaussian probability measure with minimal covariance. As a consequence, it follows that the Tikhonov regularizer \(\bar f\) is an efficient estimator of \(f\), and we derive frequentist guarantees for certain credible balls centered at \(f\).
.

2020 - Geurt Jongbloed, Kimberly S. McGarrity, Jilt Sietsma - TEST

Smooth estimation of size distributions in an oriented cylinder model

Kernel estimators are proposed for estimating the cumulative distribution functions and the probability density functions of several quantities of interest in a stereological oriented cylinder model. This oriented cylinder model was developed to represent anisotropic microstructural features in materials. The asymptotic properties of these estimators are studied, and the estimators are applied to two banded dual phase steel microstructures. The estimation method is quite general and can also be applied to distributions of other univariate quantities of interest.

2019 - Richard Nickl, Jakob Söhl - Electronic Journal of Statistics

Bernstein–von Mises theorems for statistical inverse problems II: compound Poisson processes

We study nonparametric Bayesian statistical inference for the parameters governing a pure jump process of the form

\(Y_{t}=\sum_{k=1}^{N(t)}Z_{k},~~~t\ge 0,\)

where \(N(t)\) is a standard Poisson process of intensity \)\lambda\), and \(Z_k\) are drawn i.i.d. from jump measure \(\mu\). A high-dimensional wavelet series prior for the Lévy measure ν=λμ is devised and the posterior distribution arises from observing discrete samples \(Y_{\Delta},Y_{2\Delta},\dots,Y_{n\Delta}\) at fixed observation distance \(\Delta\), giving rise to a nonlinear inverse inference problem. We derive contraction rates in uniform norm for the posterior distribution around the true Lévy density that are optimal up to logarithmic factors over Hölder classes, as sample size n increases. We prove a functional Bernstein–von Mises theorem for the distribution functions of both \(\mu\) and \(\nu\), as well as for the intensity \(\lambda\), establishing the fact that the posterior distribution is approximated by an infinite-dimensional Gaussian measure whose covariance structure is shown to attain the information lower bound for this inverse problem. As a consequence posterior based inferences, such as nonparametric credible sets, are asymptotically valid and optimal from a frequentist point of view.

2015 - Piet Groeneboom, Geurt Jongbloed - The Annals of Statistics

Nonparametric confidence intervals for monotone functions

We study nonparametric isotonic confidence intervals for monotone functions. In [Ann. Statist. 29 (2001) 1699–1731], pointwise confidence intervals, based on likelihood ratio tests using the restricted and unrestricted MLE in the current status model, are introduced. We extend the method to the treatment of other models with monotone functions, and demonstrate our method with a new proof of the results of Banerjee–Wellner [Ann. Statist. 29 (2001) 1699–1731] and also by constructing confidence intervals for monotone densities, for which a theory remained be developed. For the latter model we prove that the limit distribution of the LR test under the null hypothesis is the same as in the current status model. We compare the confidence intervals, so obtained, with confidence intervals using the smoothed maximum likelihood estimator (SMLE), using bootstrap methods. The “Lagrange-modified” cusum diagrams, developed here, are an essential tool both for the computation of the restricted MLEs and for the development of the theory for the confidence intervals, based on the LR tests.

2013 - Jakob Söhl, Mathias Trabs - Electronic Journal of Statistics

A uniform central limit theorem and efficiency for deconvolution estimators

We estimate linear functionals in the classical deconvolution problem by kernel estimators. We obtain a uniform central limit theorem with \(\sqrt{n}\)–rate on the assumption that the smoothness of the functionals is larger than the ill–posedness of the problem, which is given by the polynomial decay rate of the characteristic function of the error. The limit distribution is a generalized Brownian bridge with a covariance structure that depends on the characteristic function of the error and on the functionals. The proposed estimators are optimal in the sense of semiparametric efficiency. The class of linear functionals is wide enough to incorporate the estimation of distribution functions. The proofs are based on smoothed empirical processes and mapping properties of the deconvolution operator.

2011 - B. T. Knapik, A. W. van der Vaart, J. H. van Zanten - The Annals of Statistics

Bayesian inverse problems with Gaussian priors

The posterior distribution in a nonparametric inverse problem is shown to contract to the true parameter at a rate that depends on the smoothness of the parameter, and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the minimax rate. The frequentist coverage of credible sets is shown to depend on the combination of prior and true parameter, with smoother priors leading to zero coverage and rougher priors to conservative coverage. In the latter case credible sets are of the correct order of magnitude. The results are numerically illustrated by the problem of recovering a function from observation of a noisy version of its primitive.