Robust multivariate and high-dimensional statistics

Multivariate statistics refers to methods that examine the simultaneous effect of multiple variables. Multivariate techniques differ from univariate and bivariate analyses in that they direct attention away from the analysis of the mean and variance of a single variable, or from the pairwise relationships between two variables, and involve the analysis of covariances or correlations that reflect the extent of relationship between three or more variables, and analysis of distances which reflect similarity among variables. Dependence multivariate methods usually seek to explain or predict one or more dependent variable (response, outcome variable(s)) based upon the set of predictor variables (independent , covariate, explanatory variables). Multivariate robust methods attempt to provide valid results when the data contain anomalies such as presence of outliers, while high-dimensional statistics is concerned with the models, where the number of variables/characteristics may tend to infinity together with the sample size.

Researchers

Dr. A.F.F. (Alexis) Derumigny

Dr. H.P. (Rik) Lopuhaä

Dr. N. (Nestor) Parolya

Prof.dr. A.W. (Aad) van der Vaart

Key Publications

2024 - Hendrik Paul Lopuhaä, Valerie Gares, Anne Ruiz-Gazen

S-estimation in Linear Models with Structured Covariance Matrices

We provide a unified approach to S-estimation in balanced linear models with structured covariance matrices. Of main interest are S-estimators for linear mixed effects models, but our approach also includes S-estimators in several other standard multivariate models, such as multiple regression, multivariate regression, and multivariate location and scatter. We provide sufficient conditions for the existence of S-functionals and S-estimators, establish asymptotic properties such as consistency and asymptotic normality, and derive their robustness properties in terms of breakdown point and influence function. All the results are obtained for general identifiable covariance structures and are established under mild conditions on the distribution of the observations, which goes far beyond models with elliptically contoured densities. Some of our results are new and others are more general than existing ones in the literature. In this way this manuscript completes and improves results on S-estimation in a wide variety of multivariate models. We illustrate our results by means of a simulation study and an application to data from a trial on the treatment of lead-exposed children.

2023 - Nestor Parolya, Johannes Heiny, Dorota Kurowicka - Bernoulli

Logarithmic law of large random correlation matrices

Consider a random vector y=Σ1∕2x, where the p elements of the vector x are i.i.d. real-valued random variables with zero mean and finite fourth moment, and Σ1∕2 is a deterministic p×p matrix such that the eigenvalues of the population correlation matrix R of y are uniformly bounded away from zero and infinity. In this paper, we find that the log determinant of the sample correlation matrix ˆR based on a sample of size n from the distribution of y satisfies a CLT (central limit theorem) for p∕n→γ∈(0,1] and p≤n. Explicit formulas for the asymptotic mean and variance are provided. In case the mean of y is unknown, we show that after re-centering by the empirical mean the obtained CLT holds with a shift in the asymptotic mean. This result is of independent interest in both large dimensional random matrix theory and high-dimensional statistical literature of large sample correlation matrices for non-normal data. Finally, the obtained findings are applied for testing of uncorrelatedness of p random variables. Surprisingly, in the null case R=I, the test statistic becomes distribution-free and the extensive simulations show that the obtained CLT also holds if the moments of order four do not exist at all, which conjectures a promising and robust test statistic for heavy-tailed high-dimensional data.

2021 - Taras Bodnar, Yarema Okhrin, Nestor Parolya - Journal of Business & Economic Statistics

Optimal Shrinkage-Based Portfolio Selection in High Dimensions

In this article, we estimate the mean-variance portfolio in the high-dimensional case using the recent results from the theory of random matrices. We construct a linear shrinkage estimator which is distribution-free and is optimal in the sense of maximizing with probability 1 the asymptotic out-of-sample expected utility, that is, mean-variance objective function for different values of risk aversion coefficient which in particular leads to the maximization of the out-of-sample expected utility and to the minimization of the out-of-sample variance. One of the main features of our estimator is the inclusion of the estimation risk related to the sample mean vector into the high-dimensional portfolio optimization. The asymptotic properties of the new estimator are investigated when the number of assets p and the sample size n tend simultaneously to infinity such that p/n→c∈(0,+∞). The results are obtained under weak assumptions imposed on the distribution of the asset returns, namely the existence of the 4+ε moments is only required. Thereafter we perform numerical and empirical studies where the small- and large-sample behavior of the derived estimator is investigated. The suggested estimator shows significant improvements over the existent approaches including the nonlinear shrinkage estimator and the three-fund portfolio rule, especially when the portfolio dimension is larger than the sample size. Moreover, it is robust to deviations from normality.

2018 - Alexis Derumigny - Electronic Journal of Statistics

Improved bounds for Square-Root Lasso and Square-Root Slope

Extending the results of Bellec, Lecué and Tsybakov [1] to the setting of sparse high-dimensional linear regression with unknown variance, we show that two estimators, the Square-Root Lasso and the Square-Root Slope can achieve the optimal minimax prediction rate, which is \((s/n)\log\left (p/s\right )\), up to some constant, under some mild conditions on the design matrix. Here, \(n\) is the sample size, \(p\) is the dimension and s is the sparsity parameter. We also prove optimality for the estimation error in the \(l_{q}\)-norm, with \(q\in[1,2]\) for the Square-Root Lasso, and in the \(l_2\) and sorted \(l_1\) norms for the Square-Root Slope. Both estimators are adaptive to the unknown variance of the noise. The Square-Root Slope is also adaptive to the sparsity \(s\) of the true parameter. Next, we prove that any estimator depending on s which attains the minimax rate admits an adaptive to s version still attaining the same rate. We apply this result to the Square-root Lasso. Moreover, for both estimators, we obtain valid rates for a wide range of confidence levels, and improved concentration properties as in [1] where the case of known variance is treated. Our results are non-asymptotic.

2017 - Stéphanie van der Pas, Botond Szabó, Aad van der Vaart - Bayesian Analysis

Uncertainty Quantification for the Horseshoe (with Discussion)

We investigate the credible sets and marginal credible intervals resulting from the horseshoe prior in the sparse multivariate normal means model. We do so in an adaptive setting without assuming knowledge of the sparsity level (number of signals). We consider both the hierarchical Bayes method of putting a prior on the unknown sparsity level and the empirical Bayes method with the sparsity level estimated by maximum marginal likelihood. We show that credible balls and marginal credible intervals have good frequentist coverage and optimal size if the sparsity level of the prior is set correctly. By general theory honest confidence sets cannot adapt in size to an unknown sparsity level. Accordingly the hierarchical and empirical Bayes credible sets based on the horseshoe prior are not honest over the full parameter space. We show that this is due to over-shrinkage for certain parameters and characterise the set of parameters for which credible balls and marginal credible intervals do give correct uncertainty quantification. In particular we show that the fraction of false discoveries by the marginal Bayesian procedure is controlled by a correct choice of cut-off.

2017 - Gwenaël G. R. Leday, Mathisca C. M. de Gunst, Gino B. Kpogbezan, Aad W. van der Vaart, Wessel N. van Wieringen, Mark A. van de Wiel - The Annals of Applied Statistics

Gene network reconstruction using global-local shrinkage priors

Reconstructing a gene network from high-throughput molecular data is an important but challenging task, as the number of parameters to estimate easily is much larger than the sample size. A conventional remedy is to regularize or penalize the model likelihood. In network models, this is often done locally in the neighborhood of each node or gene. However, estimation of the many regularization parameters is often difficult and can result in large statistical uncertainties. In this paper we propose to combine local regularization with global shrinkage of the regularization parameters to borrow strength between genes and improve inference. We employ a simple Bayesian model with nonsparse, conjugate priors to facilitate the use of fast variational approximations to posteriors. We discuss empirical Bayes estimation of hyperparameters of the priors, and propose a novel approach to rank-based posterior thresholding. Using extensive model- and data-based simulations, we demonstrate that the proposed inference strategy outperforms popular (sparse) methods, yields more stable edges, and is more reproducible. The proposed method, termed ShrinkNet, is then applied to Glioblastoma to investigate the interactions between genes associated with patient survival.

2017 - Alexis Derumigny, Jean-David Fermanian - Dependence Modeling

About tests of the “simplifying” assumption for conditional copulas

We discuss the so-called “simplifying assumption” of conditional copulas in a general framework. We introduce several tests of the latter assumption for non- and semiparametric copula models. Some related test procedures based on conditioning subsets instead of point-wise events are proposed. The limiting distributions of such test statistics under the null are approximated by several bootstrap schemes, most of them being new. We prove the validity of a particular semiparametric bootstrap scheme. Some simulations illustrate the relevance of our results.

2012 - Eric A. Cator, Hendrik P. Lopuhaä - Bernoulli

Central limit theorem and influence function for the MCD estimators at general multivariate distributions

We define the minimum covariance determinant functionals for multivariate location and scatter through trimming functions and establish their existence at any multivariate distribution. We provide a precise characterization including a separating ellipsoid property and prove that the functionals are continuous. Moreover, we establish asymptotic normality for both the location and covariance estimator and derive the influence function. These results are obtained in a very general multivariate setting.