Event Date
Speaker: Qiyang Han, Assistant Professor, Statistics, Rutgers University
Title: "Universality of regularized regression estimators in high dimensions"
Abstract: The Convex Gaussian Min-Max Theorem (CGMT) has emerged as a prominent theoretical tool for analyzing the precise stochastic behavior of various statistical estimators in the so-called high dimensional proportional regime, where the sample size and the signal dimension are of the same order. However, a well recognized limitation of the existing CGMT machinery rests in its stringent requirement on the exact Gaussianity of the design matrix, therefore rendering the obtained precise high dimensional asymptotics largely a specific Gaussian theory in various important statistical models.
This work provides a structural universality framework for a broad class of regularized regression estimators that is particularly compatible with the CGMT machinery. Here universality means that if a `structure' is satisfied by the regression estimator $\hat{\mu}_G$ for a standard Gaussian design $G$, then it will also be satisfied by $\hat{\mu}_A$ for a general non-Gaussian design $A$ with independent entries. In particular, we show that with a good enough $\ell_\infty$ bound for the regression estimator $\hat{\mu}_A$, any `structural property' that can be detected via the CGMT for $\hat{\mu}_G$ also holds for $\hat{\mu}_A$ under a general design $A$ with independent entries.
As a proof of concept, we demonstrate our new universality framework in three key examples of regularized regression estimators: the Ridge, Lasso and regularized robust regression estimators, where new universality properties of risk asymptotics and/or distributions of regression estimators and other related quantities are proved. As a major statistical implication of the Lasso universality results, we validate inference procedures using the degrees-of-freedom adjusted debiased Lasso under general design and error distributions. This talk is based on joint work with Yandi Shen (Chicago).
Bio: Qiyang Han is an assistant professor of Statistics at Rutgers University. He received his Ph.D. in Statistics in 2018 from University of Washington under the supervision of Professor Jon A. Wellner. He is broadly interested in mathematical statistics and high dimensional probability, with a particular focus on empirical process theory and its applications to nonparametric and high dimensional statistics. He is a recipient of the NSF CAREER award in 2022.
Faculty Webpage: http://stat.rutgers.edu/home/qh85/
_________________________________
Time/Date: Thursday, October 27, 2022 , 2:10pm (note earlier seminar time!)
Location: MSB 1147 (Colloquium Room)