Search Results for author:
Found 56 papers, 10 papers with code
Double Cross-fit Doubly Robust Estimators: Beyond Series Regression
Then, assuming the nuisance functions are H\"{o}lder smooth, but without assuming knowledge of the true smoothness level or the covariate density, we establish that DCDR estimators with several linear smoothers are semiparametric efficient under minimal conditions and achieve fast convergence rates in the non-$\sqrt{n}$ regime.
Semi-Supervised U-statistics
Semi-supervised datasets are ubiquitous across diverse domains where obtaining fully labeled data is costly or time-consuming.
Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shift
Self-training and contrastive learning have emerged as leading techniques for incorporating unlabeled data, both under distribution shift (unsupervised domain adaptation) and when it is absent (semi-supervised learning).
The Fundamental Limits of Structure-Agnostic Functional Estimation
These first-order methods are however provably suboptimal in a minimax sense for functional estimation when the nuisance functions live in Holder-type function spaces.
RLSbench: Domain Adaptation Under Relaxed Label Shift
Despite the emergence of principled methods for domain adaptation under label shift, their sensitivity to shifts in class conditional distributions is precariously under explored.
Domain Adaptation under Missingness Shift
Rates of missing data often depend on record-keeping policies and thus may change across times and locations, even when the underlying features are comparatively stable.
Domain Adaptation under Open Set Label Shift
We introduce the problem of domain adaptation under Open Set Label Shift (OSLS) where the label distribution can change arbitrarily and a new class may arrive during deployment, but the class-conditional distributions p(x|y) are domain-invariant.
Leveraging Unlabeled Data to Predict Out-of-Distribution Performance
Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions that may cause performance drops.
Minimax Optimal Regression over Sobolev Spaces via Laplacian Eigenmaps on Neighborhood Graphs
We also show that PCR-LE is \emph{manifold adaptive}: that is, we consider the situation where the design is supported on a manifold of small intrinsic dimension $m$, and give upper bounds establishing that PCR-LE achieves the faster minimax estimation ($n^{-2s/(2s + m)}$) and testing ($n^{-4s/(4s + m)}$) rates of convergence.
Mixture Proportion Estimation and PU Learning: A Modern Approach
Formally, this task is broken down into two subtasks: (i) Mixture Proportion Estimation (MPE) -- determining the fraction of positive examples in the unlabeled data; and (ii) PU-learning -- given such an estimate, learning the desired positive-versus-negative classifier.
Heavy-tailed Streaming Statistical Estimation
We consider the task of heavy-tailed statistical estimation given streaming $p$-dimensional samples.
Plugin Estimation of Smooth Optimal Transport Maps
Our work also provides new bounds on the risk of corresponding plugin estimators for the quadratic Wasserstein distance, and we show how this problem relates to that of estimating optimal transport maps using stability arguments for smooth and strongly convex Brenier potentials.
Minimax Optimal Regression over Sobolev Spaces via Laplacian Regularization on Neighborhood Graphs
In this paper we study the statistical properties of Laplacian smoothing, a graph-based approach to nonparametric regression.
Mixture Proportion Estimation and PU Learning:A Modern Approach
Formally, this task is broken down into two subtasks: (i) Mixture Proportion Estimation (MPE)---determining the fraction of positive examples in the unlabeled data; and (ii) PU-learning---given such an estimate, learning the desired positive-versus-negative classifier.
RATT: Leveraging Unlabeled Data to Guarantee Generalization
To assess generalization, machine learning scientists typically either (i) bound the generalization gap and then (after training) plug in the empirical risk to obtain a bound on the true risk; or (ii) validate empirically on holdout data.
On Proximal Policy Optimization's Heavy-tailed Gradients
In this paper, we present a detailed empirical study to characterize the heavy-tailed nature of the gradients of the PPO surrogate reward function.
Efficient Estimators for Heavy-Tailed Machine Learning
A dramatic improvement in data collection technologies has aided in procuring massive amounts of unstructured and heterogeneous datasets.
On Learning Ising Models under Huber's Contamination Model
We study the problem of learning Ising models in a setting where some of the samples from the underlying distribution can be arbitrarily corrupted.
Two-Sample Testing on Ranked Preference Data and the Role of Modeling Assumptions
We also provide testing algorithms and associated sample complexity bounds for the problem of two-sample testing with partial (or total) ranking data. Furthermore, we empirically evaluate our results via extensive simulations as well as two real-world datasets consisting of pairwise comparisons.
A Unified View of Label Shift Estimation
Our contributions include (i) consistency conditions for MLLS, which include calibration of the classifier and a confusion matrix invertibility condition that BBSE also requires; (ii) a unified framework, casting BBSE as roughly equivalent to MLLS for a particular choice of calibration method; and (iii) a decomposition of MLLS's finite-sample error into terms reflecting miscalibration and estimation error.
Universal Inference
Constructing tests and confidence sets for such models is notoriously difficult.
Gaussian Mixture Clustering Using Relative Tests of Fit
We consider clustering based on significance tests for Gaussian Mixture Models (GMMs).
Minimax Confidence Intervals for the Sliced Wasserstein Distance
To motivate the choice of these classes, we also study minimax rates of estimating a distribution under the Sliced Wasserstein distance.
Path Length Bounds for Gradient Descent and Flow
We derive bounds on the path length $\zeta$ of gradient descent (GD) and gradient flow (GF) curves for various classes of smooth convex and nonconvex functions.
A Unified Approach to Robust Mean Estimation
Building on this connection, we provide a simple variant of recent computationally-efficient algorithms for mean estimation in Huber's model, which given our connection entails that the same efficient sample-pruning based estimators is simultaneously robust to heavy-tailed noise and Huber contamination.
How Many Samples are Needed to Estimate a Convolutional Neural Network?
We show that for an $m$-dimensional convolutional filter with linear activation acting on a $d$-dimensional input, the sample complexity of achieving population prediction error of $\epsilon$ is $\widetilde{O(m/\epsilon^2)$, whereas the sample-complexity for its FNN counterpart is lower bounded by $\Omega(d/\epsilon^2)$ samples.
Nonparametric Regression with Comparisons: Escaping the Curse of Dimensionality with Ordinal Information
Finally, we present experiments that show the efficacy of RR and investigate its robustness to various sources of noise and model-misspecification.
Regression with Comparisons: Escaping the Curse of Dimensionality with Ordinal Information
In supervised learning, we typically leverage a fully labeled dataset to design methods for function estimation or prediction.
Robust Nonparametric Regression under Huber's $ε$-contamination Model
We first show that a simple local binning median step can effectively remove the adversary noise and this median estimator is minimax optimal up to absolute constants over the H\"{o}lder function class with smoothness parameters smaller than or equal to 1.
How Many Samples are Needed to Estimate a Convolutional or Recurrent Neural Network?
It is widely believed that the practical success of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) owes to the fact that CNNs and RNNs use a more compact parametric representation than their Fully-Connected Neural Network (FNN) counterparts, and consequently require fewer training examples to accurately estimate their parameters.
Local White Matter Architecture Defines Functional Brain Dynamics
If communication efficiency is fundamentally constrained by the integrity along the entire length of a white matter bundle, then variability in the functional dynamics of brain networks should be associated with variability in the local connectome.
Optimization of Smooth Functions with Noisy Observations: Local Minimax Rates
In this setup, an algorithm is allowed to adaptively query the underlying function at different locations and receives noisy evaluations of function values at the queried points (i. e. the algorithm has access to zeroth-order information).
Robust Estimation via Robust Gradient Estimation
We provide a new computationally-efficient class of estimators for risk minimization.
Hypothesis Testing for High-Dimensional Multinomials: A Selective Review
The statistical analysis of discrete data has been the subject of extensive statistical research dating back to the work of Pearson.
Stochastic Zeroth-order Optimization in High Dimensions
We consider the problem of optimizing a high-dimensional convex function using stochastic zeroth-order queries.
Low Permutation-rank Matrices: Structural Properties and Noisy Completion
We consider the problem of noisy matrix completion, in which the goal is to reconstruct a structured matrix whose entries are partially observed in noise.
Hypothesis Testing For Densities and High-Dimensional Multinomials: Sharp Local Minimax Rates
In contrast to existing results, we show that the minimax rate and critical testing radius in these settings depend strongly, and in a precise way, on the null distribution being tested and this motivates the study of the (local) minimax rate as a function of the null distribution.
Computationally Efficient Robust Estimation of Sparse Functionals
Many conventional statistical procedures are extremely sensitive to seemingly minor deviations from modeling assumptions.
Rate Optimal Estimation and Confidence Intervals for High-dimensional Regression with Missing Covariates
We consider the problems of estimation and of constructing component-wise confidence intervals in a sparse high-dimensional linear regression model when some covariates of the design matrix are missing completely at random.
Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequences
Our first main result shows that the population likelihood function has bad local maxima even in the special case of equally-weighted mixtures of well-separated and spherical Gaussians.
A Permutation-based Model for Crowd Labeling: Optimal Estimation and Robustness
The task of aggregating and denoising crowd-labeled data has gained increased significance with the advent of crowdsourcing platforms and massive datasets.
Arbitrage-Free Combinatorial Market Making via Integer Programming
We present a new combinatorial market maker that operates arbitrage-free combinatorial prediction markets specified by integer programs.
Statistical Inference for Cluster Trees
A cluster tree provides a highly-interpretable summary of a density function by representing the hierarchy of its high-density clusters.
Feeling the Bern: Adaptive Estimators for Bernoulli Probabilities of Pairwise Comparisons
Second, we show that a regularized least squares estimator can achieve a poly-logarithmic adaptivity index, thereby demonstrating a $\sqrt{n}$-gap between optimal and computationally achievable adaptivity.
Statistical and Computational Guarantees for the Baum-Welch Algorithm
By exploiting this characterization, we provide non-asymptotic finite sample guarantees on the Baum-Welch updates, guaranteeing geometric convergence to a small ball of radius on the order of the minimax rate around a global optimum.
Stochastically Transitive Models for Pairwise Comparisons: Statistical and Computational Issues
On the other hand, unlike in the BTL and Thurstone models, computing the minimax-optimal estimator in the stochastically transitive model is non-trivial, and we explore various computationally tractable alternatives.
Estimation from Pairwise Comparisons: Sharp Minimax Bounds with Topology Dependence
Data in the form of pairwise comparisons arises in many domains, including preference elicitation, sporting competitions, and peer grading among others.
Statistical guarantees for the EM algorithm: From population to sample-based analysis
Leveraging this characterization, we then provide non-asymptotic guarantees on the EM and gradient EM algorithms when applied to a finite set of samples.
When is it Better to Compare than to Score?
When eliciting judgements from humans for an unknown quantity, one often has the choice of making direct-scoring (cardinal) or comparative (ordinal) measurements.
Tight Lower Bounds for Homology Inference
In this note we use a different construction based on the direct analysis of the likelihood ratio test to show that the upper bound of Niyogi, Smale and Weinberger is in fact tight, thus establishing rate optimal asymptotic minimax bounds for the problem.
Cluster Trees on Manifolds
In this paper we investigate the problem of estimating the cluster tree for a density $f$ supported on or near a smooth $d$-dimensional manifold $M$ isometrically embedded in $\mathbb{R}^D$.
Confidence sets for persistence diagrams
Persistent homology is a method for probing topological properties of point clouds and functions.
Optimal kernel choice for large-scale two-sample tests
A means of parameter selection for the two-sample test based on the MMD is proposed.
Recovering Block-structured Activations Using Compressive Measurements
We consider the problems of detection and localization of a contiguous block of weak activation in a large matrix, from a small number of noisy, possibly adaptive, compressive (linear) measurements.
Minimax Localization of Structural Information in Large Noisy Matrices
We consider the problem of identifying a sparse set of relevant columns and rows in a large data matrix with highly corrupted entries.
Noise Thresholds for Spectral Clustering
Although spectral clustering has enjoyed considerable empirical success in machine learning, its theoretical properties are not yet fully developed.
