Search Results for author: Vidya Muthukumar

Found 26 papers, 3 papers with code

Precise asymptotics of reweighted least-squares algorithms for linear diagonal networks

no code implementations4 Jun 2024 Chiraag Kaushik, Justin Romberg, Vidya Muthukumar

The classical iteratively reweighted least-squares (IRLS) algorithm aims to recover an unknown signal from linear measurements by performing a sequence of weighted least squares problems, where the weights are recursively updated at each step.

Sharp analysis of out-of-distribution error for "importance-weighted" estimators in the overparameterized regime

no code implementations10 May 2024 Kuo-Wei Lai, Vidya Muthukumar

Overparameterized models that achieve zero training error are observed to generalize well on average, but degrade in performance when faced with data that is under-represented in the training sample.

Just Wing It: Optimal Estimation of Missing Mass in a Markovian Sequence

1 code implementation8 Apr 2024 Ashwin Pananjady, Vidya Muthukumar, Andrew Thangaraj

Operating in the general setting in which the size of the state space may be much larger than the length $n$ of the trajectory, we develop a linear-runtime estimator called \emph{Windowed Good--Turing} (\textsc{WingIt}) and show that its risk decays as $\widetilde{\mathcal{O}}(\mathsf{T_{mix}}/n)$, where $\mathsf{T_{mix}}$ denotes the mixing time of the chain in total variation distance.

Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance

no code implementations18 Feb 2024 Chiraag Kaushik, Ran Liu, Chi-Heng Lin, Amrit Khera, Matthew Y Jin, Wenrui Ma, Vidya Muthukumar, Eva L Dyer

Classification models are expected to perform equally well for different classes, yet in practice, there are often large gaps in their performance.

Data Augmentation

Towards Last-layer Retraining for Group Robustness with Fewer Annotations

1 code implementation NeurIPS 2023 Tyler LaBonte, Vidya Muthukumar, Abhishek Kumar

In this work, we examine this impractical requirement and find that last-layer retraining can be surprisingly effective with no group annotations (other than for model selection) and only a handful of class annotations.

Model Selection

Faster Margin Maximization Rates for Generic and Adversarially Robust Optimization Methods

no code implementations NeurIPS 2023 Guanghui Wang, Zihao Hu, Claudio Gentile, Vidya Muthukumar, Jacob Abernethy

To address this limitation, we present a series of state-of-the-art implicit bias rates for mirror descent and steepest descent algorithms.

Binary Classification

New Equivalences Between Interpolation and SVMs: Kernels and Structured Features

no code implementations3 May 2023 Chiraag Kaushik, Andrew D. McRae, Mark A. Davenport, Vidya Muthukumar

The support vector machine (SVM) is a supervised learning algorithm that finds a maximum-margin linear classifier, often after mapping the data to a high-dimensional feature space via the kernel trick.

General Loss Functions Lead to (Approximate) Interpolation in High Dimensions

no code implementations13 Mar 2023 Kuo-Wei Lai, Vidya Muthukumar

We provide a unified framework, applicable to a general family of convex losses and across binary and multiclass settings in the overparameterized regime, to approximately characterize the implicit bias of gradient descent in closed form.

Vocal Bursts Intensity Prediction

Estimating Optimal Policy Value in General Linear Contextual Bandits

no code implementations19 Feb 2023 Jonathan N. Lee, Weihao Kong, Aldo Pacchiano, Vidya Muthukumar, Emma Brunskill

Whether this is possible for more realistic context distributions has remained an open and important question for tasks such as model selection.

Model Selection Multi-Armed Bandits

Adaptive Oracle-Efficient Online Learning

no code implementations17 Oct 2022 Guanghui Wang, Zihao Hu, Vidya Muthukumar, Jacob Abernethy

The classical algorithms for online learning and decision-making have the benefit of achieving the optimal performance guarantees, but suffer from computational complexity limitations when implemented at scale.

Decision Making

Harmless interpolation in regression and classification with structured features

no code implementations9 Nov 2021 Andrew D. McRae, Santhosh Karnik, Mark A. Davenport, Vidya Muthukumar

Our results recover prior independent-features results (with a much simpler analysis), but they furthermore show that harmless interpolation can occur in more general settings such as features that are a bounded orthonormal system.

Classification regression

Universal and data-adaptive algorithms for model selection in linear contextual bandits

no code implementations8 Nov 2021 Vidya Muthukumar, Akshay Krishnamurthy

In this paper, we introduce new algorithms that a) explore in a data-adaptive manner, and b) provide model selection guarantees of the form $\mathcal{O}(d^{\alpha} T^{1- \alpha})$ with no feature diversity conditions whatsoever, where $d$ denotes the dimension of the linear model and $T$ denotes the total number of rounds.

Diversity Model Selection +1

Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective

no code implementations27 Sep 2021 Adhyyan Narang, Vidya Muthukumar, Anant Sahai

We find that the learned model is susceptible to adversaries in an intermediate regime where classification generalizes but regression does not.

A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning

no code implementations6 Sep 2021 Yehuda Dar, Vidya Muthukumar, Richard G. Baraniuk

The rapid recent progress in machine learning (ML) has raised a number of scientific questions that challenge the longstanding dogma of the field.

Learning from an Exploring Demonstrator: Optimal Reward Estimation for Bandits

1 code implementation28 Jun 2021 Wenshuo Guo, Kumar Krishna Agrawal, Aditya Grover, Vidya Muthukumar, Ashwin Pananjady

We introduce the "inverse bandit" problem of estimating the rewards of a multi-armed bandit instance from observing the learning process of a low-regret demonstrator.

Experimental Design

Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolation

no code implementations NeurIPS 2021 Ke Wang, Vidya Muthukumar, Christos Thrampoulidis

The literature on "benign overfitting" in overparameterized models has been mostly restricted to regression or binary classification; however, modern machine learning operates in the multiclass setting.

Binary Classification Classification

On the Impossibility of Convergence of Mixed Strategies with No Regret Learning

no code implementations3 Dec 2020 Vidya Muthukumar, Soham Phade, Anant Sahai

We study the limiting behavior of the mixed strategies that result from optimal no-regret learning strategies in a repeated game setting where the stage game is any 2 by 2 competitive game.

Online Model Selection for Reinforcement Learning with Function Approximation

no code implementations19 Nov 2020 Jonathan N. Lee, Aldo Pacchiano, Vidya Muthukumar, Weihao Kong, Emma Brunskill

Towards this end, we consider the problem of model selection in RL with function approximation, given a set of candidate RL algorithms with known regret guarantees.

Model Selection reinforcement-learning +1

On the proliferation of support vectors in high dimensions

no code implementations22 Sep 2020 Daniel Hsu, Vidya Muthukumar, Ji Xu

The support vector machine (SVM) is a well-established classification method whose name refers to the particular training examples, called support vectors, that determine the maximum margin separating hyperplane.

General Classification Vocal Bursts Intensity Prediction

OSOM: A simultaneously optimal algorithm for multi-armed and linear contextual bandits

no code implementations24 May 2019 Niladri S. Chatterji, Vidya Muthukumar, Peter L. Bartlett

We consider the stochastic linear (multi-armed) contextual bandit problem with the possibility of hidden simple multi-armed bandit structure in which the rewards are independent of the contextual information.

Multi-Armed Bandits

Harmless interpolation of noisy data in regression

no code implementations21 Mar 2019 Vidya Muthukumar, Kailas Vodrahalli, Vignesh Subramanian, Anant Sahai

A continuing mystery in understanding the empirical success of deep neural networks is their ability to achieve zero training error and generalize well, even when the training data is noisy and there are more parameters than data points.

regression

Understanding Unequal Gender Classification Accuracy from Face Images

no code implementations30 Nov 2018 Vidya Muthukumar, Tejaswini Pedapati, Nalini Ratha, Prasanna Sattigeri, Chai-Wah Wu, Brian Kingsbury, Abhishek Kumar, Samuel Thomas, Aleksandra Mojsilovic, Kush R. Varshney

Recent work shows unequal performance of commercial face classification services in the gender classification task across intersectional groups defined by skin type and gender.

Classification Gender Classification +1

Best of many worlds: Robust model selection for online supervised learning

no code implementations22 May 2018 Vidya Muthukumar, Mitas Ray, Anant Sahai, Peter L. Bartlett

We introduce algorithms for online, full-information prediction that are competitive with contextual tree experts of unknown complexity, in both probabilistic and adversarial settings.

Model Selection

Worst-case vs Average-case Design for Estimation from Fixed Pairwise Comparisons

no code implementations19 Jul 2017 Ashwin Pananjady, Cheng Mao, Vidya Muthukumar, Martin J. Wainwright, Thomas A. Courtade

We show that when the assignment of items to the topology is arbitrary, these permutation-based models, unlike their parametric counterparts, do not admit consistent estimation for most comparison topologies used in practice.

Cannot find the paper you are looking for? You can Submit a new open access paper.