Search Results for author:
Found 60 papers, 7 papers with code
Fair k-Center Clustering for Data Summarization
In data summarization we want to choose $k$ prototypes in order to summarize a data set.
Guarantees for Spectral Clustering with Fairness Constraints
Given the widespread popularity of spectral clustering (SC) for partitioning graph data, we study a version of constrained SC in which we try to incorporate the fairness notion proposed by Chierichetti et al. (2017).
Active Sampling for Min-Max Fairness
We propose simple active sampling and reweighting strategies for optimizing min-max fairness that can be applied to any classification or regression model learned via loss minimization.
The Power of Localization for Efficiently Learning Linear Separators with Noise
For malicious noise, where the adversary can corrupt both the label and the features, we provide a polynomial-time algorithm for learning linear separators in $\Re^d$ under isotropic log-concave distributions that can tolerate a nearly information-theoretically optimal noise rate of $\eta = \Omega(\epsilon)$.
Towards Learning Sparsely Used Dictionaries with Arbitrary Supports
To address this question while circumventing the issue of non-identifiability, we study a natural semirandom model for dictionary learning where there are a large number of samples $y=Ax$ with arbitrary k-sparse supports for x, along with a few samples where the sparse supports are chosen uniformly at random.
Clustering Semi-Random Mixtures of Gaussians
Gaussian mixture models (GMM) are the most widely used statistical model for the $k$-means clustering problem and form a popular framework for clustering in machine learning and data analysis.
Robust Communication-Optimal Distributed Clustering Algorithms
In this work, we study the $k$-median and $k$-means clustering problems when the data is distributed across many servers and can contain outliers.
Efficient PAC Learning from the Crowd
When a noticeable fraction of the labelers are perfect, and the rest behave arbitrarily, we show that any $\mathcal{F}$ that can be efficiently learned in the traditional realizable PAC model can be learned in a computationally efficient manner by querying the crowd, despite high amounts of noise in the responses.
Label optimal regret bounds for online local learning
We resolve an open question from (Christiano, 2014b) posed in COLT'14 regarding the optimal dependency of the regret achievable for online local learning on the size of the label set.
On some provably correct cases of variational inference for topic models
The assumptions on the topic priors are related to the well known Dirichlet prior, introduced to the area of topic modeling by (Blei et al., 2003).
Relax, no need to round: integrality of clustering formulations
Under the same distributional model, the $k$-means LP relaxation fails to recover such clusters at separation as large as $\Delta = 4$.
Local algorithms for interactive clustering
We study the design of interactive clustering algorithms for data sets satisfying natural stability assumptions.
Efficient Learning of Linear Separators under Bounded Noise
We provide the first polynomial time algorithm that can learn linear separators to arbitrarily small excess error in this noise model under the uniform distribution over the unit ball in $\Re^d$, for some constant value of $\eta$.
Learning Mixtures of Ranking Models
We present the first polynomial time algorithm which provably learns the parameters of a mixture of two Mallows models.
Learning using Local Membership Queries
We introduce a new model of membership query (MQ) learning, where the learning algorithm is restricted to query points that are \emph{close} to random examples drawn from the underlying distribution.
Supervised Clustering
We also propose a dynamic model where the teacher sees a random subset of the points.
Crowdsourcing with Arbitrary Adversaries
Most existing works on crowdsourcing assume that the workers follow the Dawid-Skene model, or the one-coin model as its special case, where every worker makes mistakes independently of other workers and with the same error probability for every task.
On Robustness to Adversarial Examples and Polynomial Optimization
In particular, we leverage this connection to (a) design computationally efficient robust algorithms with provable guarantees for a large class of hypothesis, namely linear classifiers and degree-2 polynomial threshold functions (PTFs), (b) give a precise characterization of the price of achieving robustness in a computationally efficient manner for these classes, (c) design efficient algorithms to certify robustness and generate adversarial attacks in a principled manner for 2-layer neural networks.
Adversarially Robust Low Dimensional Representations
In particular, our adversarially robust PCA primitive leads to computationally efficient and robust algorithms for both unsupervised and supervised learning problems such as clustering and learning adversarially robust classifiers.
A Deep Conditioning Treatment of Neural Networks
We study the role of depth in training randomly initialized overparameterized neural networks.
Efficient active learning of sparse halfspaces with arbitrary bounded noise
Even in the presence of mild label noise, i. e. $\eta$ is a small constant, this is a challenging problem and only recently have label complexity bounds of the form $\tilde{O}\big(s \cdot \mathrm{polylog}(d, \frac{1}{\epsilon})\big)$ been established in [Zhang, 2018] for computationally efficient algorithms.
Adversarial Learning Guarantees for Linear Hypotheses and Neural Networks
We give upper and lower bounds for the adversarial empirical Rademacher complexity of linear hypotheses with adversarial perturbations measured in $l_r$-norm for an arbitrary $r \geq 1$.
Estimating Principal Components under Adversarial Perturbations
We design a computationally efficient algorithm that given corrupted data, recovers an estimate of the top-$r$ principal subspace with error that depends on a robustness parameter $\kappa$ that we identify.
A Notion of Individual Fairness for Clustering
A common distinction in fair machine learning, in particular in fair classification, is between group fairness and individual fairness.
Adversarial robustness via robust low rank representations
Adversarial robustness measures the susceptibility of a classifier to imperceptible perturbations made to the inputs at test time.
On the Rademacher Complexity of Linear Hypothesis Sets
Linear predictors form a rich class of hypotheses used in a variety of learning algorithms.
Beyond Individual and Group Fairness
In the adversarial setting, we design efficient algorithms with competitive ratio guarantees.
Beyond GNNs: A Sample Efficient Architecture for Graph Problems
Finally, we empirically demonstrate the effectiveness of our proposed architecture for a variety of graph problems.
Online Learning under Adversarial Corruptions
We study the design of efficient online learning algorithms tolerant to adversarially corrupted rewards.
PAC-Bayes Learning Bounds for Sample-Dependent Priors
We present a series of new PAC-Bayes learning guarantees for randomized algorithms with sample-dependent priors.
Adversarial Robustness Across Representation Spaces
In this work we extend the above setting to consider the problem of training of deep neural networks that can be made simultaneously robust to perturbations applied in multiple natural representation spaces.
Evaluating Fairness of Machine Learning Models Under Uncertain and Incomplete Information
An alternate approach that is commonly used is to separately train an attribute classifier on data with sensitive attribute information, and then use it later in the ML pipeline to evaluate the bias of a given classifier.
A Multiclass Boosting Framework for Achieving Fast and Provable Adversarial Robustness
This apparent lack of robustness has led researchers to propose methods that can help to prevent an adversary from having such capabilities.
Calibration and Consistency of Adversarial Surrogate Losses
We then give a characterization of H-calibration and prove that some surrogate losses are indeed H-calibrated for the adversarial loss, with these hypothesis sets.
A Finer Calibration Analysis for Adversarial Robustness
Moreover, our calibration results, combined with the previous study of consistency by Awasthi et al. (2021), also lead to more general $H$-consistency results covering common hypothesis sets.
Measuring Model Fairness under Noisy Covariates: A Theoretical Perspective
In this work we study the problem of measuring the fairness of a machine learning model under noisy information.
Neural Active Learning with Performance Guarantees
We investigate the problem of active learning in the streaming setting in non-parametric regimes, where the labels are stochastically generated from a class of functions on which we make no assumptions whatsoever.
Semi-supervised Active Regression
In this setting, the learner has access to a dataset $X \in \mathbb{R}^{(n_1+n_2) \times d}$ which is composed of $n_1$ unlabelled examples that an algorithm can actively query, and $n_2$ examples labelled a-priori.
On the benefits of maximum likelihood estimation for Regression and Forecasting
We also demonstrate empirically that our method instantiated with a well-designed general purpose mixture likelihood family can obtain superior performance for a variety of tasks across time-series forecasting and regression datasets with different data distributions.
Efficient Algorithms for Learning Depth-2 Neural Networks with General ReLU Activations
We present polynomial time and sample efficient algorithms for learning an unknown depth-2 feedforward neural network with general ReLU activations, under mild non-degeneracy assumptions.
On the Existence of The Adversarial Bayes Classifier
Adversarial robustness is a critical property in a variety of modern machine learning applications.
A Convergence Analysis of Gradient Descent on Graph Neural Networks
Graph Neural Networks~(GNNs) are a powerful class of architectures for solving learning problems on graphs.
On the Existence of the Adversarial Bayes Classifier (Extended Version)
Our results can provide a useful tool for a subsequent study of surrogate losses in adversarial robustness and their consistency properties.
Agnostic Learnability of Halfspaces via Logistic Loss
In this paper, we close this gap by constructing a well-behaved distribution such that the global minimizer of the logistic risk over this distribution only achieves $\Omega(\sqrt{\textrm{OPT}})$ misclassification risk, matching the upper bound in (Frei et al., 2021).
Distributionally Robust Data Join
Suppose we are given two datasets: a labeled dataset and unlabeled dataset which also has additional auxiliary features not present in the first dataset.
Do More Negative Samples Necessarily Hurt in Contrastive Learning?
Recent investigations in noise contrastive estimation suggest, both empirically as well as theoretically, that while having more "negative samples" in the contrastive loss improves downstream classification performance initially, beyond a threshold, it hurts downstream performance due to a "collision-coverage" trade-off.
$\mathscr{H}$-Consistency Estimation Error of Surrogate Loss Minimizers
We also show that previous excess error bounds can be recovered as special cases of our general results.
Trimmed Maximum Likelihood Estimation for Robust Learning in Generalized Linear Models
We study the problem of learning generalized linear models under adversarial corruptions.
Individual Preference Stability for Clustering
In this paper, we propose a natural notion of individual preference (IP) stability for clustering, which asks that every data point, on average, is closer to the points in its own cluster than to the points in any other cluster.
Agnostic Learning of General ReLU Activation Using Gradient Descent
We provide a convergence analysis of gradient descent for the problem of agnostically learning a single ReLU function under Gaussian distributions.
On the Adversarial Robustness of Mixture of Experts
We next empirically evaluate the robustness of MoEs on ImageNet using adversarial attacks and show they are indeed more robust than dense models with the same computational cost.
Congested Bandits: Optimal Routing via Short-term Resets
For the linear contextual bandit setup, our algorithm, based on an iterative least squares planner, achieves policy regret $\tilde{O}(\sqrt{dT} + \Delta)$.
Best-Effort Adaptation
We show how these bounds can guide the design of learning algorithms that we discuss in detail.
The Sample Complexity of Multi-Distribution Learning for VC Classes
Multi-distribution learning is a natural generalization of PAC learning to settings with multiple data distributions.
Improving Length-Generalization in Transformers via Task Hinting
For sorting we show that it is possible to train models on data consisting of sequences having length at most $20$, and improve the test accuracy on sequences of length $100$ from less than 1% (for standard training) to more than 92% (via task hinting).
A Weighted K-Center Algorithm for Data Subset Selection
The success of deep learning hinges on enormous data and large models, which require labor-intensive annotations and heavy computation costs.
Simulated Overparameterization
Building upon this framework, we present a novel, architecture agnostic algorithm called "majority kernels", which seamlessly integrates with predominant architectures, including Transformer models.
On Distributed Larger-Than-Memory Subset Selection With Pairwise Submodular Functions
In this paper, we relax the requirement of having a central machine for the target subset by proposing a novel distributed bounding algorithm with provable approximation guarantees.
Stacking as Accelerated Gradient Descent
Stacking, a heuristic technique for training deep residual networks by progressively increasing the number of layers and initializing new layers by copying parameters from older layers, has proven quite successful in improving the efficiency of training deep neural networks.
