Search Results for author:
Found 29 papers, 4 papers with code
The Space Complexity of Approximating Logistic Loss
We provide space complexity lower bounds for data structures that approximate logistic loss up to $\epsilon$-relative error on a logistic regression problem with data $\mathbf{X} \in \mathbb{R}^{n \times d}$ and labels $\mathbf{y} \in \{-1, 1\}^d$.
A Precise Characterization of SGD Stability Using Loss Surface Geometry
In this paper, we delve deeper into the relationship between linear stability and sharpness.
On Memorization and Privacy Risks of Sharpness Aware Minimization
In this work, we dissect these performance gains through the lens of data memorization in overparameterized models.
Generalization Guarantees via Algorithm-dependent Rademacher Complexity
Algorithm- and data-dependent generalization bounds are required to explain the generalization behavior of modern machine learning algorithms.
Feature Space Sketching for Logistic Regression
We present novel bounds for coreset construction, feature selection, and dimensionality reduction for logistic regression.
mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization
Modern deep learning models are over-parameterized, where different optima can result in widely varying generalization performance.
Fast Feature Selection with Fairness Constraints
We study the fundamental problem of selecting optimal features for model construction.
Generalization Bounds using Lower Tail Exponents in Stochastic Optimizers
Despite the ubiquitous use of stochastic optimization algorithms in machine learning, the precise impact of these algorithms and their dynamics on generalization performance in realistic non-convex settings is still poorly understood.
LocalNewton: Reducing Communication Bottleneck for Distributed Learning
To enhance practicability, we devise an adaptive scheme to choose L, and we show that this reduces the number of local iterations in worker machines between two model synchronizations as the training proceeds, successively refining the model quality at the master.
Improved guarantees and a multiple-descent curve for Column Subset Selection and the Nystrom method
The Column Subset Selection Problem (CSSP) and the Nystrom method are among the leading tools for constructing small low-rank approximations of large datasets in machine learning and scientific computing.
Adversarially-Trained Deep Nets Transfer Better: Illustration on Image Classification
Transfer learning has emerged as a powerful methodology for adapting pre-trained deep neural networks on image recognition tasks to new domains.
Boundary thickness and robustness in learning models
Using these observations, we show that noise-augmentation on mixup training further increases boundary thickness, thereby combating vulnerability to various forms of adversarial attacks and OOD transforms.
Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective
Bayesian coresets have emerged as a promising approach for implementing scalable Bayesian inference.
Improved guarantees and a multiple-descent curve for Column Subset Selection and the Nyström method
The Column Subset Selection Problem (CSSP) and the Nystr\"om method are among the leading tools for constructing small low-rank approximations of large datasets in machine learning and scientific computing.
Learning Sparse Distributions using Iterative Hard Thresholding
Iterative hard thresholding (IHT) is a projected gradient descent algorithm, known to achieve state of the art performance for a wide range of structured estimation problems, such as sparse inference.
Geometric Rates of Convergence for Kernel-based Sampling Algorithms
The rate of convergence of weighted kernel herding (WKH) and sequential Bayesian quadrature (SBQ), two kernel-based sampling algorithms for estimating integrals with respect to some target probability measure, is investigated.
Interpreting Black Box Predictions using Fisher Kernels
Research in both machine learning and psychology suggests that salient examples can help humans to interpret learning models.
Boosting Black Box Variational Inference
Finally, we present a stopping criterion drawn from the duality gap in the classic FW analyses and exhaustive experiments to illustrate the usefulness of our theoretical and algorithmic contributions.
IHT dies hard: Provable accelerated Iterative Hard Thresholding
We study --both in theory and practice-- the use of momentum motions in classic iterative hard thresholding (IHT) methods.
Boosting Variational Inference: an Optimization Perspective
Variational inference is a popular technique to approximate a possibly intractable Bayesian posterior with a more tractable one.
On Approximation Guarantees for Greedy Low Rank Optimization
We provide new approximation guarantees for greedy low rank matrix estimation under standard assumptions of restricted strong convexity and smoothness.
Scalable Greedy Feature Selection via Weak Submodularity
Furthermore, we show that a bounded submodularity ratio can be used to provide data dependent bounds that can sometimes be tighter also for submodular functions.
A Unified Optimization View on Generalized Matching Pursuit and Frank-Wolfe
Two of the most fundamental prototypes of greedy optimization are the matching pursuit and Frank-Wolfe algorithms.
Restricted Strong Convexity Implies Weak Submodularity
Our results extend the work of Das and Kempe (2011) from the setting of linear regression to arbitrary objective functions.
Examples are not enough, learn to criticize! Criticism for Interpretability
Example-based explanations are widely used in the effort to improve the interpretability of highly complex distributions.
Information Projection and Approximate Inference for Structured Sparse Variables
Approximate inference via information projection has been recently introduced as a general-purpose approach for efficient probabilistic inference given sparse variables.
Pursuits in Structured Non-Convex Matrix Factorizations
Efficiently representing real world data in a succinct and parsimonious manner is of central importance in many fields.
Towards a Better Understanding of Predict and Count Models
In a recent paper, Levy and Goldberg pointed out an interesting connection between prediction-based word embedding models and count models based on pointwise mutual information.
On Prior Distributions and Approximate Inference for Structured Variables
In cases where this projection is intractable, we propose a family of parameterized approximations indexed by subsets of the domain.
