Search Results for author:
Found 39 papers, 11 papers with code
Distributing Synergy Functions: Unifying Game-Theoretic Interaction Methods for Machine-Learning Explainability
We show that, given modest assumptions, a unique full account of interactions between features, called synergies, is possible in the continuous input setting.
Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators
Nonlinear control systems with partial information to the decision maker are prevalent in a variety of applications.
Improving Adversarial Robustness via Joint Classification and Multiple Explicit Detection Classes
This work concerns the development of deep networks that are certifiably robust to adversarial attacks.
Stochastic Differentially Private and Fair Learning
However, existing algorithms for DP fair learning are either not guaranteed to converge or require full batch of data in each iteration of the algorithm to converge.
Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithms
Finally, by analyzing a class of accelerated gradient flow dynamics, whose suitable discretization yields the two-step momentum algorithm, we establish that stochastic performance tradeoffs also extend to continuous time.
Private Stochastic Optimization With Large Worst-Case Lipschitz Parameter: Optimal Rates for (Non-Smooth) Convex Losses and Extension to Non-Convex Losses
To address these limitations, this work provides near-optimal excess risk bounds that do not depend on the uniform Lipschitz parameter of the loss.
Private Non-Convex Federated Learning Without a Trusted Server
silo data and two classes of Lipschitz continuous loss functions: First, we consider losses satisfying the Proximal Polyak-Lojasiewicz (PL) inequality, which is an extension of the classical PL condition to the constrained setting.
A Rigorous Study of Integrated Gradients Method and Extensions to Internal Neuron Attributions
Attribution methods address the issue of explainability by quantifying the importance of an input feature for a model prediction.
Robustness through Data Augmentation Loss Consistency
Our experiments show that DAIR consistently outperforms ERM and DA-ERM with little marginal computational cost and sets new state-of-the-art results in several benchmarks involving covariant data augmentation.
Ranked #1 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.2
Multi-domain Dialogue State Tracking
Visual Question Answering
Nonconvex-Nonconcave Min-Max Optimization with a Small Maximization Domain
For $0 \le k \le 2$ the surrogate function can be efficiently maximized in $y$; our general approximation result then leads to efficient algorithms for finding a near-stationary point in nonconvex-nonconcave min-max problems, for which we also provide convergence guarantees.
RIFLE: Robust Inference from Low Order Marginals
The ubiquity of missing values in real-world datasets poses a challenge for statistical inference and can prevent similar datasets from being analyzed in the same study, precluding many existing datasets from being used for new analyses.
Private Federated Learning Without a Trusted Server: Optimal Algorithms for Convex Losses
This paper studies federated learning (FL) -- especially cross-silo FL -- with data from people who do not trust the server or other silos.
Efficient Algorithms for Estimating the Parameters of Mixed Linear Regression Models
When the additive noise in MLR model is Gaussian, Expectation-Maximization (EM) algorithm is a widely-used algorithm for maximum likelihood estimation of MLR parameters.
A Stochastic Optimization Framework for Fair Risk Minimization
We consider the problem of fair classification with discrete sensitive attributes and potentially large models and data sets, requiring stochastic solvers.
Output Perturbation for Differentially Private Convex Optimization with Improved Population Loss Bounds, Runtimes and Applications to Private Adversarial Training
Finally, we apply our theory to two learning frameworks: tilted ERM and adversarial learning.
Fair Empirical Risk Minimization via Exponential Rényi Mutual Information
In this paper, we propose another notion of fairness violation, called Exponential Rényi Mutual Information (ERMI) between sensitive attributes and the predicted target.
Near-Optimal Procedures for Model Discrimination with Non-Disclosure Properties
Here we provide matching upper and lower bounds on the sample complexity as given by $\min\{1/\Delta^2,\sqrt{r}/\Delta\}$ up to a constant factor; here $\Delta$ is a measure of separation between $\mathbb{P}_0$ and $\mathbb{P}_1$ and $r$ is the rank of the design covariance matrix.
Finding Second-Order Stationary Points Efficiently in Smooth Nonconvex Linearly Constrained Optimization Problems
To the best of our knowledge, this is the first time that first-order algorithms with polynomial per-iteration complexity and global sublinear rate are designed to find SOSPs of the important class of non-convex problems with linear constraints (almost surely).
Alternating Direction Method of Multipliers for Quantization
For such optimization problems, we study the performance of the Alternating Direction Method of Multipliers for Quantization ($\texttt{ADMM-Q}$) algorithm, which is a variant of the widely-used ADMM method applied to our discrete optimization problem.
Non-convex Min-Max Optimization: Applications, Challenges, and Recent Theoretical Advances
The min-max optimization problem, also known as the saddle point problem, is a classical optimization problem which is also studied in the context of zero-sum games.
Solving Non-Convex Non-Differentiable Min-Max Games using Proximal Gradient Method
Min-max saddle point games appear in a wide range of applications in machine leaning and signal processing.
Efficient Search of First-Order Nash Equilibria in Nonconvex-Concave Smooth Min-Max Problems
As a byproduct, the choice $\varepsilon_y = O(\varepsilon_x{}^2)$ allows for the $O(\varepsilon_x{}^{-3})$ complexity of finding an $\varepsilon_x$-stationary point for the standard Moreau envelope of the primal function.
Optimization and Control 90C06, 90C25, 90C26, 91A99
Zeroth-Order Algorithms for Nonconvex Minimax Problems with Improved Complexities
We establish that under the SGC assumption, the complexities of the stochastic algorithms match that of deterministic algorithms.
When Does Non-Orthogonal Tensor Decomposition Have No Spurious Local Minima?
We study the optimization problem for decomposing $d$ dimensional fourth-order Tensors with $k$ non-orthogonal components.
SNAP: Finding Approximate Second-Order Stationary Solutions Efficiently for Non-convex Linearly Constrained Problems
This paper proposes low-complexity algorithms for finding approximate second-order stationary points (SOSPs) of problems with smooth non-convex objective and linear constraints.
Rényi Fair Inference
In this paper, we use R\'enyi correlation as a measure of fairness of machine learning models and develop a general training framework to impose fairness.
Robustness of accelerated first-order algorithms for strongly convex optimization problems
We study the robustness of accelerated first-order algorithms to stochastic uncertainties in gradient evaluation.
Training generative networks using random discriminators
This design helps us to avoid the min-max formulation and leads to an optimization problem that is stable and could be solved efficiently.
Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods
In this paper, we study the problem in the non-convex regime and show that an \varepsilon--first order stationary point of the game can be computed when one of the player's objective can be optimized to global optimality efficiently.
Solving Non-Convex Non-Concave Min-Max Games Under Polyak-Łojasiewicz Condition
In this short note, we consider the problem of solving a min-max zero-sum game.
On the Behavior of the Expectation-Maximization Algorithm for Mixture Models
Our numerical experiments show that our algorithm outperforms the Naive EM algorithm in almost all scenarios.
Gradient Primal-Dual Algorithm Converges to Second-Order Stationary Solution for Nonconvex Distributed Optimization Over Networks
In this work, we study two first-order primal-dual based algorithms, the Gradient Primal-Dual Algorithm (GPDA) and the Gradient Alternating Direction Method of Multipliers (GADMM), for solving a class of linearly constrained non-convex optimization problems.
On the Convergence and Robustness of Training GANs with Regularized Optimal Transport
A popular GAN formulation is based on the use of Wasserstein distance as a metric between probability distributions.
Learning Deep Models: Critical Points and Local Openness
Then we use our characterization to: 1) show that every local optimum of two layer linear networks is globally optimal.
On Optimal Generalizability in Parametric Learning
Such bias is measured by the cross validation procedure in practice where the data set is partitioned into a training set used for training and a validation set, which is not used in training and is left to measure the out-of-sample performance.
Discrete Rényi Classifiers
We prove that for a given set of marginals, the minimum Hirschfeld-Gebelein-Renyi (HGR) correlation principle introduced in [1] leads to a randomized classification rule which is shown to have a misclassification rate no larger than twice the misclassification rate of the optimal classifier.
Computational Intractability of Dictionary Learning for Sparse Representation
In this paper we consider the dictionary learning problem for sparse representation.
Parallel Successive Convex Approximation for Nonsmooth Nonconvex Optimization
In this work, we propose an inexact parallel BCD approach where at each iteration, a subset of the variables is updated in parallel by minimizing convex approximations of the original objective function.
Optimization and Control
A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization
The block coordinate descent (BCD) method is widely used for minimizing a continuous function f of several block variables.
Optimization and Control
