Search Results for author:
Found 44 papers, 14 papers with code
A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization
The block coordinate descent (BCD) method is widely used for minimizing a continuous function f of several block variables.
Optimization and Control
Parallel Successive Convex Approximation for Nonsmooth Nonconvex Optimization
In this work, we propose an inexact parallel BCD approach where at each iteration, a subset of the variables is updated in parallel by minimizing convex approximations of the original objective function.
Optimization and Control
Computational Intractability of Dictionary Learning for Sparse Representation
In this paper we consider the dictionary learning problem for sparse representation.
Discrete Rényi Classifiers
We prove that for a given set of marginals, the minimum Hirschfeld-Gebelein-Renyi (HGR) correlation principle introduced in [1] leads to a randomized classification rule which is shown to have a misclassification rate no larger than twice the misclassification rate of the optimal classifier.
On Optimal Generalizability in Parametric Learning
Such bias is measured by the cross validation procedure in practice where the data set is partitioned into a training set used for training and a validation set, which is not used in training and is left to measure the out-of-sample performance.
On the Convergence and Robustness of Training GANs with Regularized Optimal Transport
A popular GAN formulation is based on the use of Wasserstein distance as a metric between probability distributions.
Gradient Primal-Dual Algorithm Converges to Second-Order Stationary Solution for Nonconvex Distributed Optimization Over Networks
In this work, we study two first-order primal-dual based algorithms, the Gradient Primal-Dual Algorithm (GPDA) and the Gradient Alternating Direction Method of Multipliers (GADMM), for solving a class of linearly constrained non-convex optimization problems.
On the Behavior of the Expectation-Maximization Algorithm for Mixture Models
Our numerical experiments show that our algorithm outperforms the Naive EM algorithm in almost all scenarios.
Solving Non-Convex Non-Concave Min-Max Games Under Polyak-Łojasiewicz Condition
In this short note, we consider the problem of solving a min-max zero-sum game.
Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods
In this paper, we study the problem in the non-convex regime and show that an \varepsilon--first order stationary point of the game can be computed when one of the player's objective can be optimized to global optimality efficiently.
Training generative networks using random discriminators
This design helps us to avoid the min-max formulation and leads to an optimization problem that is stable and could be solved efficiently.
Robustness of accelerated first-order algorithms for strongly convex optimization problems
We study the robustness of accelerated first-order algorithms to stochastic uncertainties in gradient evaluation.
Rényi Fair Inference
In this paper, we use R\'enyi correlation as a measure of fairness of machine learning models and develop a general training framework to impose fairness.
SNAP: Finding Approximate Second-Order Stationary Solutions Efficiently for Non-convex Linearly Constrained Problems
This paper proposes low-complexity algorithms for finding approximate second-order stationary points (SOSPs) of problems with smooth non-convex objective and linear constraints.
When Does Non-Orthogonal Tensor Decomposition Have No Spurious Local Minima?
We study the optimization problem for decomposing $d$ dimensional fourth-order Tensors with $k$ non-orthogonal components.
Zeroth-Order Algorithms for Nonconvex Minimax Problems with Improved Complexities
We establish that under the SGC assumption, the complexities of the stochastic algorithms match that of deterministic algorithms.
Efficient Search of First-Order Nash Equilibria in Nonconvex-Concave Smooth Min-Max Problems
As a byproduct, the choice $\varepsilon_y = O(\varepsilon_x{}^2)$ allows for the $O(\varepsilon_x{}^{-3})$ complexity of finding an $\varepsilon_x$-stationary point for the standard Moreau envelope of the primal function.
Optimization and Control 90C06, 90C25, 90C26, 91A99
Solving Non-Convex Non-Differentiable Min-Max Games using Proximal Gradient Method
Min-max saddle point games appear in a wide range of applications in machine leaning and signal processing.
Non-convex Min-Max Optimization: Applications, Challenges, and Recent Theoretical Advances
The min-max optimization problem, also known as the saddle point problem, is a classical optimization problem which is also studied in the context of zero-sum games.
Alternating Direction Method of Multipliers for Quantization
For such optimization problems, we study the performance of the Alternating Direction Method of Multipliers for Quantization ($\texttt{ADMM-Q}$) algorithm, which is a variant of the widely-used ADMM method applied to our discrete optimization problem.
Finding Second-Order Stationary Points Efficiently in Smooth Nonconvex Linearly Constrained Optimization Problems
To the best of our knowledge, this is the first time that first-order algorithms with polynomial per-iteration complexity and global sublinear rate are designed to find SOSPs of the important class of non-convex problems with linear constraints (almost surely).
Near-Optimal Procedures for Model Discrimination with Non-Disclosure Properties
Here we provide matching upper and lower bounds on the sample complexity as given by $\min\{1/\Delta^2,\sqrt{r}/\Delta\}$ up to a constant factor; here $\Delta$ is a measure of separation between $\mathbb{P}_0$ and $\mathbb{P}_1$ and $r$ is the rank of the design covariance matrix.
Output Perturbation for Differentially Private Convex Optimization with Improved Population Loss Bounds, Runtimes and Applications to Private Adversarial Training
Finally, we apply our theory to two learning frameworks: tilted ERM and adversarial learning.
A Stochastic Optimization Framework for Fair Risk Minimization
We consider the problem of fair classification with discrete sensitive attributes and potentially large models and data sets, requiring stochastic solvers.
Efficient Algorithms for Estimating the Parameters of Mixed Linear Regression Models
When the additive noise in MLR model is Gaussian, Expectation-Maximization (EM) algorithm is a widely-used algorithm for maximum likelihood estimation of MLR parameters.
Private Federated Learning Without a Trusted Server: Optimal Algorithms for Convex Losses
This paper studies federated learning (FL)--especially cross-silo FL--with data from people who do not trust the server or other silos.
RIFLE: Imputation and Robust Inference from Low Order Marginals
We develop a statistical inference framework for regression and classification in the presence of missing data without imputation.
Nonconvex-Nonconcave Min-Max Optimization with a Small Maximization Domain
For $0 \le k \le 2$ the surrogate function can be efficiently maximized in $y$; our general approximation result then leads to efficient algorithms for finding a near-stationary point in nonconvex-nonconcave min-max problems, for which we also provide convergence guarantees.
Robustness through Data Augmentation Loss Consistency
Our experiments show that DAIR consistently outperforms ERM and DA-ERM with little marginal computational cost and sets new state-of-the-art results in several benchmarks involving covariant data augmentation.
Ranked #1 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.2
Multi-domain Dialogue State Tracking
Visual Question Answering
A Rigorous Study of Integrated Gradients Method and Extensions to Internal Neuron Attributions
Attribution methods address the issue of explainability by quantifying the importance of an input feature for a model prediction.
Private Non-Convex Federated Learning Without a Trusted Server
silo data and two classes of Lipschitz continuous loss functions: First, we consider losses satisfying the Proximal Polyak-Lojasiewicz (PL) inequality, which is an extension of the classical PL condition to the constrained setting.
Private Stochastic Optimization With Large Worst-Case Lipschitz Parameter: Optimal Rates for (Non-Smooth) Convex Losses and Extension to Non-Convex Losses
To address these limitations, this work provides near-optimal excess risk bounds that do not depend on the uniform Lipschitz parameter of the loss.
Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithms
Finally, by analyzing a class of accelerated gradient flow dynamics, whose suitable discretization yields the two-step momentum algorithm, we establish that stochastic performance tradeoffs also extend to continuous time.
Stochastic Differentially Private and Fair Learning
However, existing algorithms for DP fair learning are either not guaranteed to converge or require full batch of data in each iteration of the algorithm to converge.
Improving Adversarial Robustness via Joint Classification and Multiple Explicit Detection Classes
This work concerns the development of deep networks that are certifiably robust to adversarial attacks.
Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators
Nonlinear control systems with partial information to the decision maker are prevalent in a variety of applications.
Distributing Synergy Functions: Unifying Game-Theoretic Interaction Methods for Machine-Learning Explainability
We show that, given modest assumptions, a unique full account of interactions between features, called synergies, is possible in the continuous input setting.
Four Axiomatic Characterizations of the Integrated Gradients Attribution Method
Deep neural networks have produced significant progress among machine learning models in terms of accuracy and functionality, but their inner workings are still largely unknown.
Optimal Differentially Private Model Training with Public Data
We show that the optimal error rates can be attained (up to log factors) by either discarding private data and training a public model, or treating public data like it is private and using an optimal DP algorithm.
Dr. FERMI: A Stochastic Distributionally Robust Fair Empirical Risk Minimization Framework
While training fair machine learning models has been studied extensively in recent years, most developed methods rely on the assumption that the training and test data have similar distributions.
f-FERM: A Scalable Framework for Robust Fair Empirical Risk Minimization
While numerous constraints and regularization terms have been proposed in the literature to promote fairness in machine learning tasks, most of these methods are not amenable to stochastic optimization due to the complex and nonlinear structure of constraints and regularizers.
Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization
Our analysis is grounded in a novel parametric form of the neural network and an innovative connection between score matching and regression analysis, facilitating the application of advanced statistical and optimization techniques.
Differentially Private Next-Token Prediction of Large Language Models
Ensuring the privacy of Large Language Models (LLMs) is becoming increasingly important.
