Search Results for author:
Found 43 papers, 5 papers with code
Fully First-Order Methods for Decentralized Bilevel Optimization
This paper focuses on decentralized stochastic bilevel optimization (DSBO) where agents only communicate with their neighbors.
Single-Timescale Multi-Sequence Stochastic Approximation Without Fixed Point Smoothness: Theories and Applications
Stochastic approximation (SA) that involves multiple coupled sequences, known as multiple-sequence SA (MSSA), finds diverse applications in the fields of signal processing and machine learning.
Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis
To the best of our knowledge, our methods are the first to completely eliminate the need for stepsize tuning, while achieving theoretical guarantees.
Decentralized and Equitable Optimal Transport
Moreover, we propose the decentralized equitable optimal transport (DE-OT) problem.
A Single-Loop Algorithm for Decentralized Bilevel Optimization
Bilevel optimization has gained significant attention in recent years due to its broad applications in machine learning.
Zeroth-order Riemannian Averaging Stochastic Approximation Algorithms
We present Zeroth-order Riemannian Averaging Stochastic Approximation (\texttt{Zo-RASA}) algorithms for stochastic optimization on Riemannian manifolds.
A New Inexact Proximal Linear Algorithm with Adaptive Stopping Criteria for Robust Phase Retrieval
This paper considers the robust phase retrieval problem, which can be cast as a nonsmooth and nonconvex optimization problem.
A Riemannian ADMM
Our algorithm is the first ADMM type algorithm that minimizes a nonsmooth objective over manifold -- a particular nonconvex set.
Decentralized Stochastic Bilevel Optimization with Improved per-Iteration Complexity
Bilevel optimization recently has received tremendous attention due to its great success in solving important machine learning problems like meta learning, reinforcement learning, and hyperparameter optimization.
Federated Learning on Riemannian Manifolds
This paper studies FL over Riemannian manifolds, which finds important applications such as federated PCA and federated kPCA.
Efficiently Escaping Saddle Points in Bilevel Optimization
Moreover, we propose an inexact NEgative-curvature-Originated-from-Noise Algorithm (iNEON), a pure first-order algorithm that can escape saddle point and find local minimum of stochastic bilevel optimization.
On the Convergence of Projected Alternating Maximization for Equitable and Optimal Transport
This paper studies the equitable and optimal transport (EOT) problem, which has many applications such as fair division problems and optimal transport with multiple agents etc.
A Riemannian smoothing steepest descent method for non-Lipschitz optimization on submanifolds
In this paper, we propose a Riemannian smoothing steepest descent method to minimize a nonconvex and non-Lipschitz function on submanifolds.
Projection Robust Wasserstein Barycenters
One of the popular solution methods for this task is to compute the barycenter of the probability measures under the Wasserstein metric.
A Riemannian Block Coordinate Descent Method for Computing the Projection Robust Wasserstein Distance
We show that the complexity of arithmetic operations for RBCD to obtain an $\epsilon$-stationary point is $O(\epsilon^{-3})$.
Deep Multi-task Network for Delay Estimation and Echo Cancellation
In this paper, a multi-task network is proposed to address both ref-delay estimation and echo cancellation tasks.
Robust Speaker Extraction Network Based on Iterative Refined Adaptation
dataset confirm the superior performance of the proposed method over the network without IRA in terms of SI-SDR and PESQ improvement.
Robust Low-rank Matrix Completion via an Alternating Manifold Proximal Gradient Continuation Method
This problem aims to decompose a partially observed matrix into the superposition of a low-rank matrix and a sparse matrix, where the sparse matrix captures the grossly corrupted entries of the matrix.
A Manifold Proximal Linear Method for Sparse Spectral Clustering with Application to Single-Cell RNA Sequencing Data Analysis
This paper considers a widely adopted model for SSC, which can be formulated as an optimization problem over the Stiefel manifold with nonsmooth and nonconvex objective.
Manifold Proximal Point Algorithms for Dual Principal Component Pursuit and Orthogonal Dictionary Learning
Second, we propose a stochastic variant of ManPPA called StManPPA, which is well suited for large-scale computation, and establish its sublinear convergence rate.
Riemannian Stochastic Proximal Gradient Methods for Nonsmooth Optimization over the Stiefel Manifold
However, most of the existing Riemannian stochastic algorithms require the objective function to be differentiable, and they do not apply to the case where the objective function is nonsmooth.
Stochastic Zeroth-order Riemannian Derivative Estimation and Optimization
We consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in Euclidean space, where the task is to solve Riemannian optimization problem with only noisy objective function evaluations.
Zeroth-Order Algorithms for Nonconvex Minimax Problems with Improved Complexities
We establish that under the SGC assumption, the complexities of the stochastic algorithms match that of deterministic algorithms.
Accelerated Dual-Averaging Primal-Dual Method for Composite Convex Minimization
Dual averaging-type methods are widely used in industrial machine learning applications due to their ability to promoting solution structure (e. g., sparsity) efficiently.
An Alternating Manifold Proximal Gradient Method for Sparse PCA and Sparse CCA
Sparse principal component analysis (PCA) and sparse canonical correlation analysis (CCA) are two essential techniques from high-dimensional statistics and machine learning for analyzing large-scale data.
Stochastic Primal-Dual Method for Empirical Risk Minimization with O(1) Per-Iteration Complexity
Regularized empirical risk minimization problem with linear predictor appears frequently in machine learning.
Stochastic Primal-Dual Method for Empirical Risk Minimization with $\mathcal{O}(1)$ Per-Iteration Complexity
Regularized empirical risk minimization problem with linear predictor appears frequently in machine learning.
Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold
We prove that the proposed method globally converges to a stationary point.
Efficient Optimization Algorithms for Robust Principal Component Analysis and Its Variants
Robust PCA has drawn significant attention in the last decade due to its success in numerous application domains, ranging from bio-informatics, statistics, and machine learning to image and video processing in computer vision.
An ADMM-Based Interior-Point Method for Large-Scale Linear Programming
Due its connection to Newton's method, IPM is often classified as second-order method -- a genre that is attached with stability and accuracy at the expense of scalability.
Optimization and Control
Highly accurate model for prediction of lung nodule malignancy with CT scans
Here we present NoduleX, a systematic approach to predict lung nodule malignancy from CT data, based on deep learning convolutional neural networks (CNN).
Primal-Dual Optimization Algorithms over Riemannian Manifolds: an Iteration Complexity Analysis
For prohibitively large-size tensor or machine learning models, we present a sampling-based stochastic algorithm with the same iteration complexity bound in expectation.
GSOS: Gauss-Seidel Operator Splitting Algorithm for Multi-Term Nonsmooth Convex Composite Optimization
In addition, we develop a new technique to establish the global convergence of the GSOS algorithm.
Geometric descent method for convex composite minimization
In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems.
Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization
In this paper we study stochastic quasi-Newton methods for nonconvex stochastic optimization, where we assume that noisy information about the gradients of the objective function is available via a stochastic first-order oracle (SFO).
Barzilai-Borwein Step Size for Stochastic Gradient Descent
One of the major issues in stochastic gradient descent (SGD) methods is how to choose an appropriate step size while running the algorithm.
Structured Nonconvex and Nonsmooth Optimization: Algorithms and Iteration Complexity Analysis
In particular, we consider in this paper some constrained nonconvex optimization models in block decision variables, with or without coupled affine constraints.
Global Convergence of Unmodified 3-Block ADMM for a Class of Convex Minimization Problems
The alternating direction method of multipliers (ADMM) has been successfully applied to solve structured convex optimization problems due to its superior practical performance.
Efficient Algorithms for Robust and Stable Principal Component Pursuit Problems
Moreover, if the observed data matrix has also been corrupted by a dense noise matrix in addition to gross sparse error, then the stable principal component pursuit (SPCP) problem is solved to recover the low-rank matrix.
Optimization and Control
An Extragradient-Based Alternating Direction Method for Convex Minimization
The classical alternating direction type methods usually assume that the two convex functions have relatively easy proximal mappings.
Sparse Inverse Covariance Selection via Alternating Linearization Methods
Gaussian graphical models are of great interest in statistical learning.
Fast Alternating Linearization Methods for Minimizing the Sum of Two Convex Functions
We present in this paper first-order alternating linearization algorithms based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions.
Fixed Point and Bregman Iterative Methods for Matrix Rank Minimization
The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization.
Optimization and Control Information Theory Information Theory
