Search Results for author:
Found 144 papers, 22 papers with code
Double-Loop Unadjusted Langevin Algorithm
A well-known first-order method for sampling from log-concave probability distributions is the Unadjusted Langevin Algorithm (ULA).
Efficient proximal mapping of the path-norm regularizer of shallow networks
We demonstrate two new important properties of the path-norm regularizer for shallow neural networks.
Robust NAS under adversarial training: benchmark, theory, and beyond
Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data.
Generalization of Scaled Deep ResNets in the Mean-Field Regime
To derive the generalization bounds under this setting, our analysis necessitates a shift from the conventional time-invariant Gram matrix employed in the lazy training regime to a time-variant, distribution-dependent version.
Extreme Miscalibration and the Illusion of Adversarial Robustness
Deep learning-based Natural Language Processing (NLP) models are vulnerable to adversarial attacks, where small perturbations can cause a model to misclassify.
Truly No-Regret Learning in Constrained MDPs
As Efroni et al. (2020) pointed out, it is an open question whether primal-dual algorithms can provably achieve sublinear regret if we do not allow error cancellations.
Leveraging the Context through Multi-Round Interactions for Jailbreaking Attacks
We contend that the prior context--the information preceding the attack query--plays a pivotal role in enabling potent Jailbreaking attacks.
Multilinear Operator Networks
On the other hand, Polynomial Networks is a class of models that does not require activation functions, but have yet to perform on par with modern architectures.
Efficient local linearity regularization to overcome catastrophic overfitting
Our regularization term can be theoretically linked to curvature of the loss function and is computationally cheaper than previous methods by avoiding Double Backpropagation.
MADA: Meta-Adaptive Optimizers through hyper-gradient Descent
Since Adam was introduced, several novel adaptive optimizers for deep learning have been proposed.
Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate
In this paper, we introduce a novel subspace cubic regularized Newton method that achieves a dimension-independent global convergence rate of ${O}\left(\frac{1}{mk}+\frac{1}{k^2}\right)$ for solving convex optimization problems.
Stable Nonconvex-Nonconcave Training via Linear Interpolation
This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training.
Evaluating the Fairness of Discriminative Foundation Models in Computer Vision
We propose a novel taxonomy for bias evaluation of discriminative foundation models, such as Contrastive Language-Pretraining (CLIP), that are used for labeling tasks.
Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
We consider monotone variational inequality (VI) problems in multi-GPU settings where multiple processors/workers/clients have access to local stochastic dual vectors.
Adversarial Training Should Be Cast as a Non-Zero-Sum Game
One prominent approach toward resolving the adversarial vulnerability of deep neural networks is the two-player zero-sum paradigm of adversarial training, in which predictors are trained against adversarially chosen perturbations of data.
Federated Learning under Covariate Shifts with Generalization Guarantees
This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance.
Benign Overfitting in Deep Neural Networks under Lazy Training
This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification while obtaining (nearly) zero-training error under the lazy training regime.
What can online reinforcement learning with function approximation benefit from general coverage conditions?
In online reinforcement learning (RL), instead of employing standard structural assumptions on Markov decision processes (MDPs), using a certain coverage condition (original from offline RL) is enough to ensure sample-efficient guarantees (Xie et al. 2023).
Regularization of polynomial networks for image recognition
We introduce a class of PNs, which are able to reach the performance of ResNet across a range of six benchmarks.
Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems
This paper introduces a new extragradient-type algorithm for a class of nonconvex-nonconcave minimax problems.
Revisiting adversarial training for the worst-performing class
Despite progress in adversarial training (AT), there is a substantial gap between the top-performing and worst-performing classes in many datasets.
Solving stochastic weak Minty variational inequalities without increasing batch size
This paper introduces a family of stochastic extragradient-type algorithms for a class of nonconvex-nonconcave problems characterized by the weak Minty variational inequality (MVI).
Min-Max Optimization Made Simple: Approximating the Proximal Point Method via Contraction Maps
In this paper we present a first-order method that admits near-optimal convergence rates for convex/concave min-max problems while requiring a simple and intuitive analysis.
Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization
We propose an adaptive variance-reduction method, called AdaSpider, for minimization of $L$-smooth, non-convex functions with a finite-sum structure.
Extra-Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods
This work proposes a universal and adaptive second-order method for minimizing second-order smooth, convex functions.
DiGress: Discrete Denoising diffusion for graph generation
This work introduces DiGress, a discrete denoising diffusion model for generating graphs with categorical node and edge attributes.
Proximal Point Imitation Learning
Thanks to PPM, we avoid nested policy evaluation and cost updates for online IL appearing in the prior literature.
Identifiability and generalizability from multiple experts in Inverse Reinforcement Learning
While Reinforcement Learning (RL) aims to train an agent from a reward function in a given environment, Inverse Reinforcement Learning (IRL) seeks to recover the reward function from observing an expert's behavior.
Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study
Neural tangent kernel (NTK) is a powerful tool to analyze training dynamics of neural networks and their generalization bounds.
Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)
In particular, when initialized with LeCun initialization, depth helps robustness with the lazy training regime.
Generalization Properties of NAS under Activation and Skip Connection Search
To this end, we derive the lower (and upper) bounds of the minimum eigenvalue of the Neural Tangent Kernel (NTK) under the (in)finite-width regime using a certain search space including mixed activation functions, fully connected, and residual neural networks.
Understanding Deep Neural Function Approximation in Reinforcement Learning via $ε$-Greedy Exploration
To be specific, we focus on the value based algorithm with the $\epsilon$-greedy exploration via deep (and two-layer) neural networks endowed by Besov (and Barron) function spaces, respectively, which aims at approximating an $\alpha$-smooth Q-function in a $d$-dimensional feature space.
Sound and Complete Verification of Polynomial Networks
Polynomial Networks (PNs) have demonstrated promising performance on face and image recognition recently.
Adversarial Audio Synthesis with Complex-valued Polynomial Networks
Our models can encourage the systematic design of other efficient architectures on the complex field.
No-Regret Learning in Games with Noisy Feedback: Faster Rates and Adaptivity via Learning Rate Separation
We examine the problem of regret minimization when the learner is involved in a continuous game with other optimizing agents: in this case, if all players follow a no-regret algorithm, it is possible to achieve significantly lower regret relative to fully adversarial environments.
A unified stochastic approximation framework for learning in games
We develop a flexible stochastic approximation framework for analyzing the long-run behavior of learning in games (both continuous and finite).
High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize
We present our analysis in a modular way and obtain a complementary $\mathcal O (1 / T)$ convergence rate in the deterministic setting.
Score matching enables causal discovery of nonlinear additive noise models
This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models.
The Spectral Bias of Polynomial Neural Networks
Inspired by such studies, we conduct a spectral analysis of the Neural Tangent Kernel (NTK) of PNNs.
Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization
We propose a stochastic conditional gradient method (CGM) for minimizing convex finite-sum objectives formed as a sum of smooth and non-smooth terms.
Controlling the Complexity and Lipschitz Constant improves polynomial nets
While the class of Polynomial Nets demonstrates comparable performance to neural networks (NN), it currently has neither theoretical generalization characterization nor robustness guarantees.
On the Complexity of a Practical Primal-Dual Coordinate Method
We prove complexity bounds for the primal-dual algorithm with random extrapolation and coordinate descent (PURE-CD), which has been shown to obtain good practical performance for solving convex-concave min-max problems with bilinear coupling.
The Effect of the Intrinsic Dimension on the Generalization of Quadratic Classifiers
We demonstrate this by deriving an upper bound on the Rademacher Complexity that depends on two key quantities: (i) the intrinsic dimension, which is a measure of isotropy, and (ii) the largest eigenvalue of the second moment (covariance) matrix of the distribution.
STORM+: Fully Adaptive SGD with Recursive Momentum for Nonconvex Optimization
In this work we propose $\rm{STORM}^{+}$, a new method that is completely parameter-free, does not require large batch-sizes, and obtains the optimal $O(1/T^{1/3})$ rate for finding an approximate stationary point.
Convergence of adaptive algorithms for constrained weakly convex optimization
We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.
Sifting through the noise: Universal first-order methods for stochastic variational inequalities
Our first result is that the algorithm achieves the optimal rates of convergence for cocoercive problems when the profile of the randomness is known to the optimizer: $\mathcal{O}(1/\sqrt{T})$ for absolute noise profiles, and $\mathcal{O}(1/T)$ for relative ones.
Subquadratic Overparameterization for Shallow Neural Networks
Overparameterization refers to the important phenomenon where the width of a neural network is chosen such that learning algorithms can provably attain zero loss in nonconvex training.
STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization
In this work we propose STORM+, a new method that is completely parameter-free, does not require large batch-sizes, and obtains the optimal $O(1/T^{1/3})$ rate for finding an approximate stationary point.
A first-order primal-dual method with adaptivity to local smoothness
We consider the problem of finding a saddle point for the convex-concave objective $\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)$, where $f$ is a convex function with locally Lipschitz gradient and $g$ is convex and possibly non-smooth.
On the Double Descent of Random Features Models Trained with SGD
We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/over-parameterized regime.
On the benefits of deep RL in accelerated MRI sampling
Deep learning approaches have shown great promise in accelerating magnetic resonance imaging (MRI), by reconstructing high quality images from highly undersampled data.
Linear Convergence of SGD on Overparametrized Shallow Neural Networks
Despite the non-convex landscape, first-order methods can be shown to reach global minima when training overparameterized neural networks, where the number of parameters far exceed the number of training data.
Sample-efficient actor-critic algorithms with an etiquette for zero-sum Markov games
Our sample complexities also match the best-known results for global convergence of policy gradient and two time-scale actor-critic algorithms in the single agent setting.
A Rate-Distortion Approach to Domain Generalization
Domain generalization deals with the difference in the distribution between the training and testing datasets, i. e., the domain shift problem, by extracting domain-invariant features.
Protect the weak: Class focused online learning for adversarial training
In this work, we identify that the focus on the average accuracy metric can create vulnerabilities to the "weakest" class.
Self-Supervised Neural Architecture Search for Imbalanced Datasets
The results demonstrate how the proposed method can be used in imbalanced datasets, while it can be fully run on a single GPU.
GG-GAN: A Geometric Graph Generative Adversarial Network
We study the fundamental problem of graph generation.
Solving Inverse Problems with Hybrid Deep Image Priors: the challenge of preventing overfitting
When the noise level is small, it does not considerably reduce the overfitting problem.
Uncertainty-Driven Adaptive Sampling via GANs
We propose an adaptive sampling method for the linear model, driven by the uncertainty estimation with a generative adversarial network (GAN) model.
Regret minimization in stochastic non-convex learning via a proximal-gradient approach
In this setting, the minimization of external regret is beyond reach for first-order methods, so we focus on a local regret measure defined via a proximal-gradient mapping.
Random extrapolation for primal-dual coordinate descent
We introduce a randomly extrapolated primal-dual coordinate descent method that adapts to sparsity of the data matrix and the favorable structures of the objective function.
Conditional gradient methods for stochastically constrained convex minimization
We propose two novel conditional gradient-based methods for solving structured stochastic convex optimization problems with a large number of linear constraints.
Efficient Proximal Mapping of the 1-path-norm of Shallow Networks
We demonstrate two new important properties of the 1-path-norm of shallow neural networks.
Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch
We study the inverse reinforcement learning (IRL) problem under a transition dynamics mismatch between the expert and the learner.
Interaction-limited Inverse Reinforcement Learning
This paper proposes an inverse reinforcement learning (IRL) framework to accelerate learning when the learner-teacher \textit{interaction} is \textit{limited} during training.
Environment Shaping in Reinforcement Learning using State Abstraction
However, the applicability of potential-based reward shaping is limited in settings where (i) the state space is very large, and it is challenging to compute an appropriate potential function, (ii) the feedback signals are noisy, and even with shaped rewards the agent could be trapped in local optima, and (iii) changing the rewards alone is not sufficient, and effective shaping requires changing the dynamics.
On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems
This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm's convergence properties in non-convex problems.
The limits of min-max optimization algorithms: convergence to spurious non-critical sets
Compared to ordinary function minimization problems, min-max optimization algorithms encounter far greater challenges because of the existence of periodic cycles and similar phenomena.
Convergence of adaptive algorithms for weakly convex constrained optimization
We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.
Lipschitz constant estimation of Neural Networks via sparse polynomial optimization
We introduce LiPopt, a polynomial optimization framework for computing increasingly tighter upper bounds on the Lipschitz constant of neural networks.
A new regret analysis for Adam-type algorithms
In this paper, we focus on a theory-practice gap for Adam and its variants (AMSgrad, AdamNC, etc.).
A Newton Frank-Wolfe Method for Constrained Self-Concordant Minimization
We demonstrate how to scalably solve a class of constrained self-concordant minimization problems using linear minimization oracles (LMO) over the constraint set.
Robust Reinforcement Learning via Adversarial training with Langevin Dynamics
We introduce a sampling perspective to tackle the challenging task of training robust Reinforcement Learning (RL) agents.
Optimization for Reinforcement Learning: From Single Agent to Cooperative Agents
This article reviews recent advances in multi-agent reinforcement learning algorithms for large-scale control systems and communication networks, which learn to communicate and cooperate.
Distributed Optimization
Multi-agent Reinforcement Learning
+2
UniXGrad: A Universal, Adaptive Algorithm with Optimal Guarantees for Constrained Optimization
To the best of our knowledge, this is the first adaptive, unified algorithm that achieves the optimal rates in the constrained setting.
Nearly Minimal Over-Parametrization of Shallow Neural Networks
A recent line of work has shown that an overparametrized neural network can perfectly fit the training data, an otherwise often intractable nonconvex optimization problem.
Closed loop deep Bayesian inversion: Uncertainty driven acquisition for fast MRI
This work proposes a closed-loop, uncertainty-driven adaptive sampling frame- work (CLUDAS) for accelerating magnetic resonance imaging (MRI) via deep Bayesian inversion.
Fast and Provable ADMM for Learning with Generative Priors
We focus on the special case where such constraint arises from the specification that a variable should lie in the range of a neural network.
Interactive Teaching Algorithms for Inverse Reinforcement Learning
We study the problem of inverse reinforcement learning (IRL) with the added twist that the learner is assisted by a helpful teacher.
On Certifying Non-uniform Bound against Adversarial Attacks
This work studies the robustness certification problem of neural network models, which aims to find certified adversary-free regions as large as possible around data points.
An Optimal-Storage Approach to Semidefinite Programming using Approximate Complementarity
This paper develops a new storage-optimal algorithm that provably solves generic semidefinite programs (SDPs) in standard form.
Scalable Learning-Based Sampling Optimization for Compressive Dynamic MRI
Compressed sensing applied to magnetic resonance imaging (MRI) allows to reduce the scanning time by enabling images to be reconstructed from highly undersampled data.
Stochastic Frank-Wolfe for Composite Convex Minimization
A broad class of convex optimization problems can be formulated as a semidefinite program (SDP), minimization of a convex function over the positive-semidefinite cone subject to some affine constraints.
An Introductory Guide to Fano's Inequality with Applications in Statistical Estimation
Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning.
Efficient learning of smooth probability functions from Bernoulli tests with guarantees
We study the fundamental problem of learning an unknown, smooth probability function via pointwise Bernoulli tests.
Iterative Classroom Teaching
We consider the machine teaching problem in a classroom-like setting wherein the teacher has to deliver the same examples to a diverse group of students.
Kernel Conjugate Gradient Methods with Random Projections
We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space.
Adversarially Robust Optimization with Gaussian Processes
In this paper, we consider the problem of Gaussian process (GP) optimization with an added robustness requirement: The returned point may be perturbed by an adversary, and we require the function value to remain as high as possible even after this perturbation.
Finding Mixed Nash Equilibria of Generative Adversarial Networks
We reconsider the training objective of Generative Adversarial Networks (GANs) from the mixed Nash Equilibria (NE) perspective.
On the Generalization of Stochastic Gradient Descent with Momentum
While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods.
Online Adaptive Methods, Universality and Acceleration
We present a novel method for convex unconstrained optimization that, without any modifications, ensures: (i) accelerated convergence rate for smooth objectives, (ii) standard convergence rate in the general (non-smooth) setting, and (iii) standard convergence rate in the stochastic optimization setting.
Let’s be Honest: An Optimal No-Regret Framework for Zero-Sum Games
We propose a simple algorithmic framework that simultaneously achieves the best rates for honest regret as well as adversarial regret, and in addition resolves the open problem of removing the logarithmic terms in convergence to the value of the game.
Optimal Distributed Learning with Multi-pass Stochastic Gradient Methods
We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS).
Learning-Based Compressive MRI
In the area of magnetic resonance imaging (MRI), an extensive range of non-linear reconstruction algorithms have been proposed that can be used with general Fourier subsampling patterns.
Optimal Rates of Sketched-regularized Algorithms for Least-Squares Regression over Hilbert Spaces
We investigate regularized algorithms combining with projection for least-squares regression problem over a Hilbert space, covering nonparametric regression over a reproducing kernel Hilbert space.
Mirrored Langevin Dynamics
We consider the problem of sampling from constrained distributions, which has posed significant challenges to both non-asymptotic analysis and algorithmic design.
Dimension-free Information Concentration via Exp-Concavity
Information concentration of probability measures have important implications in learning theory.
High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups
In this paper, we consider the approach of Kandasamy et al. (2015), in which the high-dimensional function decomposes as a sum of lower-dimensional functions on subsets of the underlying variables.
Robust Maximization of Non-Submodular Objectives
In this work, we present a new algorithm Oblivious-Greedy and prove the first constant-factor approximation guarantees for a wider class of non-submodular objectives.
Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms
We then extend our results to spectral-regularization algorithms (SRA), including kernel ridge regression (KRR), kernel principal component analysis, and gradient methods.
Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces
In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space.
Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization
We propose a new randomized coordinate descent method for a convex optimization template with broad applications.
Streaming Robust Submodular Maximization: A Partitioned Thresholding Approach
We study the classical problem of maximizing a monotone submodular function subject to a cardinality constraint k, with two additional twists: (i) elements arrive in a streaming fashion, and (ii) m items from the algorithm's memory are removed after the stream is finished.
Phase Transitions in the Pooled Data Problem
In this paper, we study the pooled data problem of identifying the labels associated with a large collection of items, based on a sequence of pooled tests revealing the counts of each label within the pool.
Combinatorial Penalties: Which structures are preserved by convex relaxations?
We consider the homogeneous and the non-homogeneous convex relaxations for combinatorial penalty functions defined on support sets.
Fixed-Rank Approximation of a Positive-Semidefinite Matrix from Streaming Data
Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates.
Robust Submodular Maximization: A Non-Uniform Partitioning Approach
We study the problem of maximizing a monotone submodular function subject to a cardinality constraint $k$, with the added twist that a number of items $\tau$ from the returned set may be removed.
Lower Bounds on Regret for Noisy Gaussian Process Bandit Optimization
For the isotropic squared-exponential kernel in $d$ dimensions, we find that an average simple regret of $\epsilon$ requires $T = \Omega\big(\frac{1}{\epsilon^2} (\log\frac{1}{\epsilon})^{d/2}\big)$, and the average cumulative regret is at least $\Omega\big( \sqrt{T(\log T)^{d/2}} \big)$, thus matching existing upper bounds up to the replacement of $d/2$ by $2d+O(1)$ in both cases.
Faster Coordinate Descent via Adaptive Importance Sampling
Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems.
Sketchy Decisions: Convex Low-Rank Matrix Optimization with Optimal Storage
This paper concerns a fundamental class of convex matrix optimization problems.
Truncated Variance Reduction: A Unified Approach to Bayesian Optimization and Level-Set Estimation
We present a new algorithm, truncated variance reduction (TruVaR), that treats Bayesian optimization (BO) and level-set estimation (LSE) with Gaussian processes in a unified fashion.
Practical sketching algorithms for low-rank matrix approximation
This paper describes a suite of algorithms for constructing low-rank approximations of an input matrix from a random linear image of the matrix, called a sketch.
Lower Bounds on Active Learning for Graphical Model Selection
We consider the problem of estimating the underlying graph associated with a Markov random field, with the added twist that the decoding algorithm can iteratively choose which subsets of nodes to sample based on the previous samples, resulting in an active learning setting.
Convex block-sparse linear regression with expanders -- provably
Our experimental findings on synthetic and real applications support our claims for faster recovery in the convex setting -- as opposed to using dense sensing matrices, while showing a competitive recovery performance.
A single-phase, proximal path-following framework
First, it allows handling non-smooth objectives via proximal operators; this avoids lifting the problem dimension in order to accommodate non-smooth components in optimization.
On the Difficulty of Selecting Ising Models with Approximate Recovery
We adopt an \emph{approximate recovery} criterion that allows for a number of missed edges or incorrectly-included edges, in contrast with the widely-studied exact recovery problem.
Partial Recovery Bounds for the Sparse Stochastic Block Model
In this paper, we study the information-theoretic limits of community detection in the symmetric two-community stochastic block model, with intra-community and inter-community edge probabilities $\frac{a}{n}$ and $\frac{b}{n}$ respectively.
Learning Data Triage: Linear Decoding Works for Compressive MRI
The standard approach to compressive sampling considers recovering an unknown deterministic signal with certain known structure, and designing the sub-sampling pattern and recovery algorithm based on the known structure.
Time-Varying Gaussian Process Bandit Optimization
We illustrate the performance of the algorithms on both synthetic and real data, and we find the gradual forgetting of TV-GP-UCB to perform favorably compared to the sharp resetting of R-GP-UCB.
Preconditioned Spectral Descent for Deep Learning
These challenges include, but are not limited to, the non-convexity of learning objectives and estimating the quantities needed for optimization algorithms, such as gradients.
A Universal Primal-Dual Convex Optimization Framework
We propose a new primal-dual algorithmic framework for a prototypical constrained convex optimization template.
Learning-based Compressive Subsampling
In this paper, we instead take a principled learning-based approach in which a \emph{fixed} index set is chosen based on a set of training signals $\mathbf{x}_1,\dotsc,\mathbf{x}_m$.
Linear Inverse Problems with Norm and Sparsity Constraints
We describe two nonconventional algorithms for linear regression, called GAME and CLASH.
Structured Sparsity: Discrete and Convex approaches
For each, we present the models in their discrete nature, discuss how to solve the ensuing discrete problems and then describe convex relaxations.
Smooth Alternating Direction Methods for Nonsmooth Constrained Convex Optimization
We propose two new alternating direction methods to solve "fully" nonsmooth constrained convex problems.
Adaptive-Rate Sparse Signal Reconstruction With Application in Compressive Background Subtraction
We propose and analyze an online algorithm for reconstructing a sequence of signals from a limited number of linear measurements.
Composite convex minimization involving self-concordant-like cost functions
The self-concordant-like property of a smooth convex function is a new analytical structure that generalizes the self-concordant notion.
Limits on Support Recovery with Probabilistic Models: An Information-Theoretic Framework
In several cases, our bounds not only provide matching scaling laws in the necessary and sufficient number of measurements, but also sharp thresholds with matching constant factors.
Constrained convex minimization via model-based excessive gap
We introduce a model-based excessive gap technique to analyze first-order primal- dual methods for constrained convex minimization.
Time--Data Tradeoffs by Aggressive Smoothing
This paper proposes a tradeoff between sample complexity and computation time that applies to statistical estimators based on convex optimization.
A totally unimodular view of structured sparsity
This paper describes a simple framework for structured sparse recovery based on convex optimization.
Convex Optimization for Big Data
This article reviews recent advances in convex optimization algorithms for Big Data, which aim to reduce the computational, storage, and communications bottlenecks.
A Primal-Dual Algorithmic Framework for Constrained Convex Minimization
Our main analysis technique provides a fresh perspective on Nesterov's excessive gap technique in a structured fashion and unifies it with smoothing and primal-dual methods.
A variational approach to stable principal component pursuit
We introduce a new convex formulation for stable principal component pursuit (SPCP) to decompose noisy signals into low-rank and sparse representations.
Scalable sparse covariance estimation via self-concordance
We consider the class of convex minimization problems, composed of a self-concordant function, such as the $\log\det$ metric, a convex data fidelity term $h(\cdot)$ and, a regularizing -- possibly non-smooth -- function $g(\cdot)$.
High-Dimensional Gaussian Process Bandits
Many applications in machine learning require optimizing unknown functions defined over a high-dimensional space from noisy samples that are expensive to obtain.
An Inexact Proximal Path-Following Algorithm for Constrained Convex Minimization
Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets.
Learning Non-Parametric Basis Independent Models from Point Queries via Low-Rank Methods
We consider the problem of learning multi-ridge functions of the form f(x) = g(Ax) from point evaluations of f. We assume that the function f is defined on an l_2-ball in R^d, g is twice continuously differentiable almost everywhere, and A \in R^{k \times d} is a rank k matrix, where k << d. We propose a randomized, polynomial-complexity sampling scheme for estimating such functions.
Composite Self-Concordant Minimization
We propose a variable metric framework for minimizing the sum of a self-concordant function and a possibly non-smooth convex function, endowed with an easily computable proximal operator.
Energy-aware adaptive bi-Lipschitz embeddings
We propose a dimensionality reducing matrix design based on training data with constraints on its Frobenius norm and number of rows.
Group-Sparse Model Selection: Hardness and Relaxations
Group-based sparsity models are proven instrumental in linear regression problems for recovering signals from much fewer measurements than standard compressive sensing.
Active Learning of Multi-Index Function Models
We consider the problem of actively learning \textit{multi-index} functions of the form $f(\vecx) = g(\matA\vecx)= \sum_{i=1}^k g_i(\veca_i^T\vecx)$ from point evaluations of $f$.
Learning with Compressible Priors
By using order statistics, we show that N-sample iid realizations of generalized Pareto, Student’s t, log-normal, Frechet, and log-logistic distributions are compressible, i. e., they have a constant expected decay rate, which is independent of N. In contrast, we show that generalized Gaussian distribution with shape parameter q is compressible only in restricted cases since the expected decay rate of its N-sample iid realizations decreases with N as 1/[q log(N/q)].
Sparse Signal Recovery Using Markov Random Fields
Compressive Sensing (CS) combines sampling and compression into a single sub-Nyquist linear measurement process for sparse and compressible signals.
