Search Results for author: Volkan Cevher

Found 112 papers, 11 papers with code

Double-Loop Unadjusted Langevin Algorithm

no code implementations ICML 2020 Paul Rolland, Armin Eftekhari, Ali Kavis, Volkan Cevher

A well-known first-order method for sampling from log-concave probability distributions is the Unadjusted Langevin Algorithm (ULA).

Adversarial Audio Synthesis with Complex-valued Polynomial Networks

no code implementations14 Jun 2022 Yongtao Wu, Grigorios G Chrysos, Volkan Cevher

Our models can encourage the systematic design of other efficient architectures on the complex field.

Audio Generation

No-Regret Learning in Games with Noisy Feedback: Faster Rates and Adaptivity via Learning Rate Separation

no code implementations13 Jun 2022 Yu-Guan Hsieh, Kimon Antonakopoulos, Volkan Cevher, Panayotis Mertikopoulos

We examine the problem of regret minimization when the learner is involved in a continuous game with other optimizing agents: in this case, if all players follow a no-regret algorithm, it is possible to achieve significantly lower regret relative to fully adversarial environments.

Learning in games from a stochastic approximation viewpoint

no code implementations8 Jun 2022 Panayotis Mertikopoulos, Ya-Ping Hsieh, Volkan Cevher

We develop a unified stochastic approximation framework for analyzing the long-run behavior of multi-agent online learning in games.

online learning

High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize

no code implementations ICLR 2022 Ali Kavis, Kfir Yehuda Levy, Volkan Cevher

We present our analysis in a modular way and obtain a complementary $\mathcal O (1 / T)$ convergence rate in the deterministic setting.

Score matching enables causal discovery of nonlinear additive noise models

no code implementations8 Mar 2022 Paul Rolland, Volkan Cevher, Matthäus Kleindessner, Chris Russel, Bernhard Schölkopf, Dominik Janzing, Francesco Locatello

This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models.

Causal Discovery

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

1 code implementation26 Feb 2022 Gideon Dresdner, Maria-Luiza Vladarean, Gunnar Rätsch, Francesco Locatello, Volkan Cevher, Alp Yurtsever

We propose a stochastic conditional gradient method (CGM) for minimizing convex finite-sum objectives formed as a sum of smooth and non-smooth terms.

Matrix Completion

Controlling the Complexity and Lipschitz Constant improves polynomial nets

no code implementations ICLR 2022 Zhenyu Zhu, Fabian Latorre, Grigorios G Chrysos, Volkan Cevher

While the class of Polynomial Nets demonstrates comparable performance to neural networks (NN), it currently has neither theoretical generalization characterization nor robustness guarantees.

On the Complexity of a Practical Primal-Dual Coordinate Method

no code implementations19 Jan 2022 Ahmet Alacaoglu, Volkan Cevher, Stephen J. Wright

We prove complexity bounds for the primal-dual algorithm with random extrapolation and coordinate descent (PURE-CD), which has been shown to obtain good practical performance for solving convex-concave min-max problems with bilinear coupling.

STORM+: Fully Adaptive SGD with Recursive Momentum for Nonconvex Optimization

no code implementations NeurIPS 2021 Kfir Levy, Ali Kavis, Volkan Cevher

In this work we propose $\rm{STORM}^{+}$, a new method that is completely parameter-free, does not require large batch-sizes, and obtains the optimal $O(1/T^{1/3})$ rate for finding an approximate stationary point.

Sifting through the noise: Universal first-order methods for stochastic variational inequalities

no code implementations NeurIPS 2021 Kimon Antonakopoulos, Thomas Pethick, Ali Kavis, Panayotis Mertikopoulos, Volkan Cevher

Our first result is that the algorithm achieves the optimal rates of convergence for cocoercive problems when the profile of the randomness is known to the optimizer: $\mathcal{O}(1/\sqrt{T})$ for absolute noise profiles, and $\mathcal{O}(1/T)$ for relative ones.

The Effect of the Intrinsic Dimension on the Generalization of Quadratic Classifiers

no code implementations NeurIPS 2021 Fabian Latorre, Leello Tadesse Dadi, Paul Rolland, Volkan Cevher

We demonstrate this by deriving an upper bound on the Rademacher Complexity that depends on two key quantities: (i) the intrinsic dimension, which is a measure of isotropy, and (ii) the largest eigenvalue of the second moment (covariance) matrix of the distribution.

Convergence of adaptive algorithms for constrained weakly convex optimization

no code implementations NeurIPS 2021 Ahmet Alacaoglu, Yura Malitsky, Volkan Cevher

We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.

Stochastic Optimization

Subquadratic Overparameterization for Shallow Neural Networks

no code implementations NeurIPS 2021 ChaeHwan Song, Ali Ramezani-Kebrya, Thomas Pethick, Armin Eftekhari, Volkan Cevher

Overparameterization refers to the important phenomenon where the width of a neural network is chosen such that learning algorithms can provably attain zero loss in nonconvex training.

STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization

no code implementations1 Nov 2021 Kfir Y. Levy, Ali Kavis, Volkan Cevher

In this work we propose STORM+, a new method that is completely parameter-free, does not require large batch-sizes, and obtains the optimal $O(1/T^{1/3})$ rate for finding an approximate stationary point.

A first-order primal-dual method with adaptivity to local smoothness

no code implementations NeurIPS 2021 Maria-Luiza Vladarean, Yura Malitsky, Volkan Cevher

We consider the problem of finding a saddle point for the convex-concave objective $\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)$, where $f$ is a convex function with locally Lipschitz gradient and $g$ is convex and possibly non-smooth.

On the Double Descent of Random Features Models Trained with SGD

no code implementations13 Oct 2021 Fanghui Liu, Johan A. K. Suykens, Volkan Cevher

We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD).

A Rate-Distortion Approach to Domain Generalization

no code implementations29 Sep 2021 Yihang Chen, Grigorios Chrysos, Volkan Cevher

Domain generalization deals with the difference in the distribution between the training and testing datasets, i. e., the domain shift problem, by extracting domain-invariant features.

Contrastive Learning Domain Generalization

Linear Convergence of SGD on Overparametrized Shallow Neural Networks

no code implementations29 Sep 2021 Paul Rolland, Ali Ramezani-Kebrya, ChaeHwan Song, Fabian Latorre, Volkan Cevher

Despite the non-convex landscape, first-order methods can be shown to reach global minima when training overparameterized neural networks, where the number of parameters far exceed the number of training data.

Protect the weak: Class focused online learning for adversarial training

no code implementations29 Sep 2021 Thomas Pethick, Grigorios Chrysos, Volkan Cevher

In this work, we identify that the focus on the average accuracy metric can create vulnerabilities to the "weakest" class.

online learning

Sample-efficient actor-critic algorithms with an etiquette for zero-sum Markov games

no code implementations29 Sep 2021 Ahmet Alacaoglu, Luca Viano, Niao He, Volkan Cevher

Our sample complexities also match the best-known results for global convergence of policy gradient and two time-scale actor-critic algorithms in the single agent setting.

Policy Gradient Methods

On the benefits of deep RL in accelerated MRI sampling

no code implementations29 Sep 2021 Thomas Sanchez, Igor Krawczuk, Volkan Cevher

Deep learning approaches have shown great promise in accelerating magnetic resonance imaging (MRI), by reconstructing high quality images from highly undersampled data.

Self-Supervised Neural Architecture Search for Imbalanced Datasets

1 code implementation17 Sep 2021 Aleksandr Timofeev, Grigorios G. Chrysos, Volkan Cevher

The results demonstrate how the proposed method can be used in imbalanced datasets, while it can be fully run on a single GPU.

Neural Architecture Search Self-Supervised Learning

Uncertainty-Driven Adaptive Sampling via GANs

no code implementations23 Oct 2020 Thomas Sanchez, Igor Krawczuk, Zhaodong Sun, Volkan Cevher

We propose an adaptive sampling method for the linear model, driven by the uncertainty estimation with a generative adversarial network (GAN) model.

SSIM

Regret minimization in stochastic non-convex learning via a proximal-gradient approach

no code implementations13 Oct 2020 Nadav Hallak, Panayotis Mertikopoulos, Volkan Cevher

In this setting, the minimization of external regret is beyond reach for first-order methods, so we focus on a local regret measure defined via a proximal-gradient mapping.

Stochastic Optimization

Random extrapolation for primal-dual coordinate descent

no code implementations ICML 2020 Ahmet Alacaoglu, Olivier Fercoq, Volkan Cevher

We introduce a randomly extrapolated primal-dual coordinate descent method that adapts to sparsity of the data matrix and the favorable structures of the objective function.

Conditional gradient methods for stochastically constrained convex minimization

no code implementations ICML 2020 Maria-Luiza Vladarean, Ahmet Alacaoglu, Ya-Ping Hsieh, Volkan Cevher

We propose two novel conditional gradient-based methods for solving structured stochastic convex optimization problems with a large number of linear constraints.

Efficient Proximal Mapping of the 1-path-norm of Shallow Networks

no code implementations2 Jul 2020 Fabian Latorre, Paul Rolland, Nadav Hallak, Volkan Cevher

We demonstrate two new important properties of the 1-path-norm of shallow neural networks.

Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch

1 code implementation NeurIPS 2021 Luca Viano, Yu-Ting Huang, Parameswaran Kamalaruban, Adrian Weller, Volkan Cevher

We study the inverse reinforcement learning (IRL) problem under a transition dynamics mismatch between the expert and the learner.

reinforcement-learning

Interaction-limited Inverse Reinforcement Learning

no code implementations1 Jul 2020 Martin Troussard, Emmanuel Pignat, Parameswaran Kamalaruban, Sylvain Calinon, Volkan Cevher

This paper proposes an inverse reinforcement learning (IRL) framework to accelerate learning when the learner-teacher \textit{interaction} is \textit{limited} during training.

reinforcement-learning

Environment Shaping in Reinforcement Learning using State Abstraction

no code implementations23 Jun 2020 Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla

However, the applicability of potential-based reward shaping is limited in settings where (i) the state space is very large, and it is challenging to compute an appropriate potential function, (ii) the feedback signals are noisy, and even with shaped rewards the agent could be trapped in local optima, and (iii) changing the rewards alone is not sufficient, and effective shaping requires changing the dynamics.

reinforcement-learning

On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems

no code implementations NeurIPS 2020 Panayotis Mertikopoulos, Nadav Hallak, Ali Kavis, Volkan Cevher

This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm's convergence properties in non-convex problems.

The limits of min-max optimization algorithms: convergence to spurious non-critical sets

no code implementations16 Jun 2020 Ya-Ping Hsieh, Panayotis Mertikopoulos, Volkan Cevher

Compared to ordinary function minimization problems, min-max optimization algorithms encounter far greater challenges because of the existence of periodic cycles and similar phenomena.

Convergence of adaptive algorithms for weakly convex constrained optimization

no code implementations11 Jun 2020 Ahmet Alacaoglu, Yura Malitsky, Volkan Cevher

We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.

Stochastic Optimization

Lipschitz constant estimation of Neural Networks via sparse polynomial optimization

no code implementations ICLR 2020 Fabian Latorre, Paul Rolland, Volkan Cevher

We introduce LiPopt, a polynomial optimization framework for computing increasingly tighter upper bounds on the Lipschitz constant of neural networks.

A new regret analysis for Adam-type algorithms

no code implementations ICML 2020 Ahmet Alacaoglu, Yura Malitsky, Panayotis Mertikopoulos, Volkan Cevher

In this paper, we focus on a theory-practice gap for Adam and its variants (AMSgrad, AdamNC, etc.).

A Newton Frank-Wolfe Method for Constrained Self-Concordant Minimization

1 code implementation17 Feb 2020 Deyi Liu, Volkan Cevher, Quoc Tran-Dinh

We demonstrate how to scalably solve a class of constrained self-concordant minimization problems using linear minimization oracles (LMO) over the constraint set.

Experimental Design

Robust Reinforcement Learning via Adversarial training with Langevin Dynamics

1 code implementation14 Feb 2020 Parameswaran Kamalaruban, Yu-Ting Huang, Ya-Ping Hsieh, Paul Rolland, Cheng Shi, Volkan Cevher

We introduce a sampling perspective to tackle the challenging task of training robust Reinforcement Learning (RL) agents.

reinforcement-learning

Optimization for Reinforcement Learning: From Single Agent to Cooperative Agents

no code implementations1 Dec 2019 Donghwan Lee, Niao He, Parameswaran Kamalaruban, Volkan Cevher

This article reviews recent advances in multi-agent reinforcement learning algorithms for large-scale control systems and communication networks, which learn to communicate and cooperate.

Distributed Optimization Multi-agent Reinforcement Learning +1

UniXGrad: A Universal, Adaptive Algorithm with Optimal Guarantees for Constrained Optimization

no code implementations NeurIPS 2019 Ali Kavis, Kfir. Y. Levy, Francis Bach, Volkan Cevher

To the best of our knowledge, this is the first adaptive, unified algorithm that achieves the optimal rates in the constrained setting.

Nearly Minimal Over-Parametrization of Shallow Neural Networks

no code implementations9 Oct 2019 Armin Eftekhari, ChaeHwan Song, Volkan Cevher

A recent line of work has shown that an overparametrized neural network can perfectly fit the training data, an otherwise often intractable nonconvex optimization problem.

Closed loop deep Bayesian inversion: Uncertainty driven acquisition for fast MRI

no code implementations25 Sep 2019 Thomas Sanchez, Igor Krawczuk, Zhaodong Sun, Volkan Cevher

This work proposes a closed-loop, uncertainty-driven adaptive sampling frame- work (CLUDAS) for accelerating magnetic resonance imaging (MRI) via deep Bayesian inversion.

SSIM

Fast and Provable ADMM for Learning with Generative Priors

no code implementations NeurIPS 2019 Fabian Latorre Gómez, Armin Eftekhari, Volkan Cevher

We focus on the special case where such constraint arises from the specification that a variable should lie in the range of a neural network.

Compressive Sensing Denoising

Interactive Teaching Algorithms for Inverse Reinforcement Learning

no code implementations28 May 2019 Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla

We study the problem of inverse reinforcement learning (IRL) with the added twist that the learner is assisted by a helpful teacher.

reinforcement-learning

On Certifying Non-uniform Bound against Adversarial Attacks

no code implementations15 Mar 2019 Chen Liu, Ryota Tomioka, Volkan Cevher

This work studies the robustness certification problem of neural network models, which aims to find certified adversary-free regions as large as possible around data points.

An Optimal-Storage Approach to Semidefinite Programming using Approximate Complementarity

no code implementations9 Feb 2019 Lijun Ding, Alp Yurtsever, Volkan Cevher, Joel A. Tropp, Madeleine Udell

This paper develops a new storage-optimal algorithm that provably solves generic semidefinite programs (SDPs) in standard form.

Scalable Learning-Based Sampling Optimization for Compressive Dynamic MRI

1 code implementation1 Feb 2019 Thomas Sanchez, Baran Gözcü, Ruud B. van Heeswijk, Armin Eftekhari, Efe Ilıcak, Tolga Çukur, Volkan Cevher

Compressed sensing applied to magnetic resonance imaging (MRI) allows to reduce the scanning time by enabling images to be reconstructed from highly undersampled data.

Stochastic Frank-Wolfe for Composite Convex Minimization

1 code implementation NeurIPS 2019 Francesco Locatello, Alp Yurtsever, Olivier Fercoq, Volkan Cevher

A broad class of convex optimization problems can be formulated as a semidefinite program (SDP), minimization of a convex function over the positive-semidefinite cone subject to some affine constraints.

Stochastic Optimization

An Introductory Guide to Fano's Inequality with Applications in Statistical Estimation

no code implementations2 Jan 2019 Jonathan Scarlett, Volkan Cevher

Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning.

Density Estimation Model Selection

Efficient learning of smooth probability functions from Bernoulli tests with guarantees

no code implementations11 Dec 2018 Paul Rolland, Ali Kavis, Alex Immer, Adish Singla, Volkan Cevher

We study the fundamental problem of learning an unknown, smooth probability function via pointwise Bernoulli tests.

Iterative Classroom Teaching

no code implementations8 Nov 2018 Teresa Yeo, Parameswaran Kamalaruban, Adish Singla, Arpit Merchant, Thibault Asselborn, Louis Faucon, Pierre Dillenbourg, Volkan Cevher

We consider the machine teaching problem in a classroom-like setting wherein the teacher has to deliver the same examples to a diverse group of students.

Kernel Conjugate Gradient Methods with Random Projections

no code implementations5 Nov 2018 Junhong Lin, Volkan Cevher

We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space.

Adversarially Robust Optimization with Gaussian Processes

no code implementations NeurIPS 2018 Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka, Volkan Cevher

In this paper, we consider the problem of Gaussian process (GP) optimization with an added robustness requirement: The returned point may be perturbed by an adversary, and we require the function value to remain as high as possible even after this perturbation.

Gaussian Processes

Finding Mixed Nash Equilibria of Generative Adversarial Networks

no code implementations ICLR 2019 Ya-Ping Hsieh, Chen Liu, Volkan Cevher

We reconsider the training objective of Generative Adversarial Networks (GANs) from the mixed Nash Equilibria (NE) perspective.

Online Adaptive Methods, Universality and Acceleration

no code implementations NeurIPS 2018 Kfir. Y. Levy, Alp Yurtsever, Volkan Cevher

We present a novel method for convex unconstrained optimization that, without any modifications, ensures: (i) accelerated convergence rate for smooth objectives, (ii) standard convergence rate in the general (non-smooth) setting, and (iii) standard convergence rate in the stochastic optimization setting.

online learning Stochastic Optimization

Let’s be Honest: An Optimal No-Regret Framework for Zero-Sum Games

no code implementations ICML 2018 Ehsan Asadi Kangarshahi, Ya-Ping Hsieh, Mehmet Fatih Sahin, Volkan Cevher

We propose a simple algorithmic framework that simultaneously achieves the best rates for honest regret as well as adversarial regret, and in addition resolves the open problem of removing the logarithmic terms in convergence to the value of the game.

Optimal Distributed Learning with Multi-pass Stochastic Gradient Methods

no code implementations ICML 2018 Junhong Lin, Volkan Cevher

We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS).

Learning-Based Compressive MRI

no code implementations3 May 2018 Baran Gözcü, Rabeeh Karimi Mahabadi, Yen-Huan Li, Efe Ilıcak, Tolga Çukur, Jonathan Scarlett, Volkan Cevher

In the area of magnetic resonance imaging (MRI), an extensive range of non-linear reconstruction algorithms have been proposed that can be used with general Fourier subsampling patterns.

Learning Theory

Optimal Rates of Sketched-regularized Algorithms for Least-Squares Regression over Hilbert Spaces

no code implementations ICML 2018 Junhong Lin, Volkan Cevher

We investigate regularized algorithms combining with projection for least-squares regression problem over a Hilbert space, covering nonparametric regression over a reproducing kernel Hilbert space.

Mirrored Langevin Dynamics

no code implementations NeurIPS 2018 Ya-Ping Hsieh, Ali Kavis, Paul Rolland, Volkan Cevher

We consider the problem of sampling from constrained distributions, which has posed significant challenges to both non-asymptotic analysis and algorithmic design.

Dimension-free Information Concentration via Exp-Concavity

no code implementations26 Feb 2018 Ya-Ping Hsieh, Volkan Cevher

Information concentration of probability measures have important implications in learning theory.

Learning Theory

Robust Maximization of Non-Submodular Objectives

no code implementations20 Feb 2018 Ilija Bogunovic, Junyao Zhao, Volkan Cevher

In this work, we present a new algorithm Oblivious-Greedy and prove the first constant-factor approximation guarantees for a wider class of non-submodular objectives.

High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups

1 code implementation20 Feb 2018 Paul Rolland, Jonathan Scarlett, Ilija Bogunovic, Volkan Cevher

In this paper, we consider the approach of Kandasamy et al. (2015), in which the high-dimensional function decomposes as a sum of lower-dimensional functions on subsets of the underlying variables.

Additive models Hyperparameter Optimization

Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

no code implementations22 Jan 2018 Junhong Lin, Volkan Cevher

We then extend our results to spectral-regularization algorithms (SRA), including kernel ridge regression (KRR), kernel principal component analysis, and gradient methods.

Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spaces

no code implementations20 Jan 2018 Junhong Lin, Alessandro Rudi, Lorenzo Rosasco, Volkan Cevher

In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space.

Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization

no code implementations NeurIPS 2017 Ahmet Alacaoglu, Quoc Tran-Dinh, Olivier Fercoq, Volkan Cevher

We propose a new randomized coordinate descent method for a convex optimization template with broad applications.

Streaming Robust Submodular Maximization: A Partitioned Thresholding Approach

no code implementations NeurIPS 2017 Slobodan Mitrović, Ilija Bogunovic, Ashkan Norouzi-Fard, Jakub Tarnawski, Volkan Cevher

We study the classical problem of maximizing a monotone submodular function subject to a cardinality constraint k, with two additional twists: (i) elements arrive in a streaming fashion, and (ii) m items from the algorithm's memory are removed after the stream is finished.

Data Summarization

Phase Transitions in the Pooled Data Problem

no code implementations NeurIPS 2017 Jonathan Scarlett, Volkan Cevher

In this paper, we study the pooled data problem of identifying the labels associated with a large collection of items, based on a sequence of pooled tests revealing the counts of each label within the pool.

Combinatorial Penalties: Which structures are preserved by convex relaxations?

no code implementations17 Oct 2017 Marwa El Halabi, Francis Bach, Volkan Cevher

We consider the homogeneous and the non-homogeneous convex relaxations for combinatorial penalty functions defined on support sets.

Fixed-Rank Approximation of a Positive-Semidefinite Matrix from Streaming Data

no code implementations NeurIPS 2017 Joel A. Tropp, Alp Yurtsever, Madeleine Udell, Volkan Cevher

Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates.

Robust Submodular Maximization: A Non-Uniform Partitioning Approach

no code implementations ICML 2017 Ilija Bogunovic, Slobodan Mitrović, Jonathan Scarlett, Volkan Cevher

We study the problem of maximizing a monotone submodular function subject to a cardinality constraint $k$, with the added twist that a number of items $\tau$ from the returned set may be removed.

Data Summarization

Lower Bounds on Regret for Noisy Gaussian Process Bandit Optimization

no code implementations31 May 2017 Jonathan Scarlett, Ilijia Bogunovic, Volkan Cevher

For the isotropic squared-exponential kernel in $d$ dimensions, we find that an average simple regret of $\epsilon$ requires $T = \Omega\big(\frac{1}{\epsilon^2} (\log\frac{1}{\epsilon})^{d/2}\big)$, and the average cumulative regret is at least $\Omega\big( \sqrt{T(\log T)^{d/2}} \big)$, thus matching existing upper bounds up to the replacement of $d/2$ by $2d+O(1)$ in both cases.

Faster Coordinate Descent via Adaptive Importance Sampling

no code implementations7 Mar 2017 Dmytro Perekrestenko, Volkan Cevher, Martin Jaggi

Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems.

Truncated Variance Reduction: A Unified Approach to Bayesian Optimization and Level-Set Estimation

no code implementations NeurIPS 2016 Ilija Bogunovic, Jonathan Scarlett, Andreas Krause, Volkan Cevher

We present a new algorithm, truncated variance reduction (TruVaR), that treats Bayesian optimization (BO) and level-set estimation (LSE) with Gaussian processes in a unified fashion.

Gaussian Processes

Practical sketching algorithms for low-rank matrix approximation

no code implementations31 Aug 2016 Joel A. Tropp, Alp Yurtsever, Madeleine Udell, Volkan Cevher

This paper describes a suite of algorithms for constructing low-rank approximations of an input matrix from a random linear image of the matrix, called a sketch.

Lower Bounds on Active Learning for Graphical Model Selection

no code implementations8 Jul 2016 Jonathan Scarlett, Volkan Cevher

We consider the problem of estimating the underlying graph associated with a Markov random field, with the added twist that the decoding algorithm can iteratively choose which subsets of nodes to sample based on the previous samples, resulting in an active learning setting.

Active Learning Model Selection

Convex block-sparse linear regression with expanders -- provably

no code implementations21 Mar 2016 Anastasios Kyrillidis, Bubacarr Bah, Rouzbeh Hasheminezhad, Quoc Tran-Dinh, Luca Baldassarre, Volkan Cevher

Our experimental findings on synthetic and real applications support our claims for faster recovery in the convex setting -- as opposed to using dense sensing matrices, while showing a competitive recovery performance.

A single-phase, proximal path-following framework

no code implementations5 Mar 2016 Quoc Tran-Dinh, Anastasios Kyrillidis, Volkan Cevher

First, it allows handling non-smooth objectives via proximal operators; this avoids lifting the problem dimension in order to accommodate non-smooth components in optimization.

On the Difficulty of Selecting Ising Models with Approximate Recovery

no code implementations11 Feb 2016 Jonathan Scarlett, Volkan Cevher

We adopt an \emph{approximate recovery} criterion that allows for a number of missed edges or incorrectly-included edges, in contrast with the widely-studied exact recovery problem.

Partial Recovery Bounds for the Sparse Stochastic Block Model

no code implementations2 Feb 2016 Jonathan Scarlett, Volkan Cevher

In this paper, we study the information-theoretic limits of community detection in the symmetric two-community stochastic block model, with intra-community and inter-community edge probabilities $\frac{a}{n}$ and $\frac{b}{n}$ respectively.

Community Detection Stochastic Block Model

Learning Data Triage: Linear Decoding Works for Compressive MRI

no code implementations1 Feb 2016 Yen-Huan Li, Volkan Cevher

The standard approach to compressive sampling considers recovering an unknown deterministic signal with certain known structure, and designing the sub-sampling pattern and recovery algorithm based on the known structure.

Time-Varying Gaussian Process Bandit Optimization

no code implementations25 Jan 2016 Ilija Bogunovic, Jonathan Scarlett, Volkan Cevher

We illustrate the performance of the algorithms on both synthetic and real data, and we find the gradual forgetting of TV-GP-UCB to perform favorably compared to the sharp resetting of R-GP-UCB.

Preconditioned Spectral Descent for Deep Learning

no code implementations NeurIPS 2015 David E. Carlson, Edo Collins, Ya-Ping Hsieh, Lawrence Carin, Volkan Cevher

These challenges include, but are not limited to, the non-convexity of learning objectives and estimating the quantities needed for optimization algorithms, such as gradients.

A Universal Primal-Dual Convex Optimization Framework

no code implementations NeurIPS 2015 Alp Yurtsever, Quoc Tran Dinh, Volkan Cevher

We propose a new primal-dual algorithmic framework for a prototypical constrained convex optimization template.

Learning-based Compressive Subsampling

no code implementations21 Oct 2015 Luca Baldassarre, Yen-Huan Li, Jonathan Scarlett, Baran Gözcü, Ilija Bogunovic, Volkan Cevher

In this paper, we instead take a principled learning-based approach in which a \emph{fixed} index set is chosen based on a set of training signals $\mathbf{x}_1,\dotsc,\mathbf{x}_m$.

Combinatorial Optimization

Structured Sparsity: Discrete and Convex approaches

no code implementations20 Jul 2015 Anastasios Kyrillidis, Luca Baldassarre, Marwa El-Halabi, Quoc Tran-Dinh, Volkan Cevher

For each, we present the models in their discrete nature, discuss how to solve the ensuing discrete problems and then describe convex relaxations.

Compressive Sensing

Smooth Alternating Direction Methods for Nonsmooth Constrained Convex Optimization

no code implementations14 Jul 2015 Quoc Tran-Dinh, Volkan Cevher

We propose two new alternating direction methods to solve "fully" nonsmooth constrained convex problems.

Composite convex minimization involving self-concordant-like cost functions

no code implementations4 Feb 2015 Quoc Tran-Dinh, Yen-Huan Li, Volkan Cevher

The self-concordant-like property of a smooth convex function is a new analytical structure that generalizes the self-concordant notion.

Limits on Support Recovery with Probabilistic Models: An Information-Theoretic Framework

no code implementations29 Jan 2015 Jonathan Scarlett, Volkan Cevher

In several cases, our bounds not only provide matching scaling laws in the necessary and sufficient number of measurements, but also sharp thresholds with matching constant factors.

Compressive Sensing

Constrained convex minimization via model-based excessive gap

no code implementations NeurIPS 2014 Quoc Tran-Dinh, Volkan Cevher

We introduce a model-based excessive gap technique to analyze first-order primal- dual methods for constrained convex minimization.

Time--Data Tradeoffs by Aggressive Smoothing

no code implementations NeurIPS 2014 John J. Bruer, Joel A. Tropp, Volkan Cevher, Stephen Becker

This paper proposes a tradeoff between sample complexity and computation time that applies to statistical estimators based on convex optimization.

A totally unimodular view of structured sparsity

no code implementations7 Nov 2014 Marwa El Halabi, Volkan Cevher

This paper describes a simple framework for structured sparse recovery based on convex optimization.

Convex Optimization for Big Data

no code implementations4 Nov 2014 Volkan Cevher, Stephen Becker, Mark Schmidt

This article reviews recent advances in convex optimization algorithms for Big Data, which aim to reduce the computational, storage, and communications bottlenecks.

A Primal-Dual Algorithmic Framework for Constrained Convex Minimization

no code implementations20 Jun 2014 Quoc Tran-Dinh, Volkan Cevher

Our main analysis technique provides a fresh perspective on Nesterov's excessive gap technique in a structured fashion and unifies it with smoothing and primal-dual methods.

A variational approach to stable principal component pursuit

1 code implementation4 Jun 2014 Aleksandr Aravkin, Stephen Becker, Volkan Cevher, Peder Olsen

We introduce a new convex formulation for stable principal component pursuit (SPCP) to decompose noisy signals into low-rank and sparse representations.

Scalable sparse covariance estimation via self-concordance

no code implementations13 May 2014 Anastasios Kyrillidis, Rabeeh Karimi Mahabadi, Quoc Tran-Dinh, Volkan Cevher

We consider the class of convex minimization problems, composed of a self-concordant function, such as the $\log\det$ metric, a convex data fidelity term $h(\cdot)$ and, a regularizing -- possibly non-smooth -- function $g(\cdot)$.

High-Dimensional Gaussian Process Bandits

no code implementations NeurIPS 2013 Josip Djolonga, Andreas Krause, Volkan Cevher

Many applications in machine learning require optimizing unknown functions defined over a high-dimensional space from noisy samples that are expensive to obtain.

An Inexact Proximal Path-Following Algorithm for Constrained Convex Minimization

no code implementations7 Nov 2013 Quoc Tran Dinh, Anastasios Kyrillidis, Volkan Cevher

Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets.

Learning Non-Parametric Basis Independent Models from Point Queries via Low-Rank Methods

no code implementations7 Oct 2013 Hemant Tyagi, Volkan Cevher

We consider the problem of learning multi-ridge functions of the form f(x) = g(Ax) from point evaluations of f. We assume that the function f is defined on an l_2-ball in R^d, g is twice continuously differentiable almost everywhere, and A \in R^{k \times d} is a rank k matrix, where k << d. We propose a randomized, polynomial-complexity sampling scheme for estimating such functions.

Composite Self-Concordant Minimization

no code implementations13 Aug 2013 Quoc Tran-Dinh, Anastasios Kyrillidis, Volkan Cevher

We propose a variable metric framework for minimizing the sum of a self-concordant function and a possibly non-smooth convex function, endowed with an easily computable proximal operator.

Energy-aware adaptive bi-Lipschitz embeddings

no code implementations12 Jul 2013 Bubacarr Bah, Ali Sadeghian, Volkan Cevher

We propose a dimensionality reducing matrix design based on training data with constraints on its Frobenius norm and number of rows.

Compressive Sensing

Group-Sparse Model Selection: Hardness and Relaxations

no code implementations13 Mar 2013 Luca Baldassarre, Nirav Bhan, Volkan Cevher, Anastasios Kyrillidis, Siddhartha Satpathi

Group-based sparsity models are proven instrumental in linear regression problems for recovering signals from much fewer measurements than standard compressive sensing.

Compressive Sensing Model Selection

Active Learning of Multi-Index Function Models

no code implementations NeurIPS 2012 Tyagi Hemant, Volkan Cevher

We consider the problem of actively learning \textit{multi-index} functions of the form $f(\vecx) = g(\matA\vecx)= \sum_{i=1}^k g_i(\veca_i^T\vecx)$ from point evaluations of $f$.

Active Learning

Learning with Compressible Priors

no code implementations NeurIPS 2009 Volkan Cevher

By using order statistics, we show that N-sample iid realizations of generalized Pareto, Student’s t, log-normal, Frechet, and log-logistic distributions are compressible, i. e., they have a constant expected decay rate, which is independent of N. In contrast, we show that generalized Gaussian distribution with shape parameter q is compressible only in restricted cases since the expected decay rate of its N-sample iid realizations decreases with N as 1/[q log(N/q)].

Bayesian Inference

Sparse Signal Recovery Using Markov Random Fields

no code implementations NeurIPS 2008 Volkan Cevher, Marco F. Duarte, Chinmay Hegde, Richard Baraniuk

Compressive Sensing (CS) combines sampling and compression into a single sub-Nyquist linear measurement process for sparse and compressible signals.

Compressive Sensing

Cannot find the paper you are looking for? You can Submit a new open access paper.