no code implementations • ICML 2020 • Fabian Latorre, Paul Rolland, Shaul Nadav Hallak, Volkan Cevher

We demonstrate two new important properties of the path-norm regularizer for shallow neural networks.

no code implementations • ICML 2020 • Paul Rolland, Armin Eftekhari, Ali Kavis, Volkan Cevher

A well-known first-order method for sampling from log-concave probability distributions is the Unadjusted Langevin Algorithm (ULA).

1 code implementation • 5 Nov 2024 • Zhan Li, Yongtao Wu, Yihang Chen, Francesco Tonin, Elias Abad Rocamora, Volkan Cevher

Large vision-language models (VLLMs) exhibit promising capabilities for processing multi-modal tasks across various application scenarios.

no code implementations • 31 Oct 2024 • Moritz Haas, Jin Xu, Volkan Cevher, Leena Chennuru Vankadara

In contrast, we identify a stable parameterization with layerwise perturbation scaling, which we call $\textit{Maximal Update and Perturbation Parameterization}$ ($\mu$P$^2$), that ensures all layers are both feature learning and effectively perturbed in the limit.

no code implementations • 29 Oct 2024 • Zhiqi Bu, Xiaomeng Jin, Bhanukiran Vinzamuri, Anil Ramakrishna, Kai-Wei Chang, Volkan Cevher, Mingyi Hong

Machine unlearning has been used to remove unwanted knowledge acquired by large language models (LLMs).

1 code implementation • 14 Oct 2024 • Wanyun Xie, Thomas Pethick, Volkan Cevher

Sharpness-aware minimization (SAM) has been shown to improve the generalization of neural networks.

no code implementations • 2 Sep 2024 • Soumajyoti Sarkar, Leonard Lausen, Volkan Cevher, Sheng Zha, Thomas Brox, George Karypis

Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling.

no code implementations • 7 Aug 2024 • Beatriz Borges, Negar Foroutan, Deniz Bayazit, Anna Sotnikova, Syrielle Montariol, Tanya Nazaretzky, Mohammadreza Banaei, Alireza Sakhaeirad, Philippe Servant, Seyed Parsa Neshaei, Jibril Frej, Angelika Romanou, Gail Weiss, Sepideh Mamooler, Zeming Chen, Simin Fan, Silin Gao, Mete Ismayilzada, Debjit Paul, Alexandre Schöpfer, Andrej Janchevski, Anja Tiede, Clarence Linden, Emanuele Troiani, Francesco Salvi, Freya Behrens, Giacomo Orsi, Giovanni Piccioli, Hadrien Sevel, Louis Coulon, Manuela Pineros-Rodriguez, Marin Bonnassies, Pierre Hellich, Puck van Gerwen, Sankalp Gambhir, Solal Pirelli, Thomas Blanchard, Timothée Callens, Toni Abi Aoun, Yannick Calvino Alonso, Yuri Cho, Alberto Chiappa, Antonio Sclocchi, Étienne Bruno, Florian Hofhammer, Gabriel Pescia, Geovani Rizk, Leello Dadi, Lucas Stoffl, Manoel Horta Ribeiro, Matthieu Bovel, Yueyang Pan, Aleksandra Radenovic, Alexandre Alahi, Alexander Mathis, Anne-Florence Bitbol, Boi Faltings, Cécile Hébert, Devis Tuia, François Maréchal, George Candea, Giuseppe Carleo, Jean-Cédric Chappelier, Nicolas Flammarion, Jean-Marie Fürbringer, Jean-Philippe Pellet, Karl Aberer, Lenka Zdeborová, Marcel Salathé, Martin Jaggi, Martin Rajman, Mathias Payer, Matthieu Wyart, Michael Gastpar, Michele Ceriotti, Ola Svensson, Olivier Lévêque, Paolo Ienne, Rachid Guerraoui, Robert West, Sanidhya Kashyap, Valerio Piazza, Viesturs Simanis, Viktor Kuncak, Volkan Cevher, Philippe Schwaller, Sacha Friedli, Patrick Jermann, Tanja Kaser, Antoine Bosselut

We investigate the potential scale of this vulnerability by measuring the degree to which AI assistants can complete assessment questions in standard university-level STEM courses.

1 code implementation • 17 Jul 2024 • Wanyun Xie, Fabian Latorre, Kimon Antonakopoulos, Thomas Pethick, Volkan Cevher

This paper rethinks Sharpness-Aware Minimization (SAM), which is originally formulated as a zero-sum game where the weights of a network and a bounded perturbation try to minimize/maximize, respectively, the same differentiable loss.

no code implementations • 12 Jul 2024 • Youngsuk Park, Kailash Budhathoki, Liangfu Chen, Jonas Kübler, Jiaji Huang, Matthäus Kleindessner, Jun Huan, Volkan Cevher, Yida Wang, George Karypis

Powerful foundation models, including large language models (LLMs), with Transformer architectures have ushered in a new era of Generative AI across various industries.

1 code implementation • 26 Jun 2024 • Pol Puigdemont, Stratis Skoulakis, Grigorios Chrysos, Volkan Cevher

Cutting plane methods are a fundamental approach for solving integer linear programs (ILPs).

no code implementations • 7 Jun 2024 • Ioannis Mavrothalassitis, Stratis Skoulakis, Leello Tadesse Dadi, Volkan Cevher

Given a sequence of functions $f_1,\ldots, f_n$ with $f_i:\mathcal{D}\mapsto \mathbb{R}$, finite-sum minimization seeks a point ${x}^\star \in \mathcal{D}$ minimizing $\sum_{j=1}^n f_j(x)/n$.

no code implementations • 5 Jun 2024 • Yihang Chen, Fanghui Liu, Taiji Suzuki, Volkan Cevher

We first derive the asymptotic expansion of high dimensional kernels under covariate shifts.

1 code implementation • 29 May 2024 • Justin Deschenaux, Igor Krawczuk, Grigorios Chrysos, Volkan Cevher

In this work, we go further and study DDPMs trained on strictly separate subsets of the data distribution with large gaps on the support of the latent factors.

no code implementations • 27 May 2024 • Sonny Achten, Francesco Tonin, Volkan Cevher, Johan A. K. Suykens

Clustering nodes in heterophilous graphs presents unique challenges due to the asymmetric relationships often overlooked by traditional methods, which moreover assume that good clustering corresponds to high intra-cluster and low inter-cluster connectivity.

1 code implementation • 24 May 2024 • Angeliki Kamoutsi, Peter Schmitt-Förster, Tobias Sutter, Volkan Cevher, John Lygeros

This work studies discrete-time discounted Markov decision processes with continuous state and action spaces and addresses the inverse problem of inferring a cost function from observed optimal behavior.

1 code implementation • 7 May 2024 • Elias Abad Rocamora, Yongtao Wu, Fanghui Liu, Grigorios G. Chrysos, Volkan Cevher

Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels.

no code implementations • 3 May 2024 • Luca Viano, Stratis Skoulakis, Volkan Cevher

We present a new algorithm for imitation learning in infinite horizon linear MDPs dubbed ILARL which greatly improves the bound on the number of trajectories that the learner needs to sample from the environment.

no code implementations • 29 Apr 2024 • Fanghui Liu, Leello Dadi, Volkan Cevher

In this paper, we study a suitable function space for over-parameterized two-layer neural networks with bounded norms (e. g., the path norm, the Barron norm) in the perspective of sample complexity and generalization properties.

no code implementations • 19 Mar 2024 • Yongtao Wu, Fanghui Liu, Carl-Johann Simon-Gabriel, Grigorios G Chrysos, Volkan Cevher

Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data.

no code implementations • 14 Mar 2024 • Yihang Chen, Fanghui Liu, Yiping Lu, Grigorios G. Chrysos, Volkan Cevher

To derive the generalization bounds under this setting, our analysis necessitates a shift from the conventional time-invariant Gram matrix employed in the lazy training regime to a time-variant, distribution-dependent version.

no code implementations • 27 Feb 2024 • Vyas Raina, Samson Tan, Volkan Cevher, Aditya Rawal, Sheng Zha, George Karypis

Deep learning-based Natural Language Processing (NLP) models are vulnerable to adversarial attacks, where small perturbations can cause a model to misclassify.

no code implementations • 24 Feb 2024 • Adrian Müller, Pragnya Alatur, Volkan Cevher, Giorgia Ramponi, Niao He

As Efroni et al. (2020) pointed out, it is an open question whether primal-dual algorithms can provably achieve sublinear regret if we do not allow error cancellations.

no code implementations • 14 Feb 2024 • Yixin Cheng, Markos Georgopoulos, Volkan Cevher, Grigorios G. Chrysos

Large Language Models (LLMs) are susceptible to Jailbreaking attacks, which aim to extract harmful information by subtly modifying the attack query.

1 code implementation • 31 Jan 2024 • Yixin Cheng, Grigorios G. Chrysos, Markos Georgopoulos, Volkan Cevher

On the other hand, Polynomial Networks is a class of models that does not require activation functions, but have yet to perform on par with modern architectures.

1 code implementation • 21 Jan 2024 • Elias Abad Rocamora, Fanghui Liu, Grigorios G. Chrysos, Pablo M. Olmos, Volkan Cevher

Our regularization term can be theoretically linked to curvature of the loss function and is computationally cheaper than previous methods by avoiding Double Backpropagation.

no code implementations • 17 Jan 2024 • Kaan Ozkara, Can Karakus, Parameswaran Raman, Mingyi Hong, Shoham Sabach, Branislav Kveton, Volkan Cevher

Following the introduction of Adam, several novel adaptive optimizers for deep learning have been proposed.

no code implementations • 5 Jan 2024 • Ruichen Jiang, Parameswaran Raman, Shoham Sabach, Aryan Mokhtari, Mingyi Hong, Volkan Cevher

In this paper, we introduce a novel subspace cubic regularized Newton method that achieves a dimension-independent global convergence rate of ${O}\left(\frac{1}{mk}+\frac{1}{k^2}\right)$ for solving convex optimization problems.

1 code implementation • NeurIPS 2023 • Thomas Pethick, Wanyun Xie, Volkan Cevher

This paper presents a theoretical analysis of linear interpolation as a principled method for stabilizing (large-scale) neural network training.

no code implementations • 18 Oct 2023 • Junaid Ali, Matthaeus Kleindessner, Florian Wenzel, Kailash Budhathoki, Volkan Cevher, Chris Russell

We propose a novel taxonomy for bias evaluation of discriminative foundation models, such as Contrastive Language-Pretraining (CLIP), that are used for labeling tasks.

1 code implementation • 17 Aug 2023 • Ali Ramezani-Kebrya, Kimon Antonakopoulos, Igor Krawczuk, Justin Deschenaux, Volkan Cevher

We consider monotone variational inequality (VI) problems in multi-GPU settings where multiple processors/workers/clients have access to local stochastic dual vectors.

no code implementations • 19 Jun 2023 • Alexander Robey, Fabian Latorre, George J. Pappas, Hamed Hassani, Volkan Cevher

One prominent approach toward resolving the adversarial vulnerability of deep neural networks is the two-player zero-sum paradigm of adversarial training, in which predictors are trained against adversarially chosen perturbations of data.

no code implementations • 8 Jun 2023 • Ali Ramezani-Kebrya, Fanghui Liu, Thomas Pethick, Grigorios Chrysos, Volkan Cevher

This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance.

no code implementations • 30 May 2023 • Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos, Francesco Locatello, Volkan Cevher

This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification while obtaining (nearly) zero-training error under the lazy training regime.

no code implementations • 25 Apr 2023 • Fanghui Liu, Luca Viano, Volkan Cevher

In online reinforcement learning (RL), instead of employing standard structural assumptions on Markov decision processes (MDPs), using a certain coverage condition (original from offline RL) is enough to ensure sample-efficient guarantees (Xie et al. 2023).

no code implementations • CVPR 2023 • Grigorios G Chrysos, Bohan Wang, Jiankang Deng, Volkan Cevher

We introduce a class of PNs, which are able to reach the performance of ResNet across a range of six benchmarks.

1 code implementation • ICLR 2022 • Thomas Pethick, Puya Latafat, Panagiotis Patrinos, Olivier Fercoq, Volkan Cevher

This paper introduces a new extragradient-type algorithm for a class of nonconvex-nonconcave minimax problems.

1 code implementation • 17 Feb 2023 • Thomas Pethick, Grigorios G. Chrysos, Volkan Cevher

Despite progress in adversarial training (AT), there is a substantial gap between the top-performing and worst-performing classes in many datasets.

1 code implementation • 17 Feb 2023 • Thomas Pethick, Olivier Fercoq, Puya Latafat, Panagiotis Patrinos, Volkan Cevher

This paper introduces a family of stochastic extragradient-type algorithms for a class of nonconvex-nonconcave problems characterized by the weak Minty variational inequality (MVI).

no code implementations • 10 Jan 2023 • Volkan Cevher, Georgios Piliouras, Ryann Sim, Stratis Skoulakis

In this paper we present a first-order method that admits near-optimal convergence rates for convex/concave min-max problems while requiring a simple and intuitive analysis.

no code implementations • 3 Nov 2022 • Ali Kavis, Stratis Skoulakis, Kimon Antonakopoulos, Leello Tadesse Dadi, Volkan Cevher

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of $L$-smooth, non-convex functions with a finite-sum structure.

no code implementations • 3 Nov 2022 • Kimon Antonakopoulos, Ali Kavis, Volkan Cevher

This work proposes a universal and adaptive second-order method for minimizing second-order smooth, convex functions.

2 code implementations • 29 Sep 2022 • Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, Pascal Frossard

This work introduces DiGress, a discrete denoising diffusion model for generating graphs with categorical node and edge attributes.

2 code implementations • 22 Sep 2022 • Luca Viano, Angeliki Kamoutsi, Gergely Neu, Igor Krawczuk, Volkan Cevher

Thanks to PPM, we avoid nested policy evaluation and cost updates for online IL appearing in the prior literature.

1 code implementation • 22 Sep 2022 • Paul Rolland, Luca Viano, Norman Schuerhoff, Boris Nikolov, Volkan Cevher

While Reinforcement Learning (RL) aims to train an agent from a reward function in a given environment, Inverse Reinforcement Learning (IRL) seeks to recover the reward function from observing an expert's behavior.

no code implementations • 16 Sep 2022 • Yongtao Wu, Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos, Volkan Cevher

Neural tangent kernel (NTK) is a powerful tool to analyze training dynamics of neural networks and their generalization bounds.

no code implementations • 15 Sep 2022 • Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos, Volkan Cevher

In particular, when initialized with LeCun initialization, depth helps robustness with the lazy training regime.

no code implementations • 15 Sep 2022 • Fanghui Liu, Luca Viano, Volkan Cevher

To be specific, we focus on the value based algorithm with the $\epsilon$-greedy exploration via deep (and two-layer) neural networks endowed by Besov (and Barron) function spaces, respectively, which aims at approximating an $\alpha$-smooth Q-function in a $d$-dimensional feature space.

1 code implementation • 15 Sep 2022 • Elias Abad Rocamora, Mehmet Fatih Sahin, Fanghui Liu, Grigorios G Chrysos, Volkan Cevher

Polynomial Networks (PNs) have demonstrated promising performance on face and image recognition recently.

no code implementations • 15 Sep 2022 • Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos, Volkan Cevher

To this end, we derive the lower (and upper) bounds of the minimum eigenvalue of the Neural Tangent Kernel (NTK) under the (in)finite-width regime using a certain search space including mixed activation functions, fully connected, and residual neural networks.

no code implementations • 14 Jun 2022 • Yongtao Wu, Grigorios G Chrysos, Volkan Cevher

Our models can encourage the systematic design of other efficient architectures on the complex field.

no code implementations • 13 Jun 2022 • Yu-Guan Hsieh, Kimon Antonakopoulos, Volkan Cevher, Panayotis Mertikopoulos

We examine the problem of regret minimization when the learner is involved in a continuous game with other optimizing agents: in this case, if all players follow a no-regret algorithm, it is possible to achieve significantly lower regret relative to fully adversarial environments.

no code implementations • 8 Jun 2022 • Panayotis Mertikopoulos, Ya-Ping Hsieh, Volkan Cevher

We develop a flexible stochastic approximation framework for analyzing the long-run behavior of learning in games (both continuous and finite).

no code implementations • ICLR 2022 • Ali Kavis, Kfir Yehuda Levy, Volkan Cevher

We present our analysis in a modular way and obtain a complementary $\mathcal O (1 / T)$ convergence rate in the deterministic setting.

no code implementations • 8 Mar 2022 • Paul Rolland, Volkan Cevher, Matthäus Kleindessner, Chris Russel, Bernhard Schölkopf, Dominik Janzing, Francesco Locatello

This paper demonstrates how to recover causal graphs from the score of the data distribution in non-linear additive (Gaussian) noise models.

no code implementations • ICLR 2022 • Moulik Choraria, Leello Tadesse Dadi, Grigorios Chrysos, Julien Mairal, Volkan Cevher

Inspired by such studies, we conduct a spectral analysis of the Neural Tangent Kernel (NTK) of PNNs.

1 code implementation • 26 Feb 2022 • Gideon Dresdner, Maria-Luiza Vladarean, Gunnar Rätsch, Francesco Locatello, Volkan Cevher, Alp Yurtsever

We propose a stochastic conditional gradient method (CGM) for minimizing convex finite-sum objectives formed as a sum of smooth and non-smooth terms.

no code implementations • ICLR 2022 • Zhenyu Zhu, Fabian Latorre, Grigorios G Chrysos, Volkan Cevher

While the class of Polynomial Nets demonstrates comparable performance to neural networks (NN), it currently has neither theoretical generalization characterization nor robustness guarantees.

no code implementations • 19 Jan 2022 • Ahmet Alacaoglu, Volkan Cevher, Stephen J. Wright

We prove complexity bounds for the primal-dual algorithm with random extrapolation and coordinate descent (PURE-CD), which has been shown to obtain good practical performance for solving convex-concave min-max problems with bilinear coupling.

no code implementations • NeurIPS 2021 • Kimon Antonakopoulos, Thomas Pethick, Ali Kavis, Panayotis Mertikopoulos, Volkan Cevher

Our first result is that the algorithm achieves the optimal rates of convergence for cocoercive problems when the profile of the randomness is known to the optimizer: $\mathcal{O}(1/\sqrt{T})$ for absolute noise profiles, and $\mathcal{O}(1/T)$ for relative ones.

no code implementations • NeurIPS 2021 • Ahmet Alacaoglu, Yura Malitsky, Volkan Cevher

We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.

no code implementations • NeurIPS 2021 • Fabian Latorre, Leello Tadesse Dadi, Paul Rolland, Volkan Cevher

We demonstrate this by deriving an upper bound on the Rademacher Complexity that depends on two key quantities: (i) the intrinsic dimension, which is a measure of isotropy, and (ii) the largest eigenvalue of the second moment (covariance) matrix of the distribution.

no code implementations • NeurIPS 2021 • Kfir Levy, Ali Kavis, Volkan Cevher

In this work we propose $\rm{STORM}^{+}$, a new method that is completely parameter-free, does not require large batch-sizes, and obtains the optimal $O(1/T^{1/3})$ rate for finding an approximate stationary point.

no code implementations • NeurIPS 2021 • ChaeHwan Song, Ali Ramezani-Kebrya, Thomas Pethick, Armin Eftekhari, Volkan Cevher

Overparameterization refers to the important phenomenon where the width of a neural network is chosen such that learning algorithms can provably attain zero loss in nonconvex training.

no code implementations • 1 Nov 2021 • Kfir Y. Levy, Ali Kavis, Volkan Cevher

In this work we propose STORM+, a new method that is completely parameter-free, does not require large batch-sizes, and obtains the optimal $O(1/T^{1/3})$ rate for finding an approximate stationary point.

no code implementations • NeurIPS 2021 • Maria-Luiza Vladarean, Yura Malitsky, Volkan Cevher

We consider the problem of finding a saddle point for the convex-concave objective $\min_x \max_y f(x) + \langle Ax, y\rangle - g^*(y)$, where $f$ is a convex function with locally Lipschitz gradient and $g$ is convex and possibly non-smooth.

no code implementations • 13 Oct 2021 • Fanghui Liu, Johan A. K. Suykens, Volkan Cevher

We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/over-parameterized regime.

no code implementations • 29 Sep 2021 • Ahmet Alacaoglu, Luca Viano, Niao He, Volkan Cevher

Our sample complexities also match the best-known results for global convergence of policy gradient and two time-scale actor-critic algorithms in the single agent setting.

no code implementations • 29 Sep 2021 • Yihang Chen, Grigorios Chrysos, Volkan Cevher

Domain generalization deals with the difference in the distribution between the training and testing datasets, i. e., the domain shift problem, by extracting domain-invariant features.

no code implementations • 29 Sep 2021 • Paul Rolland, Ali Ramezani-Kebrya, ChaeHwan Song, Fabian Latorre, Volkan Cevher

Despite the non-convex landscape, first-order methods can be shown to reach global minima when training overparameterized neural networks, where the number of parameters far exceed the number of training data.

no code implementations • 29 Sep 2021 • Thomas Sanchez, Igor Krawczuk, Volkan Cevher

Deep learning approaches have shown great promise in accelerating magnetic resonance imaging (MRI), by reconstructing high quality images from highly undersampled data.

no code implementations • 29 Sep 2021 • Thomas Pethick, Grigorios Chrysos, Volkan Cevher

In this work, we identify that the focus on the average accuracy metric can create vulnerabilities to the "weakest" class.

1 code implementation • 17 Sep 2021 • Aleksandr Timofeev, Grigorios G. Chrysos, Volkan Cevher

The results demonstrate how the proposed method can be used in imbalanced datasets, while it can be fully run on a single GPU.

no code implementations • 1 Jan 2021 • Igor Krawczuk, Pedro Abranches, Andreas Loukas, Volkan Cevher

We study the fundamental problem of graph generation.

1 code implementation • 3 Nov 2020 • Zhaodong Sun, Thomas Sanchez, Fabian Latorre, Volkan Cevher

When the noise level is small, it does not considerably reduce the overfitting problem.

no code implementations • 23 Oct 2020 • Thomas Sanchez, Igor Krawczuk, Zhaodong Sun, Volkan Cevher

We propose an adaptive sampling method for the linear model, driven by the uncertainty estimation with a generative adversarial network (GAN) model.

no code implementations • 13 Oct 2020 • Nadav Hallak, Panayotis Mertikopoulos, Volkan Cevher

In this setting, the minimization of external regret is beyond reach for first-order methods, so we focus on a local regret measure defined via a proximal-gradient mapping.

no code implementations • ICML 2020 • Ahmet Alacaoglu, Olivier Fercoq, Volkan Cevher

We introduce a randomly extrapolated primal-dual coordinate descent method that adapts to sparsity of the data matrix and the favorable structures of the objective function.

no code implementations • ICML 2020 • Maria-Luiza Vladarean, Ahmet Alacaoglu, Ya-Ping Hsieh, Volkan Cevher

We propose two novel conditional gradient-based methods for solving structured stochastic convex optimization problems with a large number of linear constraints.

1 code implementation • NeurIPS 2021 • Luca Viano, Yu-Ting Huang, Parameswaran Kamalaruban, Adrian Weller, Volkan Cevher

We study the inverse reinforcement learning (IRL) problem under a transition dynamics mismatch between the expert and the learner.

no code implementations • 2 Jul 2020 • Fabian Latorre, Paul Rolland, Nadav Hallak, Volkan Cevher

We demonstrate two new important properties of the 1-path-norm of shallow neural networks.

no code implementations • 1 Jul 2020 • Martin Troussard, Emmanuel Pignat, Parameswaran Kamalaruban, Sylvain Calinon, Volkan Cevher

This paper proposes an inverse reinforcement learning (IRL) framework to accelerate learning when the learner-teacher \textit{interaction} is \textit{limited} during training.

no code implementations • 23 Jun 2020 • Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla

However, the applicability of potential-based reward shaping is limited in settings where (i) the state space is very large, and it is challenging to compute an appropriate potential function, (ii) the feedback signals are noisy, and even with shaped rewards the agent could be trapped in local optima, and (iii) changing the rewards alone is not sufficient, and effective shaping requires changing the dynamics.

no code implementations • NeurIPS 2020 • Panayotis Mertikopoulos, Nadav Hallak, Ali Kavis, Volkan Cevher

This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm's convergence properties in non-convex problems.

no code implementations • 16 Jun 2020 • Ya-Ping Hsieh, Panayotis Mertikopoulos, Volkan Cevher

Compared to ordinary function minimization problems, min-max optimization algorithms encounter far greater challenges because of the existence of periodic cycles and similar phenomena.

no code implementations • 11 Jun 2020 • Ahmet Alacaoglu, Yura Malitsky, Volkan Cevher

We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective.

no code implementations • ICLR 2020 • Fabian Latorre, Paul Rolland, Volkan Cevher

We introduce LiPopt, a polynomial optimization framework for computing increasingly tighter upper bounds on the Lipschitz constant of neural networks.

no code implementations • ICML 2020 • Ahmet Alacaoglu, Yura Malitsky, Panayotis Mertikopoulos, Volkan Cevher

In this paper, we focus on a theory-practice gap for Adam and its variants (AMSgrad, AdamNC, etc.).

1 code implementation • 17 Feb 2020 • Deyi Liu, Volkan Cevher, Quoc Tran-Dinh

We demonstrate how to scalably solve a class of constrained self-concordant minimization problems using linear minimization oracles (LMO) over the constraint set.

1 code implementation • 14 Feb 2020 • Parameswaran Kamalaruban, Yu-Ting Huang, Ya-Ping Hsieh, Paul Rolland, Cheng Shi, Volkan Cevher

We introduce a sampling perspective to tackle the challenging task of training robust Reinforcement Learning (RL) agents.

no code implementations • 1 Dec 2019 • Donghwan Lee, Niao He, Parameswaran Kamalaruban, Volkan Cevher

This article reviews recent advances in multi-agent reinforcement learning algorithms for large-scale control systems and communication networks, which learn to communicate and cooperate.

Distributed Optimization
Multi-agent Reinforcement Learning
**+3**

no code implementations • NeurIPS 2019 • Ali Kavis, Kfir. Y. Levy, Francis Bach, Volkan Cevher

To the best of our knowledge, this is the first adaptive, unified algorithm that achieves the optimal rates in the constrained setting.

no code implementations • 9 Oct 2019 • Armin Eftekhari, ChaeHwan Song, Volkan Cevher

A recent line of work has shown that an overparametrized neural network can perfectly fit the training data, an otherwise often intractable nonconvex optimization problem.

no code implementations • 25 Sep 2019 • Thomas Sanchez, Igor Krawczuk, Zhaodong Sun, Volkan Cevher

This work proposes a closed-loop, uncertainty-driven adaptive sampling frame- work (CLUDAS) for accelerating magnetic resonance imaging (MRI) via deep Bayesian inversion.

no code implementations • NeurIPS 2019 • Fabian Latorre Gómez, Armin Eftekhari, Volkan Cevher

We focus on the special case where such constraint arises from the specification that a variable should lie in the range of a neural network.

no code implementations • 28 May 2019 • Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla

We study the problem of inverse reinforcement learning (IRL) with the added twist that the learner is assisted by a helpful teacher.

no code implementations • 15 Mar 2019 • Chen Liu, Ryota Tomioka, Volkan Cevher

This work studies the robustness certification problem of neural network models, which aims to find certified adversary-free regions as large as possible around data points.

no code implementations • 9 Feb 2019 • Lijun Ding, Alp Yurtsever, Volkan Cevher, Joel A. Tropp, Madeleine Udell

This paper develops a new storage-optimal algorithm that provably solves generic semidefinite programs (SDPs) in standard form.

1 code implementation • 1 Feb 2019 • Thomas Sanchez, Baran Gözcü, Ruud B. van Heeswijk, Armin Eftekhari, Efe Ilıcak, Tolga Çukur, Volkan Cevher

Compressed sensing applied to magnetic resonance imaging (MRI) allows to reduce the scanning time by enabling images to be reconstructed from highly undersampled data.

1 code implementation • NeurIPS 2019 • Francesco Locatello, Alp Yurtsever, Olivier Fercoq, Volkan Cevher

A broad class of convex optimization problems can be formulated as a semidefinite program (SDP), minimization of a convex function over the positive-semidefinite cone subject to some affine constraints.

no code implementations • 2 Jan 2019 • Jonathan Scarlett, Volkan Cevher

Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning.

no code implementations • 11 Dec 2018 • Paul Rolland, Ali Kavis, Alex Immer, Adish Singla, Volkan Cevher

We study the fundamental problem of learning an unknown, smooth probability function via pointwise Bernoulli tests.

no code implementations • 8 Nov 2018 • Teresa Yeo, Parameswaran Kamalaruban, Adish Singla, Arpit Merchant, Thibault Asselborn, Louis Faucon, Pierre Dillenbourg, Volkan Cevher

We consider the machine teaching problem in a classroom-like setting wherein the teacher has to deliver the same examples to a diverse group of students.

no code implementations • 5 Nov 2018 • Junhong Lin, Volkan Cevher

We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space.

no code implementations • NeurIPS 2018 • Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka, Volkan Cevher

In this paper, we consider the problem of Gaussian process (GP) optimization with an added robustness requirement: The returned point may be perturbed by an adversary, and we require the function value to remain as high as possible even after this perturbation.

no code implementations • ICLR 2019 • Ya-Ping Hsieh, Chen Liu, Volkan Cevher

We reconsider the training objective of Generative Adversarial Networks (GANs) from the mixed Nash Equilibria (NE) perspective.

no code implementations • 12 Sep 2018 • Ali Ramezani-Kebrya, Kimon Antonakopoulos, Volkan Cevher, Ashish Khisti, Ben Liang

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods.

no code implementations • NeurIPS 2018 • Kfir. Y. Levy, Alp Yurtsever, Volkan Cevher

We present a novel method for convex unconstrained optimization that, without any modifications, ensures: (i) accelerated convergence rate for smooth objectives, (ii) standard convergence rate in the general (non-smooth) setting, and (iii) standard convergence rate in the stochastic optimization setting.

no code implementations • ICML 2018 • Junhong Lin, Volkan Cevher

We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS).

no code implementations • ICML 2018 • Ehsan Asadi Kangarshahi, Ya-Ping Hsieh, Mehmet Fatih Sahin, Volkan Cevher

We propose a simple algorithmic framework that simultaneously achieves the best rates for honest regret as well as adversarial regret, and in addition resolves the open problem of removing the logarithmic terms in convergence to the value of the game.

no code implementations • 3 May 2018 • Baran Gözcü, Rabeeh Karimi Mahabadi, Yen-Huan Li, Efe Ilıcak, Tolga Çukur, Jonathan Scarlett, Volkan Cevher

In the area of magnetic resonance imaging (MRI), an extensive range of non-linear reconstruction algorithms have been proposed that can be used with general Fourier subsampling patterns.

no code implementations • ICML 2018 • Junhong Lin, Volkan Cevher

We investigate regularized algorithms combining with projection for least-squares regression problem over a Hilbert space, covering nonparametric regression over a reproducing kernel Hilbert space.

no code implementations • NeurIPS 2018 • Ya-Ping Hsieh, Ali Kavis, Paul Rolland, Volkan Cevher

We consider the problem of sampling from constrained distributions, which has posed significant challenges to both non-asymptotic analysis and algorithmic design.

no code implementations • 26 Feb 2018 • Ya-Ping Hsieh, Volkan Cevher

Information concentration of probability measures have important implications in learning theory.

no code implementations • 20 Feb 2018 • Ilija Bogunovic, Junyao Zhao, Volkan Cevher

In this work, we present a new algorithm Oblivious-Greedy and prove the first constant-factor approximation guarantees for a wider class of non-submodular objectives.

1 code implementation • 20 Feb 2018 • Paul Rolland, Jonathan Scarlett, Ilija Bogunovic, Volkan Cevher

In this paper, we consider the approach of Kandasamy et al. (2015), in which the high-dimensional function decomposes as a sum of lower-dimensional functions on subsets of the underlying variables.

no code implementations • 22 Jan 2018 • Junhong Lin, Volkan Cevher

We then extend our results to spectral-regularization algorithms (SRA), including kernel ridge regression (KRR), kernel principal component analysis, and gradient methods.

no code implementations • 20 Jan 2018 • Junhong Lin, Alessandro Rudi, Lorenzo Rosasco, Volkan Cevher

In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space.

no code implementations • NeurIPS 2017 • Ahmet Alacaoglu, Quoc Tran-Dinh, Olivier Fercoq, Volkan Cevher

We propose a new randomized coordinate descent method for a convex optimization template with broad applications.

no code implementations • NeurIPS 2017 • Slobodan Mitrović, Ilija Bogunovic, Ashkan Norouzi-Fard, Jakub Tarnawski, Volkan Cevher

We study the classical problem of maximizing a monotone submodular function subject to a cardinality constraint k, with two additional twists: (i) elements arrive in a streaming fashion, and (ii) m items from the algorithm's memory are removed after the stream is finished.

no code implementations • NeurIPS 2017 • Jonathan Scarlett, Volkan Cevher

In this paper, we study the pooled data problem of identifying the labels associated with a large collection of items, based on a sequence of pooled tests revealing the counts of each label within the pool.

no code implementations • 17 Oct 2017 • Marwa El Halabi, Francis Bach, Volkan Cevher

We consider the homogeneous and the non-homogeneous convex relaxations for combinatorial penalty functions defined on support sets.

no code implementations • NeurIPS 2017 • Joel A. Tropp, Alp Yurtsever, Madeleine Udell, Volkan Cevher

Several important applications, such as streaming PCA and semidefinite programming, involve a large-scale positive-semidefinite (psd) matrix that is presented as a sequence of linear updates.

no code implementations • ICML 2017 • Ilija Bogunovic, Slobodan Mitrović, Jonathan Scarlett, Volkan Cevher

We study the problem of maximizing a monotone submodular function subject to a cardinality constraint $k$, with the added twist that a number of items $\tau$ from the returned set may be removed.

no code implementations • 31 May 2017 • Jonathan Scarlett, Ilijia Bogunovic, Volkan Cevher

For the isotropic squared-exponential kernel in $d$ dimensions, we find that an average simple regret of $\epsilon$ requires $T = \Omega\big(\frac{1}{\epsilon^2} (\log\frac{1}{\epsilon})^{d/2}\big)$, and the average cumulative regret is at least $\Omega\big( \sqrt{T(\log T)^{d/2}} \big)$, thus matching existing upper bounds up to the replacement of $d/2$ by $2d+O(1)$ in both cases.

no code implementations • 7 Mar 2017 • Dmytro Perekrestenko, Volkan Cevher, Martin Jaggi

Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems.

1 code implementation • 22 Feb 2017 • Alp Yurtsever, Madeleine Udell, Joel A. Tropp, Volkan Cevher

This paper concerns a fundamental class of convex matrix optimization problems.

no code implementations • NeurIPS 2016 • Ilija Bogunovic, Jonathan Scarlett, Andreas Krause, Volkan Cevher

We present a new algorithm, truncated variance reduction (TruVaR), that treats Bayesian optimization (BO) and level-set estimation (LSE) with Gaussian processes in a unified fashion.

no code implementations • 31 Aug 2016 • Joel A. Tropp, Alp Yurtsever, Madeleine Udell, Volkan Cevher

This paper describes a suite of algorithms for constructing low-rank approximations of an input matrix from a random linear image of the matrix, called a sketch.

no code implementations • 8 Jul 2016 • Jonathan Scarlett, Volkan Cevher

We consider the problem of estimating the underlying graph associated with a Markov random field, with the added twist that the decoding algorithm can iteratively choose which subsets of nodes to sample based on the previous samples, resulting in an active learning setting.

no code implementations • 21 Mar 2016 • Anastasios Kyrillidis, Bubacarr Bah, Rouzbeh Hasheminezhad, Quoc Tran-Dinh, Luca Baldassarre, Volkan Cevher

Our experimental findings on synthetic and real applications support our claims for faster recovery in the convex setting -- as opposed to using dense sensing matrices, while showing a competitive recovery performance.

no code implementations • 5 Mar 2016 • Quoc Tran-Dinh, Anastasios Kyrillidis, Volkan Cevher

First, it allows handling non-smooth objectives via proximal operators; this avoids lifting the problem dimension in order to accommodate non-smooth components in optimization.

no code implementations • 11 Feb 2016 • Jonathan Scarlett, Volkan Cevher

We adopt an \emph{approximate recovery} criterion that allows for a number of missed edges or incorrectly-included edges, in contrast with the widely-studied exact recovery problem.

no code implementations • 2 Feb 2016 • Jonathan Scarlett, Volkan Cevher

In this paper, we study the information-theoretic limits of community detection in the symmetric two-community stochastic block model, with intra-community and inter-community edge probabilities $\frac{a}{n}$ and $\frac{b}{n}$ respectively.

no code implementations • 1 Feb 2016 • Yen-Huan Li, Volkan Cevher

The standard approach to compressive sampling considers recovering an unknown deterministic signal with certain known structure, and designing the sub-sampling pattern and recovery algorithm based on the known structure.

no code implementations • 25 Jan 2016 • Ilija Bogunovic, Jonathan Scarlett, Volkan Cevher

We illustrate the performance of the algorithms on both synthetic and real data, and we find the gradual forgetting of TV-GP-UCB to perform favorably compared to the sharp resetting of R-GP-UCB.

no code implementations • NeurIPS 2015 • David E. Carlson, Edo Collins, Ya-Ping Hsieh, Lawrence Carin, Volkan Cevher

These challenges include, but are not limited to, the non-convexity of learning objectives and estimating the quantities needed for optimization algorithms, such as gradients.

no code implementations • NeurIPS 2015 • Alp Yurtsever, Quoc Tran Dinh, Volkan Cevher

We propose a new primal-dual algorithmic framework for a prototypical constrained convex optimization template.

no code implementations • 21 Oct 2015 • Luca Baldassarre, Yen-Huan Li, Jonathan Scarlett, Baran Gözcü, Ilija Bogunovic, Volkan Cevher

In this paper, we instead take a principled learning-based approach in which a \emph{fixed} index set is chosen based on a set of training signals $\mathbf{x}_1,\dotsc,\mathbf{x}_m$.

no code implementations • 20 Jul 2015 • Volkan Cevher, Sina Jafarpour, Anastasios Kyrillidis

We describe two nonconventional algorithms for linear regression, called GAME and CLASH.

no code implementations • 20 Jul 2015 • Anastasios Kyrillidis, Luca Baldassarre, Marwa El-Halabi, Quoc Tran-Dinh, Volkan Cevher

For each, we present the models in their discrete nature, discuss how to solve the ensuing discrete problems and then describe convex relaxations.

no code implementations • 14 Jul 2015 • Quoc Tran-Dinh, Volkan Cevher

We propose two new alternating direction methods to solve "fully" nonsmooth constrained convex problems.

2 code implementations • 11 Mar 2015 • Joao F. C. Mota, Nikos Deligiannis, Aswin C. Sankaranarayanan, Volkan Cevher, Miguel R. D. Rodrigues

We propose and analyze an online algorithm for reconstructing a sequence of signals from a limited number of linear measurements.

no code implementations • 4 Feb 2015 • Quoc Tran-Dinh, Yen-Huan Li, Volkan Cevher

The self-concordant-like property of a smooth convex function is a new analytical structure that generalizes the self-concordant notion.

no code implementations • 29 Jan 2015 • Jonathan Scarlett, Volkan Cevher

In several cases, our bounds not only provide matching scaling laws in the necessary and sufficient number of measurements, but also sharp thresholds with matching constant factors.

no code implementations • NeurIPS 2014 • John J. Bruer, Joel A. Tropp, Volkan Cevher, Stephen Becker

This paper proposes a tradeoff between sample complexity and computation time that applies to statistical estimators based on convex optimization.

no code implementations • NeurIPS 2014 • Quoc Tran-Dinh, Volkan Cevher

We introduce a model-based excessive gap technique to analyze first-order primal- dual methods for constrained convex minimization.

no code implementations • 7 Nov 2014 • Marwa El Halabi, Volkan Cevher

This paper describes a simple framework for structured sparse recovery based on convex optimization.

no code implementations • 4 Nov 2014 • Volkan Cevher, Stephen Becker, Mark Schmidt

This article reviews recent advances in convex optimization algorithms for Big Data, which aim to reduce the computational, storage, and communications bottlenecks.

no code implementations • 20 Jun 2014 • Quoc Tran-Dinh, Volkan Cevher

Our main analysis technique provides a fresh perspective on Nesterov's excessive gap technique in a structured fashion and unifies it with smoothing and primal-dual methods.

1 code implementation • 4 Jun 2014 • Aleksandr Aravkin, Stephen Becker, Volkan Cevher, Peder Olsen

We introduce a new convex formulation for stable principal component pursuit (SPCP) to decompose noisy signals into low-rank and sparse representations.

no code implementations • 13 May 2014 • Anastasios Kyrillidis, Rabeeh Karimi Mahabadi, Quoc Tran-Dinh, Volkan Cevher

We consider the class of convex minimization problems, composed of a self-concordant function, such as the $\log\det$ metric, a convex data fidelity term $h(\cdot)$ and, a regularizing -- possibly non-smooth -- function $g(\cdot)$.

no code implementations • NeurIPS 2013 • Josip Djolonga, Andreas Krause, Volkan Cevher

Many applications in machine learning require optimizing unknown functions defined over a high-dimensional space from noisy samples that are expensive to obtain.

no code implementations • 7 Nov 2013 • Quoc Tran Dinh, Anastasios Kyrillidis, Volkan Cevher

Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets.

no code implementations • 7 Oct 2013 • Hemant Tyagi, Volkan Cevher

We consider the problem of learning multi-ridge functions of the form f(x) = g(Ax) from point evaluations of f. We assume that the function f is defined on an l_2-ball in R^d, g is twice continuously differentiable almost everywhere, and A \in R^{k \times d} is a rank k matrix, where k << d. We propose a randomized, polynomial-complexity sampling scheme for estimating such functions.

no code implementations • 13 Aug 2013 • Quoc Tran-Dinh, Anastasios Kyrillidis, Volkan Cevher

We propose a variable metric framework for minimizing the sum of a self-concordant function and a possibly non-smooth convex function, endowed with an easily computable proximal operator.

no code implementations • 12 Jul 2013 • Bubacarr Bah, Ali Sadeghian, Volkan Cevher

We propose a dimensionality reducing matrix design based on training data with constraints on its Frobenius norm and number of rows.

no code implementations • 13 Mar 2013 • Luca Baldassarre, Nirav Bhan, Volkan Cevher, Anastasios Kyrillidis, Siddhartha Satpathi

Group-based sparsity models are proven instrumental in linear regression problems for recovering signals from much fewer measurements than standard compressive sensing.

no code implementations • NeurIPS 2012 • Tyagi Hemant, Volkan Cevher

We consider the problem of actively learning \textit{multi-index} functions of the form $f(\vecx) = g(\matA\vecx)= \sum_{i=1}^k g_i(\veca_i^T\vecx)$ from point evaluations of $f$.

no code implementations • NeurIPS 2009 • Volkan Cevher

By using order statistics, we show that N-sample iid realizations of generalized Pareto, Student’s t, log-normal, Frechet, and log-logistic distributions are compressible, i. e., they have a constant expected decay rate, which is independent of N. In contrast, we show that generalized Gaussian distribution with shape parameter q is compressible only in restricted cases since the expected decay rate of its N-sample iid realizations decreases with N as 1/[q log(N/q)].

no code implementations • NeurIPS 2008 • Volkan Cevher, Marco F. Duarte, Chinmay Hegde, Richard Baraniuk

Compressive Sensing (CS) combines sampling and compression into a single sub-Nyquist linear measurement process for sparse and compressible signals.

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.