no code implementations • ICML 2020 • Pooria Joulani, Anant Raj, András György, Csaba Szepesvari
In this paper, we show that there is a simpler approach to obtaining accelerated rates: applying generic, well-known optimistic online learning algorithms and using the online average of their predictions to query the (deterministic or stochastic) first-order optimization oracle at each time step.
no code implementations • 4 Jan 2025 • Yikai Zhang, Jiahe Lin, Fengpei Li, Songzhu Zheng, Anant Raj, Anderson Schneider, Yuriy Nevmyvaka
In this work, we study the weighted empirical risk minimization (weighted ERM) schema, in which an additional data-dependent weight function is incorporated when the empirical risk function is being minimized.
1 code implementation • 21 May 2024 • Michael Lu, Matin Aghaei, Anant Raj, Sharan Vaswani
We show that the proposed algorithm offers similar theoretical guarantees as the state-of-the art results, but does not require the knowledge of oracle-like quantities.
no code implementations • 10 Apr 2024 • Sahil Garg, Anderson Schneider, Anant Raj, Kashif Rasul, Yuriy Nevmyvaka, Sneihil Gopal, Amit Dhurandhar, Guillermo Cecchi, Irina Rish
In addition to the data efficiency gained from direct sampling, we propose an algorithm that offers a significant reduction in sample complexity for estimating the divergence of the data distribution with respect to the marginal distribution.
1 code implementation • 27 Feb 2024 • Saurabh Mishra, Anant Raj, Sharan Vaswani
Inverse optimization involves inferring unknown parameters of an optimization problem from known solutions and is widely used in fields such as transportation, power systems, and healthcare.
no code implementations • 25 Sep 2023 • Yikai Zhang, Songzhu Zheng, Mina Dalirrooyfard, Pengxiang Wu, Anderson Schneider, Anant Raj, Yuriy Nevmyvaka, Chao Chen
Learning and decision-making in domains with naturally high noise-to-signal ratio, such as Finance or Healthcare, is often challenging, while the stakes are very high.
no code implementations • 16 Mar 2023 • Belinda Tzen, Anant Raj, Maxim Raginsky, Francis Bach
Mirror descent, introduced by Nemirovski and Yudin in the 1970s, is a primal-dual convex optimization method that can be tailored to the geometry of the optimization problem at hand through the choice of a strongly convex potential function.
no code implementations • 27 Jan 2023 • Anant Raj, Lingjiong Zhu, Mert Gürbüzbalaban, Umut Şimşekli
Very recently, new generalization bounds have been proven, indicating a non-monotonic relationship between the generalization error and heavy tails, which is more pertinent to the reported empirical observations.
no code implementations • 11 Dec 2022 • Gyanendra Das, Xavier Thomas, Anant Raj, Vikram Gupta
Our approach integrates semantic similarity and uncertainty of both visual and language dimensions in the acquisition function.
1 code implementation • 9 Jun 2022 • Antonio Orvieto, Anant Raj, Hans Kersting, Francis Bach
Injecting noise within gradient descent has several desirable features, such as smoothing and regularizing properties.
no code implementations • 8 Jun 2022 • Aniket Das, Dheeraj Nagaraj, Anant Raj
We consider stochastic approximations of sampling algorithms, such as Stochastic Gradient Langevin Dynamics (SGLD) and the Random Batch Method (RBM) for Interacting Particle Dynamcs (IPD).
no code implementations • 2 Jun 2022 • Anant Raj, Melih Barsbey, Mert Gürbüzbalaban, Lingjiong Zhu, Umut Şimşekli
Recent studies have shown that heavy tails can emerge in stochastic optimization and that the heaviness of the tails have links to the generalization error.
no code implementations • 29 Oct 2021 • Anant Raj, Francis Bach
Uncertainty sampling in active learning is heavily used in practice to reduce the annotation cost.
no code implementations • 13 Nov 2020 • Anant Raj, Pierre Gaillard, Christophe Saad
To the best of our knowledge, this work is the first extension of non-stationary online regression to non-stationary kernel regression.
no code implementations • 20 Oct 2020 • Anant Raj, Cameron Musco, Lester Mackey, Nicolo Fusi
Model selection requires repeatedly evaluating models on a given dataset and measuring their relative performances.
1 code implementation • NeurIPS 2020 • Jackson Gorham, Anant Raj, Lester Mackey
Stein discrepancies (SDs) monitor convergence and non-convergence in approximate inference when exact integration and sampling are intractable.
no code implementations • 6 Jul 2020 • Ashkan Soleymani, Anant Raj, Stefan Bauer, Bernhard Schölkopf, Michel Besserve
The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines.
no code implementations • 30 Mar 2020 • Anant Raj, Francis Bach
For accelerated coordinate descent, we obtain a new algorithm that has better convergence properties than existing stochastic gradient methods in the interpolating regime.
no code implementations • 4 Nov 2019 • Anant Raj, Cameron Musco, Lester Mackey
Unfortunately, sensitivity sampling is difficult to apply since (1) it is unclear how to efficiently compute the sensitivity scores and (2) the sample size required is often impractically large.
1 code implementation • NeurIPS 2020 • Krikamol Muandet, Arash Mehrjou, Si Kai Lee, Anant Raj
We present a novel algorithm for non-linear instrumental variable (IV) regression, DualIV, which simplifies traditional two-stage methods via a dual formulation.
no code implementations • 6 Mar 2019 • Anant Raj, Luigi Gresele, Michel Besserve, Bernhard Schölkopf, Stefan Bauer
The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines.
2 code implementations • 1 Aug 2018 • Anant Raj, Ho Chung Leon Law, Dino Sejdinovic, Mijung Park
As a result, a simple chi-squared test is obtained, where a test statistic depends on a mean and covariance of empirical differences between the samples, which we perturb for a privacy guarantee.
no code implementations • 30 May 2018 • Youssef Mroueh, Tom Sercu, Anant Raj
We study a simplification of GAN training: the problem of transporting particles from a source to a target distribution.
no code implementations • 2 May 2018 • Anant Raj, Sebastian U. Stich
Variance reduced stochastic gradient (SGD) methods converge significantly faster than the vanilla SGD counterpart.
no code implementations • ICML 2018 • Francesco Locatello, Anant Raj, Sai Praneeth Karimireddy, Gunnar Rätsch, Bernhard Schölkopf, Sebastian U. Stich, Martin Jaggi
Exploiting the connection between the two algorithms, we present a unified analysis of both, providing affine invariant sublinear $\mathcal{O}(1/t)$ rates on smooth objectives and linear convergence on strongly convex objectives.
2 code implementations • ICLR 2018 • Youssef Mroueh, Chun-Liang Li, Tom Sercu, Anant Raj, Yu Cheng
We show that the Sobolev IPM compares two distributions in high dimensions based on weighted conditional Cumulative Distribution Functions (CDF) of each coordinate on a leave one out basis.
no code implementations • NeurIPS 2017 • Sebastian U. Stich, Anant Raj, Martin Jaggi
Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications.
no code implementations • ICML 2017 • Sebastian U. Stich, Anant Raj, Martin Jaggi
We propose a new selection rule for the coordinate selection in coordinate descent methods for huge-scale optimization.
no code implementations • 6 Dec 2016 • Anant Raj, Abhishek Kumar, Youssef Mroueh, P. Thomas Fletcher, Bernhard Schölkopf
We consider transformations that form a \emph{group} and propose an approach based on kernel methods to derive local group invariant representations.
no code implementations • 23 Sep 2016 • Anant Raj, Jakob Olbrich, Bernd Gärtner, Bernhard Schölkopf, Martin Jaggi
We propose a new framework for deriving screening rules for convex optimization problems.
no code implementations • 26 Mar 2016 • Ayush Mittal, Anant Raj, Vinay P. Namboodiri, Tinne Tuytelaars
Most methods for unsupervised domain adaptation proposed in the literature to date, assume that the set of classes present in the target domain is identical to the set of classes present in the source domain.
no code implementations • 20 Jul 2015 • Anant Raj, Vinay P. Namboodiri, Tinne Tuytelaars
In this paper, we propose subspace alignment based domain adaptation of the state of the art RCNN based object detector.
no code implementations • 16 Jan 2015 • Anant Raj, Vinay P. Namboodiri, Tinne Tuytelaars
Domain adaptation techniques aim at adapting a classifier learnt on a source domain to work on the target domain.
1 code implementation • NeurIPS 2014 • Bo Dai, Bo Xie, Niao He, YIngyu Liang, Anant Raj, Maria-Florina Balcan, Le Song
The general perception is that kernel methods are not scalable, and neural nets are the methods of choice for nonlinear learning problems.