Search Results for author:
Found 32 papers, 6 papers with code
A simpler approach to accelerated optimization: iterative averaging meets optimism
In this paper, we show that there is a simpler approach to obtaining accelerated rates: applying generic, well-known optimistic online learning algorithms and using the online average of their predictions to query the (deterministic or stochastic) first-order optimization oracle at each time step.
Deep Generative Sampling in the Dual Divergence Space: A Data-efficient & Interpretative Approach for Generative AI
In addition to the data efficiency gained from direct sampling, we propose an algorithm that offers a significant reduction in sample complexity for estimating the divergence of the data distribution with respect to the marginal distribution.
From Inverse Optimization to Feasibility to ERM
For a linear prediction model, we reduce CILP to a convex feasibility problem allowing the use of standard algorithms such as alternating projections.
Learning to Abstain From Uninformative Data
Learning and decision-making in domains with naturally high noise-to-signal ratio, such as Finance or Healthcare, is often challenging, while the stakes are very high.
Variational Principles for Mirror Descent and Mirror Langevin Dynamics
Mirror descent, introduced by Nemirovski and Yudin in the 1970s, is a primal-dual convex optimization method that can be tailored to the geometry of the optimization problem at hand through the choice of a strongly convex potential function.
Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions
Very recently, new generalization bounds have been proven, indicating a non-monotonic relationship between the generalization error and heavy tails, which is more pertinent to the reported empirical observations.
MAViC: Multimodal Active Learning for Video Captioning
Our approach integrates semantic similarity and uncertainty of both visual and language dimensions in the acquisition function.
Explicit Regularization in Overparametrized Models via Noise Injection
Injecting noise within gradient descent has several desirable features, such as smoothing and regularizing properties.
Utilising the CLT Structure in Stochastic Gradient based Sampling : Improved Analysis and Faster Algorithms
We consider stochastic approximations of sampling algorithms, such as Stochastic Gradient Langevin Dynamics (SGLD) and the Random Batch Method (RBM) for Interacting Particle Dynamcs (IPD).
Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares
Recent studies have shown that heavy tails can emerge in stochastic optimization and that the heaviness of the tails have links to the generalization error.
Convergence of Uncertainty Sampling for Active Learning
Uncertainty sampling in active learning is heavily used in practice to reduce the annotation cost.
Non-stationary Online Regression
To the best of our knowledge, this work is the first extension of non-stationary online regression to non-stationary kernel regression.
Model-specific Data Subsampling with Influence Functions
Model selection requires repeatedly evaluating models on a given dataset and measuring their relative performances.
Stochastic Stein Discrepancies
Stein discrepancies (SDs) monitor convergence and non-convergence in approximate inference when exact integration and sampling are intractable.
Causal Feature Selection via Orthogonal Search
The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines.
Explicit Regularization of Stochastic Gradient Methods through Duality
For accelerated coordinate descent, we obtain a new algorithm that has better convergence properties than existing stochastic gradient methods in the interpolating regime.
Importance Sampling via Local Sensitivity
Unfortunately, sensitivity sampling is difficult to apply since (1) it is unclear how to efficiently compute the sensitivity scores and (2) the sample size required is often impractically large.
Dual Instrumental Variable Regression
We present a novel algorithm for non-linear instrumental variable (IV) regression, DualIV, which simplifies traditional two-stage methods via a dual formulation.
Orthogonal Structure Search for Efficient Causal Discovery from Observational Data
The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines.
A Differentially Private Kernel Two-Sample Test
As a result, a simple chi-squared test is obtained, where a test statistic depends on a mean and covariance of empirical differences between the samples, which we perturb for a privacy guarantee.
Sobolev Descent
We study a simplification of GAN training: the problem of transporting particles from a source to a target distribution.
k-SVRG: Variance Reduction for Large Scale Optimization
Variance reduced stochastic gradient (SGD) methods converge significantly faster than the vanilla SGD counterpart.
On Matching Pursuit and Coordinate Descent
Exploiting the connection between the two algorithms, we present a unified analysis of both, providing affine invariant sublinear $\mathcal{O}(1/t)$ rates on smooth objectives and linear convergence on strongly convex objectives.
Sobolev GAN
We show that the Sobolev IPM compares two distributions in high dimensions based on weighted conditional Cumulative Distribution Functions (CDF) of each coordinate on a leave one out basis.
Safe Adaptive Importance Sampling
Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications.
Approximate Steepest Coordinate Descent
We propose a new selection rule for the coordinate selection in coordinate descent methods for huge-scale optimization.
Local Group Invariant Representations via Orbit Embeddings
We consider transformations that form a \emph{group} and propose an approach based on kernel methods to derive local group invariant representations.
Screening Rules for Convex Problems
We propose a new framework for deriving screening rules for convex optimization problems.
Unsupervised Domain Adaptation in the Wild: Dealing with Asymmetric Label Sets
Most methods for unsupervised domain adaptation proposed in the literature to date, assume that the set of classes present in the target domain is identical to the set of classes present in the source domain.
Subspace Alignment Based Domain Adaptation for RCNN Detector
In this paper, we propose subspace alignment based domain adaptation of the state of the art RCNN based object detector.
Mind the Gap: Subspace based Hierarchical Domain Adaptation
Domain adaptation techniques aim at adapting a classifier learnt on a source domain to work on the target domain.
Scalable Kernel Methods via Doubly Stochastic Gradients
The general perception is that kernel methods are not scalable, and neural nets are the methods of choice for nonlinear learning problems.
