Search Results for author:
Found 49 papers, 3 papers with code
Online Learning with Unknown Constraints
We consider the problem of online learning where the sequence of actions played by the learner must adhere to an unknown safety constraint at every round.
Contextual Bandits and Imitation Learning via Preference-Based Active Queries
We consider the problem of contextual bandits and imitation learning, where the learner lacks direct knowledge of the executed action's reward.
From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descent
When these potentials further satisfy certain self-bounding properties, we show that they can be used to provide a convergence guarantee for Gradient Descent (GD) and SGD (even when the paths of GF and GD/SGD are quite far apart).
On the Complexity of Adversarial Decision Making
A central problem in online learning and decision making -- from bandits to reinforcement learning -- is to understand what modeling assumptions lead to sample-efficient learning guarantees.
Guarantees for Epsilon-Greedy Reinforcement Learning with Function Approximation
This paper presents a theoretical analysis of such policies and provides the first regret and sample-complexity bounds for reinforcement learning with myopic exploration.
SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs
We show that there is an SCO problem such that GD with any step size and number of iterations can only learn at a suboptimal rate: at least $\widetilde{\Omega}(1/n^{5/12})$.
Agnostic Reinforcement Learning with Low-Rank MDPs and Rich Observations
In this work, we consider the more realistic setting of agnostic RL with rich observation spaces and a fixed class of policies $\Pi$ that may not contain any near-optimal policy.
Online learning with dynamics: A minimax perspective
In this setting, we study the problem of minimizing policy regret and provide non-constructive upper bounds on the minimax rate for the problem.
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations
We design an algorithm which finds an $\epsilon$-approximate stationary point (with $\|\nabla F(x)\|\le \epsilon$) using $O(\epsilon^{-3})$ stochastic gradient and Hessian-vector products, matching guarantees that were previously available only under a stronger assumption of access to multiple queries with the same random seed.
Reinforcement Learning with Feedback Graphs
We study episodic reinforcement learning in Markov decision processes when the agent receives additional feedback per step in the form of several transition observations.
Hypothesis Set Stability and Generalization
Our main result is a generalization bound for data-dependent hypothesis sets expressed in terms of a notion of hypothesis set stability and a notion of Rademacher complexity for data-dependent hypothesis sets that we introduce.
Distributed Learning with Sublinear Communication
In distributed statistical learning, $N$ samples are split across $m$ machines and a learner wishes to use minimal communication to learn as well as if the examples were on a single machine.
The Complexity of Making the Gradient Small in Stochastic Convex Optimization
Notably, we show that in the global oracle/statistical learning model, only logarithmic dependence on smoothness is required to find a near-stationary point, whereas polynomial dependence on smoothness is necessary in the local stochastic oracle model.
Uniform Convergence of Gradients for Non-Convex Learning and Optimization
We investigate 1) the rate at which refined properties of the empirical risk---in particular, gradients---converge to their population counterparts in standard non-convex learning tasks, and 2) the consequences of this convergence for optimization.
Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals
This new formulation leads to an algorithm that produces a stochastic classifier by playing a two-player non-zero-sum game solving for what we call a semi-coarse correlated equilibrium, which in turn corresponds to an approximately optimal and feasible solution to the constrained optimization problem.
Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints
Classifiers can be trained with data-dependent constraints to satisfy fairness goals, reduce churn, achieve a targeted false positive rate, or other policy goals.
Two-Player Games for Efficient Non-Convex Constrained Optimization
For both the proxy-Lagrangian and Lagrangian formulations, however, we prove that this classifier, instead of having unbounded size, can be taken to be a distribution over no more than m+1 models (where m is the number of constraints).
Logistic Regression: The Importance of Being Improper
Starting with the simple observation that the logistic loss is $1$-mixable, we design a new efficient improper learning algorithm for online logistic regression that circumvents the aforementioned lower bound with a regret bound exhibiting a doubly-exponential improvement in dependence on the predictor norm.
Online Learning: Sufficient Statistics and the Burkholder Method
We uncover a fairly general principle in online learning: If regret can be (approximately) expressed as a function of certain "sufficient statistics" for the data sequence, then there exists a special Burkholder function that 1) can be used algorithmically to achieve the regret bound and 2) only depends on these sufficient statistics, not the entire data sequence, so that the online strategy is only required to keep the sufficient statistics in memory.
Parameter-free online learning via model selection
We introduce an efficient algorithmic framework for model selection in online learning, also known as parameter-free online learning.
Small-loss bounds for online learning with partial information
We develop a black-box approach for such problems where the learner observes as feedback only losses of a subset of the actions that includes the selected action.
ZigZag: A new approach to adaptive online learning
To develop a general theory of when this type of adaptive regret bound is achievable we establish a connection to the theory of decoupling inequalities for martingales in Banach spaces.
Inference in Sparse Graphs with Pairwise Measurements and Side Information
For two-dimensional grids, our results improve over Globerson et al. (2015) by obtaining optimal recovery in the constant-height regime.
A Tutorial on Online Supervised Learning with Applications to Node Classification in Social Networks
We revisit the elegant observation of T. Cover '65 which, perhaps, is not as well-known to the broader community as it should be.
Learning in Games: Robustness of Fast Convergence
We show that learning algorithms satisfying a $\textit{low approximate regret}$ property experience fast convergence to approximate optimality in a large class of repeated games.
BISTRO: An Efficient Relaxation-Based Method for Contextual Bandits
We present efficient algorithms for the problem of contextual bandits with i. i. d.
Exploiting the Structure: Stochastic Gradient Methods Using Raw Clusters
The amount of data available in the world is growing faster than our ability to deal with it.
Private Causal Inference
Causal inference has the potential to have significant impact on medical research, prevention and control of diseases, and identifying factors that impact economic changes to name just a few.
On Equivalence of Martingale Tail Bounds and Deterministic Regret Inequalities
We study an equivalence of (i) deterministic pathwise statements appearing in the online learning literature (termed \emph{regret bounds}), (ii) high-probability tail bounds for the supremum of a collection of martingales (of a specific form arising from uniform laws of large numbers for martingales), and (iii) in-expectation bounds for the supremum.
Adaptive Online Learning
We propose a general framework for studying adaptive regret bounds in the online learning framework, including model selection bounds and data-dependent bounds.
Hierarchies of Relaxations for Online Prediction Problems with Evolving Constraints
We study online prediction where regret of the algorithm is measured against a benchmark defined via evolving constraints.
Learning with Square Loss: Localization through Offset Rademacher Complexity
We consider regression with square loss and general classes of functions without the boundedness assumption.
Sequential Probability Assignment with Binary Alphabets and Large Classes of Experts
We analyze the problem of sequential probability assignment for binary outcomes with side information and logarithmic loss, where regret---or, redundancy---is measured with respect to a (possibly infinite) class of experts.
Online Nonparametric Regression with General Loss Functions
This paper establishes minimax rates for online regression with arbitrary classes of functions and general losses.
Online Optimization : Competing with Dynamic Comparators
Recent literature on online learning has focused on developing adaptive algorithms that take advantage of a regularity of the sequence of observations, yet retain worst-case performance guarantees.
Online Nonparametric Regression
The optimal rates are shown to exhibit a phase transition analogous to the i. i. d./statistical learning case, studied in (Rakhlin, Sridharan, Tsybakov 2013).
Optimization, Learning, and Games with Predictable Sequences
We provide several applications of Optimistic Mirror Descent, an online learning algorithm based on the idea of predictable sequences.
Empirical entropy, minimax regret and minimax risk
Furthermore, for $p\in(0, 2)$, the excess risk rate matches the behavior of the minimax risk of function estimation in regression problems under the well-specified model.
Relax and Randomize : From Value to Algorithms
We show a principled way of deriving online learning algorithms from a minimax analysis.
Online Learning with Predictable Sequences
Variance and path-length bounds can be seen as particular examples of online learning with simple predictable sequences.
Better Mini-Batch Algorithms via Accelerated Gradient Methods
Mini-batch algorithms have recently received significant attention as a way to speed-up stochastic convex optimization problems.
Online Learning: Stochastic, Constrained, and Smoothed Adversaries
We define the minimax value of a game where the adversary is restricted in his moves, capturing stochastic and non-stochastic assumptions on data.
On the Universality of Online Mirror Descent
We show that for a general class of convex online learning problems, Mirror Descent can always achieve a (nearly) optimal regret guarantee.
Smoothness, Low Noise and Fast Rates
We establish an excess risk bound of O(H R_n^2 + sqrt{H L*} R_n) for ERM with an H-smooth loss function and a hypothesis class with Rademacher complexity R_n, where L* is the best risk achievable by the hypothesis class.
Online Learning: Random Averages, Combinatorial Parameters, and Learnability
We develop a theory of online learning by defining several complexity measures.
Online Learning via Sequential Complexities
We consider the problem of sequential prediction and provide tools to study the minimax value of the associated game.
Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity
The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model.
On the Complexity of Linear Prediction: Risk Bounds, Margin Bounds, and Regularization
We provide sharp bounds for Rademacher and Gaussian complexities of (constrained) linear classes.
Fast Rates for Regularized Objectives
We show that the empirical minimizer of a stochastic strongly convex objective, where the stochastic component is linear, converges to the population minimizer with rate $O(1/n)$.
