Search Results for author:
Found 42 papers, 4 papers with code
Non-Stationary Bandits with Intermediate Observations
In fact, if the timescale of the change is comparable to the delay, it is impossible to learn about the environment, since the available observations are already obsolete.
A simpler approach to accelerated optimization: iterative averaging meets optimism
In this paper, we show that there is a simpler approach to obtaining accelerated rates: applying generic, well-known optimistic online learning algorithms and using the online average of their predictions to query the (deterministic or stochastic) first-order optimization oracle at each time step.
Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset
Neural networks are traditionally trained under the assumption that data come from a stationary distribution.
Toward Understanding In-context vs. In-weight Learning
These theoretical findings are then corroborated experimentally by comparing the behaviour of a full transformer on the simplified distributions to that of the stylized model, demonstrating aligned results.
Learning Continually by Spectral Regularization
From this perspective, we derive a new spectral regularizer for continual learning that better sustains these beneficial initialization properties throughout training.
To Believe or Not to Believe Your LLM
Such quantification, for instance, allows to detect hallucinations (cases when epistemic uncertainty is high) in both single- and multi-answer responses.
Mitigating LLM Hallucinations via Conformal Abstention
We develop a principled procedure for determining when a large language model (LLM) should abstain from responding (e. g., by saying "I don't know") in a general domain, instead of resorting to possibly "hallucinating" a non-sensical or incorrect answer.
Prior-Dependent Allocations for Bayesian Fixed-Budget Best-Arm Identification in Structured Bandits
We study the problem of Bayesian fixed-budget best-arm identification (BAI) in structured bandits.
Online RL in Linearly $q^π$-Realizable MDPs Is as Easy as in Linear MDPs If You Learn What to Ignore
We consider online reinforcement learning (RL) in episodic Markov decision processes (MDPs) under the linear $q^\pi$-realizability assumption, where it is assumed that the action-values of all policies can be expressed as linear functions of state-action features.
Optimistic Natural Policy Gradient: a Simple Efficient Policy Optimization Framework for Online RL
While policy optimization algorithms have played an important role in recent empirical success of Reinforcement Learning (RL), the existing theoretical understanding of policy optimization remains rather limited -- they are either restricted to tabular MDPs or suffer from highly suboptimal sample complexity, especial in online RL where exploration is necessary.
A Second-Order Method for Stochastic Bandit Convex Optimisation
We introduce a simple and efficient algorithm for unconstrained zeroth-order stochastic convex bandits and prove its regret is at most $(1 + r/d)[d^{1. 5} \sqrt{n} + d^3] polylog(n, d, r)$ where $n$ is the horizon, $d$ the dimension and $r$ is the radius of a known ball containing the minimiser of the loss.
Generalization Bounds for Few-Shot Transfer Learning with Pretrained Classifiers
We study the ability of foundation models to learn representations for classification that are transferable to new, unseen classes.
Understanding Self-Predictive Learning for Reinforcement Learning
We identify that a faster paced optimization of the predictor and semi-gradient updates on the representation, are crucial to preventing the representation collapse.
Confident Approximate Policy Iteration for Efficient Local Planning in $q^π$-realizable MDPs
Our first contribution is a new variant of Approximate Policy Iteration (API), called Confident Approximate Policy Iteration (CAPI), which computes a deterministic stationary policy with an optimal error bound scaling linearly with the product of the effective horizon $H$ and the worst-case approximation error $\epsilon$ of the action-value functions of stationary policies.
Distributed Contextual Linear Bandits with Minimax Optimal Communication Cost
In particular, for scenarios with known context distribution, the communication cost of DisBE-LUCB is only $\tilde{\mathcal{O}}(dN)$ and its regret is ${\tilde{\mathcal{O}}}(\sqrt{dNT})$, which is of the same order as that incurred by an optimal single-agent algorithm for $NT$ rounds.
Non-stationary Bandits and Meta-Learning with a Small Set of Optimal Arms
We study a sequential decision problem where the learner faces a sequence of $K$-armed bandit tasks.
On the Role of Neural Collapse in Transfer Learning
We study the ability of foundation models to learn representations for classification that are transferable to new, unseen classes.
TensorPlan and the Few Actions Lower Bound for Planning in MDPs under Linear Realizability of Optimal Value Functions
Furthermore, we show that the upper bound of TensorPlan can be extended to hold under (iii) and, for MDPs with deterministic transitions and stochastic rewards, also under (ii).
Mutual Information Constraints for Monte-Carlo Objectives
A common failure mode of density models trained as variational autoencoders is to model the data without relying on their latent variables, rendering these variables useless.
ImpatientCapsAndRuns: Approximately Optimal Algorithm Configuration from an Infinite Pool
Algorithm configuration procedures optimize parameters of a given algorithm to perform well over a distribution of inputs.
Adapting to Delays and Data in Adversarial Multi-Armed Bandits
We consider the adversarial multi-armed bandit problem under delayed feedback.
Mirror Descent and the Information Ratio
We establish a connection between the stability of mirror descent and the information ratio by Russo and Van Roy [2014].
Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting
We consider off-policy evaluation in the contextual bandit setting for the purpose of obtaining a robust off-policy selection strategy, where the selection strategy is evaluated based on the value of the chosen policy in a set of proposal (target) policies.
Think out of the "Box": Generically-Constrained Asynchronous Composite Optimization and Hedging
ASYNCADA is, to our knowledge, the first asynchronous stochastic optimization algorithm with finite-time data-dependent convergence guarantees for generic convex constraints.
Meta-learning of Sequential Strategies
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class.
Detecting Overfitting via Adversarial Examples
It utilizes a new unbiased error estimate that is based on adversarial examples generated from the test data and importance weighting.
Degenerate Feedback Loops in Recommender Systems
Machine learning is used extensively in recommender systems deployed in products.
Learning from Delayed Outcomes via Proxies with Applications to Recommender Systems
Predicting delayed outcomes is an important problem in recommender systems (e. g., if customers will finish reading an ebook).
LeapsAndBounds: A Method for Approximately Optimal Algorithm Configuration
We consider the problem of configuring general-purpose solvers to run efficiently on problem instances drawn from an unknown distribution.
Adaptive MCMC via Combining Local Samplers
Here we take a different approach and, similarly to parallel MCMC methods, instead of trying to find a single chain that samples from the whole distribution, we combine samples from several chains run in parallel, each exploring only parts of the state space (e. g., a few modes only).
A Reinforcement Learning Approach to Age of Information in Multi-User Networks
Scheduling the transmission of time-sensitive data to multiple users over error-prone communication channels is studied with the goal of minimizing the long-term average age of information (AoI) at the users under a constraint on the average number of transmissions at the source node.
A Modular Analysis of Adaptive (Non-)Convex Optimization: Optimism, Composite Objectives, and Variational Bounds
Recently, much work has been done on extending the scope of online learning and incremental stochastic optimization algorithms.
Following the Leader and Fast Rates in Linear Prediction: Curved Constraint Sets and Other Regularities
The follow the leader (FTL) algorithm, perhaps the simplest of all online learning algorithms, is known to perform well when the loss functions it is used on are convex and positively curved.
SDP Relaxation with Randomized Rounding for Energy Disaggregation
We develop a scalable, computationally efficient method for the task of energy disaggregation for home appliance monitoring.
(Bandit) Convex Optimization with Biased Noisy Gradient Oracles
Algorithms for bandit convex optimization and online learning often rely on constructing noisy gradient estimates, which are then used in appropriately adjusted first-order algorithms, replacing actual gradients.
Chaining Bounds for Empirical Risk Minimization
This paper extends the standard chaining technique to prove excess risk upper bounds for empirical risk minimization with random design settings even if the magnitude of the noise and the estimates is unbounded.
Online Learning with Gaussian Payoffs and Side Observations
For the first time in the literature, we provide non-asymptotic problem-dependent lower bounds on the regret of any algorithm, which recover existing asymptotic problem-dependent lower bounds and finite-time minimax lower bounds available in the literature.
Fast Cross-Validation for Incremental Learning
Cross-validation (CV) is one of the main tools for performance estimation and parameter tuning in machine learning.
Adaptive Monte Carlo via Bandit Allocation
We consider the problem of sequentially choosing between a set of unbiased Monte Carlo estimators to minimize the mean-squared-error (MSE) of a final combined estimate.
Efficient Multi-Start Strategies for Local Search Algorithms
In particular, we prove that at most a quadratic increase in the number of times the target function is evaluated is needed to achieve the performance of a local search algorithm started from the attraction region of the optimum.
Online Learning under Delayed Feedback
Online learning with delayed feedback has received increasing attention recently due to its several applications in distributed, web-based learning problems.
Online Markov Decision Processes under Bandit Feedback
We consider online learning in finite stochastic Markovian environments where in each time step a new reward function is chosen by an oblivious adversary.
