Search Results for author:
Found 39 papers, 2 papers with code
Bad Values but Good Behavior: Learning Highly Misspecified Bandits and MDPs
Parametric, feature-based reward models are employed by a variety of algorithms in decision-making settings such as bandits and Markov decision processes (MDPs).
A Unified Framework for Discovering Discrete Symmetries
At the core of the framework is a novel architecture composed of linear, matrix-valued and non-linear functions that expresses functions invariant to these subgroups in a principled manner.
On the Minimax Regret for Linear Bandits in a wide variety of Action Spaces
As noted in the works of \cite{lattimore2020bandit}, it has been mentioned that it is an open problem to characterize the minimax regret of linear bandits in a wide variety of action spaces.
Exploration in Linear Bandits with Rich Action Sets and its Implications for Inference
Furthermore, while the previous result is shown to hold only in the asymptotic regime (as $n \to \infty$), our result for these "locally rich" action spaces is any-time.
Actor-Critic based Improper Reinforcement Learning
For the AC-based approach we provide convergence rate guarantees to a stationary point in the basic AC case and to a global optimum in the NAC case.
Demystifying Approximate Value-based RL with $ε$-greedy Exploration: A Differential Inclusion View
Q-learning and SARSA with $\epsilon$-greedy exploration are leading reinforcement learning methods.
Adaptive Estimation of Random Vectors with Bandit Feedback: A mean-squared error viewpoint
We consider the problem of sequentially learning to estimate, in the mean squared error (MSE) sense, a Gaussian $K$-vector of unknown covariance by observing only $m < K$ of its entries in each round.
Bregman Deviations of Generic Exponential Families
For the practitioner, we instantiate this novel bound to several classical families, e. g., Gaussian, Bernoulli, Exponential, Weibull, Pareto, Poisson and Chi-square yielding explicit forms of the confidence sets and the Bregman information gain.
On Slowly-varying Non-stationary Bandits
We also provide the first minimax regret lower bound for this problem, enabling us to show that our algorithm is essentially minimax optimal.
Bandit Quickest Changepoint Detection
Many industrial and security applications employ a suite of sensors for detecting abrupt changes in temporal behavior patterns.
Better than the Best: Gradient-based Improper Reinforcement Learning for Network Scheduling
We consider the problem of scheduling in constrained queueing networks with a view to minimizing packet delay.
Improper Reinforcement Learning with Gradient-based Policy Optimization
We consider an improper reinforcement learning setting where a learner is given $M$ base controllers for an unknown Markov decision process, and wishes to combine them optimally to produce a potentially new controller that can outperform each of the base ones.
Stochastic Linear Bandits with Protected Subspace
We study a variant of the stochastic linear bandit problem wherein we optimize a linear objective function but rewards are accrued only orthogonal to an unknown subspace (which we interpret as a \textit{protected space}) given only zero-order stochastic oracle access to both the objective itself and protected subspace.
No-regret Algorithms for Multi-task Bayesian Optimization
We consider multi-objective optimization (MOO) of an unknown vector-valued function in the non-parametric Bayesian optimization (BO) setting, with the aim being to learn points on the Pareto front of the objectives.
Explicit Best Arm Identification in Linear Bandits Using No-Regret Learners
We study the problem of best arm identification in linearly parameterised multi-armed bandits.
How Reliable are Test Numbers for Revealing the COVID-19 Ground Truth and Applying Interventions?
We use the simulation framework to compare the performance of three testing policies: Random Symptomatic Testing (RST), Contact Tracing (CT), and a new Location Based Testing policy (LBT).
Regret Minimization in Stochastic Contextual Dueling Bandits
We consider the problem of stochastic $K$-armed dueling bandit in the contextual setting, where at each round the learner is presented with a context set of $K$ items, each represented by a $d$-dimensional feature vector, and the goal of the learner is to identify the best arm of each context sets.
Best-item Learning in Random Utility Models with Subset Choices
We consider the problem of PAC learning the most valuable item from a pool of $n$ items using sequential, adaptively chosen plays of subsets of $k$ items, when, upon playing a subset, the learner receives relative feedback sampled according to a general Random Utility Model (RUM) with independent noise perturbations to the latent item utilities.
Sequential Mode Estimation with Oracle Queries
We consider the problem of adaptively PAC-learning a probability distribution $\mathcal{P}$'s mode by querying an oracle for information about a sequence of i. i. d.
Towards Optimal and Efficient Best Arm Identification in Linear Bandits
We give a new algorithm for best arm identification in linearly parameterised bandits in the fixed confidence setting.
On Online Learning in Kernelized Markov Decision Processes
We develop algorithms with low regret for learning episodic Markov decision processes based on kernel approximation techniques.
On Batch Bayesian Optimization
We present two algorithms for Bayesian optimization in the batch feedback setting, based on Gaussian process upper confidence bound and Thompson sampling approaches, along with frequentist regret guarantees and numerical results.
On Adaptivity in Information-constrained Online Learning
We then consider revealing action-partial monitoring games -- a version of label efficient prediction with additive information costs, which in general are known to lie in the \textit{hard} class of games having minimax regret of order $T^{\frac{2}{3}}$.
Bayesian Optimization under Heavy-tailed Payoffs
We resolve this gap by developing novel Bayesian optimization algorithms, based on kernel approximation techniques, with regret bounds matching the lower bound in order for the SE kernel.
Combinatorial Bandits with Relative Feedback
We consider combinatorial online learning with subset choices when only relative feedback information from subsets is available, instead of bandit or semi-bandit feedback which is absolute.
From PAC to Instance-Optimal Sample Complexity in the Plackett-Luce Model
In the setting where subsets of a fixed size can be tested and top-ranked feedback is made available to the learner, we give an algorithm with optimal instance-dependent sample complexity, for PAC best arm identification, of $O\bigg(\frac{\theta_{[k]}}{k}\sum_{i = 2}^n\max\Big(1,\frac{1}{\Delta_i^2}\Big) \ln\frac{k}{\delta}\Big(\ln \frac{1}{\Delta_i}\Big)\bigg)$, $\Delta_i$ being the Plackett-Luce parameter gap between the best and the $i^{th}$ best item, and $\theta_{[k]}$ is the sum of the \pl\, parameters for the top-$k$ items.
Active Ranking with Subset-wise Preferences
When, however, it is possible to elicit top-$m$ ($\leq k$) ranking feedback according to the PL model from each adaptively chosen subset of size $k$, we show that an $(\epsilon, \delta)$-PAC ranking sample complexity of $O\left(\frac{n}{m \epsilon^2} \ln \frac{n}{\delta} \right)$ is achievable with explicit algorithms, which represents an $m$-wise reduction in sample complexity compared to the pairwise case.
PAC Battling Bandits in the Plackett-Luce Model
We introduce the probably approximately correct (PAC) \emph{Battling-Bandit} problem with the Plackett-Luce (PL) subset choice model--an online learning framework where at each trial the learner chooses a subset of $k$ arms from a fixed set of $n$ arms, and subsequently observes a stochastic feedback indicating preference information of the items in the chosen subset, e. g., the most preferred item or ranking of the top $m$ most preferred items etc.
Online Learning in Kernelized Markov Decision Processes
We consider online learning for minimizing regret in unknown, episodic Markov decision processes (MDPs) with continuous states and actions.
Online Learning for Structured Loss Spaces
We consider prediction with expert advice when the loss vectors are assumed to lie in a set described by the sum of atomic norm balls.
Misspecified Linear Bandits
Regret guarantees for state-of-the-art linear bandit algorithms such as Optimism in the Face of Uncertainty Linear bandit (OFUL) hold under the assumption that the arms expected rewards are perfectly linear in their features.
On Kernelized Multi-armed Bandits
We consider the stochastic bandit problem with a continuous set of arms, with the expected reward function over the arms assumed to be fixed but unknown.
Bandit algorithms to emulate human decision making using probabilistic distortions
For the $K$-armed bandit setting, we derive an upper bound on the expected regret for our proposed algorithm, and then we prove a matching lower bound to establish the order-optimality of our algorithm.
Low-rank Bandits with Latent Mixtures
This induces a low-rank structure on the matrix of expected rewards r a, b from recommending item a to user b.
Optimal Recommendation to Users that React: Online Learning for a Class of POMDPs
We then analyze the performance of the learning algorithm and characterize the regret.
Collaborative Learning of Stochastic Bandits over a Social Network
A key finding of this paper is that natural extensions of widely-studied single agent learning policies to the network setting need not perform well in terms of regret.
Thompson Sampling for Learning Parameterized Markov Decision Processes
We consider reinforcement learning in parameterized Markov Decision Processes (MDPs), where the parameterization may induce correlation across transition probabilities or rewards.
Thompson Sampling for Complex Bandit Problems
We consider stochastic multi-armed bandit problems with complex actions over a set of basic arms, where the decision maker plays a complex action rather than a basic arm in each round.
Thompson Sampling for Online Learning with Linear Experts
In this note, we present a version of the Thompson sampling algorithm for the problem of online linear generalization with full information (i. e., the experts setting), studied by Kalai and Vempala, 2005.
