Search Results for author:
Found 98 papers, 4 papers with code
Rate-Optimal Policy Optimization for Linear Markov Decision Processes
We study regret minimization in online episodic linear Markov Decision Processes, and obtain rate-optimal $\widetilde O (\sqrt K)$ regret where $K$ denotes the number of episodes.
The tree reconstruction game: phylogenetic reconstruction using reinforcement learning
Our proposed method does not require likelihood calculation with every step, nor is it limited to greedy uphill moves in the likelihood space.
Efficient Rate Optimal Regret for Adversarial Contextual MDPs Using Online Function Approximation
To the best of our knowledge, our algorithm is the first efficient rate optimal regret minimization algorithm for adversarial CMDPs that operates under the minimal standard assumption of online function approximation.
On Differentially Private Online Predictions
In this work we introduce an interactive variant of joint differential privacy towards handling online processes in which existing privacy definitions seem too restrictive.
Pseudonorm Approachability and Applications to Regret Minimization
We then use that to show, modulo mild normalization assumptions, that there exists an $\ell_\infty$-approachability algorithm whose convergence is independent of the dimension of the original vectorial payoff.
Uniswap Liquidity Provision: An Online Learning Approach
Decentralized Exchanges (DEXs) are new types of marketplaces leveraging Blockchain technology.
Improved Regret for Efficient Online Reinforcement Learning with Linear Function Approximation
We study reinforcement learning with linear function approximation and adversarially changing cost functions, a setup that has mostly been considered under simplifying assumptions such as full information feedback or exploratory conditions. We present a computationally efficient policy optimization algorithm for the challenging general setting of unknown dynamics and bandit feedback, featuring a combination of mirror-descent and least squares policy evaluation in an auxiliary MDP used to compute exploration bonuses. Our algorithm obtains an $\widetilde O(K^{6/7})$ regret bound, improving significantly over previous state-of-the-art of $\widetilde O (K^{14/15})$ in this setting.
Concurrent Shuffle Differential Privacy Under Continual Observation
the counter problem) and show that the concurrent shuffle model allows for significantly improved error compared to a standard (single) shuffle model.
Differentially-Private Bayes Consistency
We construct a universally Bayes consistent learning rule that satisfies differential privacy (DP).
Eluder-based Regret for Stochastic Contextual MDPs
To the best of our knowledge, our algorithm is the first efficient and rate-optimal regret minimization algorithm for CMDPs that operates under the general offline function approximation setting.
Dueling Convex Optimization with General Preferences
We address the problem of \emph{convex optimization with dueling feedback}, where the goal is to minimize a convex function given a weaker form of \emph{dueling} feedback.
Regret Minimization and Convergence to Equilibria in General-sum Markov Games
Our key observation is that online learning via policy optimization in Markov games essentially reduces to a form of weighted regret minimization, with unknown weights determined by the path length of the agents' policy sequence.
Optimism in Face of a Context: Regret Guarantees for Stochastic Contextual MDP
For the latter, our algorithm obtains regret bound of $\widetilde{O}( (H+{1}/{p_{min}})H|S|^{3/2}\sqrt{|A|T\log(\max\{|\mathcal{G}|,|\mathcal{P}|\}/\delta)})$ with probability $1-\delta$, where $\mathcal{P}$ and $\mathcal{G}$ are finite and realizable function classes used to approximate the dynamics and rewards respectively, $p_{min}$ is the minimum reachability parameter, $S$ is the set of states, $A$ the set of actions, $H$ the horizon, and $T$ the number of episodes.
Guarantees for Epsilon-Greedy Reinforcement Learning with Function Approximation
This paper presents a theoretical analysis of such policies and provides the first regret and sample-complexity bounds for reinforcement learning with myopic exploration.
There is no Accuracy-Interpretability Tradeoff in Reinforcement Learning for Mazes
Interpretability is an essential building block for trustworthiness in reinforcement learning systems.
What killed the Convex Booster ?
A landmark negative result of Long and Servedio established a worst-case spectacular failure of a supervised learning trio (loss, algorithm, model) otherwise praised for its high precision machinery.
Strategizing against Learners in Bayesian Games
We study repeated two-player games where one of the players, the learner, employs a no-regret learning strategy, while the other, the optimizer, is a rational utility maximizer.
Modeling Attrition in Recommender Systems with Departing Bandits
We first address the case where all users share the same type, demonstrating that a recent UCB-based algorithm is optimal.
Learning Efficiently Function Approximation for Contextual MDP
We study learning contextual MDPs using a function approximation for both the rewards and the dynamics.
Benign Underfitting of Stochastic Gradient Descent
We study to what extent may stochastic gradient descent (SGD) be understood as a "conventional" learning rule that achieves generalization performance by obtaining a good fit to training data.
Finding Safe Zones of policies Markov Decision Processes
Given a policy of a Markov Decision Process, we define a SafeZone as a subset of states, such that most of the policy's trajectories are confined to this subset.
Learning Revenue Maximization using Posted Prices for Stochastic Strategic Patient Buyers
Following this, we show how to compute both the optimal pure and mixed strategies.
A Characterization of Semi-Supervised Adversarially-Robust PAC Learnability
This shows that there is a significant benefit in semi-supervised robust learning even in the worst-case distribution-free model, and establishes a gap between the supervised and semi-supervised label complexities which is known not to hold in standard non-robust PAC learning.
Monotone Learning
Our transformation readily implies monotone learners in a variety of contexts: for example it extends Pestov's result to classification tasks with an arbitrary number of labels.
Cooperative Online Learning in Stochastic and Adversarial MDPs
We study cooperative online learning in stochastic and adversarial Markov decision process (MDP).
Fair Wrapping for Black-box Predictions
We introduce a new family of techniques to post-process ("wrap") a black-box classifier in order to reduce its bias.
Near-Optimal Regret for Adversarial MDP with Delayed Bandit Feedback
The standard assumption in reinforcement learning (RL) is that agents observe feedback for their actions immediately.
Differentially-Private Clustering of Easy Instances
Clustering is a fundamental problem in data analysis.
Nonstochastic Bandits with Composite Anonymous Feedback
We investigate a nonstochastic bandit setting in which the loss of an action is not immediately charged to the player, but rather spread over the subsequent rounds in an adversarial way.
FriendlyCore: Practical Differentially Private Aggregation
Differentially private algorithms for common metric aggregation tasks, such as clustering or averaging, often have limited practicality due to their complexity or to the large number of data points that is required for accurate results.
Dueling Bandits with Team Comparisons
We introduce the dueling teams problem, a new online-learning setting in which the learner observes noisy comparisons of disjoint pairs of $k$-sized teams from a universe of $n$ players.
Optimal Rates for Random Order Online Optimization
We study online convex optimization in the random order model, recently proposed by \citet{garber2020online}, where the loss functions may be chosen by an adversary, but are then presented to the online algorithm in a uniformly random order.
Agnostic Reinforcement Learning with Low-Rank MDPs and Rich Observations
In this work, we consider the more realistic setting of agnostic RL with rich observation spaces and a fixed class of policies $\Pi$ that may not contain any near-optimal policy.
Differentially Private Multi-Armed Bandits in the Shuffle Model
We give an $(\varepsilon,\delta)$-differentially private algorithm for the multi-armed bandit (MAB) problem in the shuffle model with a distribution-dependent regret of $O\left(\left(\sum_{a\in [k]:\Delta_a>0}\frac{\log T}{\Delta_a}\right)+\frac{k\sqrt{\log\frac{1}{\delta}}\log T}{\varepsilon}\right)$, and a distribution-independent regret of $O\left(\sqrt{kT\log T}+\frac{k\sqrt{\log\frac{1}{\delta}}\log T}{\varepsilon}\right)$, where $T$ is the number of rounds, $\Delta_a$ is the suboptimality gap of the arm $a$, and $k$ is the total number of arms.
Stochastic Multi-Armed Bandits with Unrestricted Delay Distributions
We study the stochastic Multi-Armed Bandit (MAB) problem with random delays in the feedback received by the algorithm.
Minimax Regret for Stochastic Shortest Path
In this work we show that the minimax regret for this setting is $\widetilde O(\sqrt{ (B_\star^2 + B_\star) |S| |A| K})$ where $B_\star$ is a bound on the expected cost of the optimal policy from any state, $S$ is the state space, and $A$ is the action space.
Online Markov Decision Processes with Aggregate Bandit Feedback
We study a novel variant of online finite-horizon Markov Decision Processes with adversarially changing loss functions and initially unknown dynamics.
Separating Adaptive Streaming from Oblivious Streaming
We present a streaming problem for which every adversarially-robust streaming algorithm must use polynomial space, while there exists a classical (oblivious) streaming algorithm that uses only polylogarithmic space.
Data Structures and Algorithms
Learning Adversarial Markov Decision Processes with Delayed Feedback
We present novel algorithms based on policy optimization that achieve near-optimal high-probability regret of $\widetilde O ( \sqrt{K} + \sqrt{D} )$ under full-information feedback, where $K$ is the number of episodes and $D = \sum_{k} d^k$ is the total delay.
Adversarial Dueling Bandits
We introduce the problem of regret minimization in Adversarial Dueling Bandits.
The Sparse Vector Technique, Revisited
This simple algorithm privately tests whether the value of a given query on a database is close to what we expect it to be.
Oracle-Efficient Regret Minimization in Factored MDPs with Unknown Structure
We study regret minimization in non-episodic factored Markov decision processes (FMDPs), where all existing algorithms make the strong assumption that the factored structure of the FMDP is known to the learner in advance.
Beyond Individual and Group Fairness
In the adversarial setting, we design efficient algorithms with competitive ratio guarantees.
Detecting malicious PDF using CNN
We show, using a data set of 90000 files downloadable online, that our approach maintains a high detection rate (94%) of PDF malware and even detects new malicious files, still undetected by most antiviruses.
Competing Bandits: The Perils of Exploration Under Competition
Users arrive one by one and choose between the two firms, so that each firm makes progress on its bandit problem only if it is chosen.
A Theory of Multiple-Source Adaptation with Limited Target Labeled Data
We present a theoretical and algorithmic study of the multiple-source domain adaptation problem in the common scenario where the learner has access only to a limited amount of labeled target data, but where the learner has at disposal a large amount of labeled data from multiple source domains.
Stochastic Shortest Path with Adversarially Changing Costs
Stochastic shortest path (SSP) is a well-known problem in planning and control, in which an agent has to reach a goal state in minimum total expected cost.
Reinforcement Learning with Feedback Graphs
We study episodic reinforcement learning in Markov decision processes when the agent receives additional feedback per step in the form of several transition observations.
Sample Complexity of Uniform Convergence for Multicalibration
There is a growing interest in societal concerns in machine learning systems, especially in fairness.
Private Learning of Halfspaces: Simplifying the Construction and Reducing the Sample Complexity
We present a differentially private learner for halfspaces over a finite grid $G$ in $\mathbb{R}^d$ with sample complexity $\approx d^{2. 5}\cdot 2^{\log^*|G|}$, which improves the state-of-the-art result of [Beimel et al., COLT 2019] by a $d^2$ factor.
Adversarially Robust Streaming Algorithms via Differential Privacy
A streaming algorithm is said to be adversarially robust if its accuracy guarantees are maintained even when the data stream is chosen maliciously, by an adaptive adversary.
Three Approaches for Personalization with Applications to Federated Learning
The standard objective in machine learning is to train a single model for all users.
Prediction with Corrupted Expert Advice
We revisit the fundamental problem of prediction with expert advice, in a setting where the environment is benign and generates losses stochastically, but the feedback observed by the learner is subject to a moderate adversarial corruption.
Near-optimal Regret Bounds for Stochastic Shortest Path
In this work we remove this dependence on the minimum cost---we give an algorithm that guarantees a regret bound of $\widetilde{O}(B_\star |S| \sqrt{|A| K})$, where $B_\star$ is an upper bound on the expected cost of the optimal policy, $S$ is the set of states, $A$ is the set of actions and $K$ is the number of episodes.
Online Stochastic Shortest Path with Bandit Feedback and Unknown Transition Function
We consider online learning in episodic loop-free Markov decision processes (MDPs), where the loss function can change arbitrarily between episodes.
Privately Learning Thresholds: Closing the Exponential Gap
This problem has received much attention recently; unlike the non-private case, where the sample complexity is independent of the domain size and just depends on the desired accuracy and confidence, for private learning the sample complexity must depend on the domain size $X$ (even for approximate differential privacy).
Apprenticeship Learning via Frank-Wolfe
Specifically, we show that a variation of the FW method that is based on taking "away steps" achieves a linear rate of convergence when applied to AL and that a stochastic version of the FW algorithm can be used to avoid precise estimation of feature expectations.
Individual Regret in Cooperative Nonstochastic Multi-Armed Bandits
We study agents communicating over an underlying network by exchanging messages, in order to optimize their individual regret in a common nonstochastic multi-armed bandit problem.
Thompson Sampling for Adversarial Bit Prediction
We also bound the regret of those sequences, the worse case sequences have regret $O(\sqrt{T})$ and the best case sequence have regret $O(1)$.
Graph-based Discriminators: Sample Complexity and Expressiveness
A function $g\in \mathcal{G}$ distinguishes between two distributions, if the expected value of $g$, on a $k$-tuple of i. i. d examples, on the two distributions is (significantly) different.
ROI Maximization in Stochastic Online Decision-Making
We introduce a novel theoretical framework for Return On Investment (ROI) maximization in repeated decision-making.
Top-k Combinatorial Bandits with Full-Bandit Feedback
Top-k Combinatorial Bandits generalize multi-armed bandits, where at each round any subset of $k$ out of $n$ arms may be chosen and the sum of the rewards is gained.
Efficient candidate screening under multiple tests and implications for fairness
We analyze the optimal employer policy both when the employer sets a fixed number of tests per candidate and when the employer can set a dynamic policy, assigning further tests adaptively based on results from the previous tests.
Unknown mixing times in apprenticeship and reinforcement learning
We derive and analyze learning algorithms for apprenticeship learning, policy evaluation, and policy gradient for average reward criteria.
Online Convex Optimization in Adversarial Markov Decision Processes
We consider online learning in episodic loop-free Markov decision processes (MDPs), where the loss function can change arbitrarily between episodes, and the transition function is not known to the learner.
Competitive ratio versus regret minimization: achieving the best of both worlds
Our end result is an online algorithm that can combine a "base" online algorithm, having a guaranteed competitive ratio, with a range of online algorithms that guarantee a small regret over any interval of time.
Planning in Hierarchical Reinforcement Learning: Guarantees for Using Local Policies
We consider a settings of hierarchical reinforcement learning, in which the reward is a sum of components.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
Learning Linear-Quadratic Regulators Efficiently with only $\sqrt{T}$ Regret
We present the first computationally-efficient algorithm with $\widetilde O(\sqrt{T})$ regret for learning in Linear Quadratic Control systems with unknown dynamics.
Learning to Screen
(ii) In the second variant it is assumed that before the process starts, the algorithm has an access to a training set of $n$ items drawn independently from the same unknown distribution (e. g.\ data of candidates from previous recruitment seasons).
Differentially Private Learning of Geometric Concepts
We present differentially private efficient algorithms for learning union of polygons in the plane (which are not necessarily convex).
Adversarial Online Learning with noise
We present and study models of adversarial online learning where the feedback observed by the learner is noisy, and the feedback is either full information feedback or bandit feedback.
Improved Generalization Bounds for Adversarially Robust Learning
For binary classification, the algorithm of Feige et al. (2015) uses a regret minimization algorithm and an ERM oracle as a black box; we adapt it for the multiclass and regression settings.
Online Linear Quadratic Control
We study the problem of controlling linear time-invariant systems with known noisy dynamics and adversarially chosen quadratic losses.
Are All Experts Equally Good? A Study of Analyst Earnings Estimates
We note that this would make it possible to aggregate multiple predictions into a result that is more accurate than their consensus average, and that the improvement prospects grow with the amount of differentiation.
Planning and Learning with Stochastic Action Sets
From an RL perspective, we show that Q-learning with sampled action sets is sound.
Hierarchical Reinforcement Learning: Approximating Optimal Discounted TSP Using Local Policies
In this work, we provide theoretical guarantees for reward decomposition in deterministic MDPs.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
Multi-Armed Bandits with Metric Movement Costs
We consider the non-stochastic Multi-Armed Bandit problem in a setting where there is a fixed and known metric on the action space that determines a cost for switching between any pair of actions.
Discriminative Learning of Prediction Intervals
Most current methods for constructing prediction intervals offer guarantees for a single new test point.
Submultiplicative Glivenko-Cantelli and Uniform Convergence of Revenues
In this work we derive a variant of the classic Glivenko-Cantelli Theorem, which asserts uniform convergence of the empirical Cumulative Distribution Function (CDF) to the CDF of the underlying distribution.
Efficient PAC Learning from the Crowd
When a noticeable fraction of the labelers are perfect, and the rest behave arbitrarily, we show that any $\mathcal{F}$ that can be efficiently learned in the traditional realizable PAC model can be learned in a computationally efficient manner by querying the crowd, despite high amounts of noise in the responses.
Competing Bandits: Learning under Competition
Most modern systems strive to learn from interactions with users, and many engage in exploration: making potentially suboptimal choices for the sake of acquiring new information.
Bandits with Movement Costs and Adaptive Pricing
In this setting, we give a new algorithm that establishes a regret of $\widetilde{O}(\sqrt{kT} + T/k)$, where $k$ is the number of actions and $T$ is the time horizon.
Automatic Representation for Lifetime Value Recommender Systems
The LTV approach considers the future implications of the item recommendation, and seeks to maximize the cumulative gain over time.
Online Pricing with Strategic and Patient Buyers
We consider a seller with an unlimited supply of a single good, who is faced with a stream of $T$ buyers.
Predicting Counterfactuals from Large Historical Data and Small Randomized Trials
The conventional way to answer this counterfactual question is to estimate the effect of the new treatment in comparison to that of the conventional treatment by running a controlled, randomized experiment.
Online Learning with Low Rank Experts
We consider the problem of prediction with expert advice when the losses of the experts have low-dimensional structure: they are restricted to an unknown $d$-dimensional subspace.
Bayesian Exploration: Incentivizing Exploration in Bayesian Games
As a key technical tool, we introduce the concept of explorable actions, the actions which some incentive-compatible policy can recommend with non-zero probability.
Delay and Cooperation in Nonstochastic Bandits
We introduce \textsc{Exp3-Coop}, a cooperative version of the {\sc Exp3} algorithm and prove that with $K$ actions and $N$ agents the average per-agent regret after $T$ rounds is at most of order $\sqrt{\bigl(d+1 + \tfrac{K}{N}\alpha_{\le d}\bigr)(T\ln K)}$, where $\alpha_{\le d}$ is the independence number of the $d$-th power of the connected communication graph $G$.
Label Efficient Learning by Exploiting Multi-class Output Codes
We present a new perspective on the popular multi-class algorithmic techniques of one-vs-all and error correcting output codes.
Classification with Low Rank and Missing Data
We consider classification and regression tasks where we have missing data and assume that the (clean) data resides in a low rank subspace.
On the Complexity of Learning with Kernels
In this paper, we study lower bounds on the error attainable by such methods as a function of the number of entries observed in the kernel matrix or the rank of an approximate kernel matrix.
Nonstochastic Multi-Armed Bandits with Graph-Structured Feedback
This naturally models several situations where the losses of different actions are related, and knowing the loss of one action provides information on the loss of other actions.
Thompson Sampling for Complex Bandit Problems
We consider stochastic multi-armed bandit problems with complex actions over a set of basic arms, where the decision maker plays a complex action rather than a basic arm in each round.
From Bandits to Experts: A Tale of Domination and Independence
We consider the partial observability model for multi-armed bandits, introduced by Mannor and Shamir.
Learning Multiple Tasks using Shared Hypotheses
In this work we consider a setting where we have a very large number of related tasks with few examples from each individual task.
Learning Bounds for Importance Weighting
This paper presents an analysis of importance weighting for learning from finite samples and gives a series of theoretical and algorithmic results.
Domain Adaptation: Learning Bounds and Algorithms
This motivates our analysis of the problem of minimizing the empirical discrepancy for various loss functions for which we also give novel algorithms.
Domain Adaptation with Multiple Sources
The problem consists of combining these hypotheses to derive a hypothesis with small error with respect to the target domain.
