Search Results for author:
Found 55 papers, 9 papers with code
Model-free Low-Rank Reinforcement Learning via Leveraged Entry-wise Matrix Estimation
The algorithm exploits these new samples to complete the matrix estimation using a CUR-like method.
Conformal Predictions under Markovian Data
This gap strongly depends on the mixing properties of the underlying Markov chain, and we prove that it typically scales as $\sqrt{t_\mathrm{mix}\ln(n)/n}$ (where $t_\mathrm{mix}$ is the mixing time of the chain).
Model-Free Active Exploration in Reinforcement Learning
We study the problem of exploration in Reinforcement Learning and present a novel model-free solution.
Low-Rank Bandits via Tight Two-to-Infinity Singular Subspace Recovery
For policy evaluation and best policy identification, we show that our algorithms are nearly minimax optimal.
Best Arm Identification with Fixed Budget: A Large Deviation Perspective
In particular, we present \sred (Continuous Rejects), a truly adaptive algorithm that can reject arms in {\it any} round based on the observed empirical gaps between the rewards of various arms.
Sub-linear Regret in Adaptive Model Predictive Control
Our result relies on a recently proposed exponential decay of sensitivity property and, to the best of our knowledge, is the first of its kind in this setting.
On Universally Optimal Algorithms for A/B Testing
For the problem with two arms, also known as the A/B testing problem, we prove that there is no algorithm that (i) performs as well as the algorithm sampling each arm equally (referred to as the {\it uniform sampling} algorithm) in all instances, and that (ii) strictly outperforms uniform sampling on at least one instance.
Revisiting Instance-Optimal Cluster Recovery in the Labeled Stochastic Block Model
In this paper, we investigate the problem of recovering hidden communities in the Labeled Stochastic Block Model (LSBM) with a finite number of clusters whose sizes grow linearly with the total number of nodes.
Conformal Off-Policy Evaluation in Markov Decision Processes
Reinforcement Learning aims at identifying and evaluating efficient control policies from data.
On the Sample Complexity of Representation Learning in Multi-task Bandits with Global and Local structure
Arms consist of two components: one that is shared across tasks (that we call representation) and one that is task-specific (that we call predictor).
Self-Tuning Tube-based Model Predictive Control
We present Self-Tuning Tube-based Model Predictive Control (STT-MPC), an adaptive robust control algorithm for uncertain linear systems with additive disturbances based on the least-squares estimator and polytopic tubes.
Tube-Based Zonotopic Data-Driven Predictive Control
We present a novel tube-based data-driven predictive control method for linear systems affected by a bounded addictive disturbance.
Best Policy Identification in Linear MDPs
We investigate the problem of best policy identification in discounted linear Markov Decision Processes in the fixed confidence setting under a generative model.
Measurement-based Admission Control in Sliced Networks: A Best Arm Identification Approach
The objective is to devise a joint measurement and decision strategy that returns a correct decision (e. g., the least loaded slice) with a certain level of confidence while minimizing the measurement cost (the number of measurements made before committing to the decision).
Learning Optimal Antenna Tilt Control Policies: A Contextual Linear Bandit Approach
In this paper, we devise algorithms learning optimal tilt control policies from existing data (in the so-called passive learning setting) or from data actively generated by the algorithms (the active learning setting).
Fast Pure Exploration via Frank-Wolfe
For this problem, instance-specific lower bounds on the expected sample complexity reveal the optimal proportions of arm draws an Oracle algorithm would apply.
Distributed Zeroth-Order Optimization: Convergence Rates That Match Centralized Counterpart
Zeroth-order optimization has become increasingly important in complex optimization and machine learning when cost functions are impossible to be described in closed analytical forms.
Minimal Expected Regret in Linear Quadratic Control
Quantifying the impact of such a constantly-varying control policy on the performance of these estimates and on the regret constitutes one of the technical challenges tackled in this paper.
Balancing detectability and performance of attacks on the control channel of Markov Decision Processes
In such an attack, drawing inspiration from adversarial examples used in supervised learning, the amplitude of the adversarial perturbation is limited according to some norm, with the hope that this constraint will make the attack imperceptible.
Online Learning of Optimally Diverse Rankings
In absence of any contextual information about the query, one often has to adhere to the {\it diversity} principle, i. e., to return a list covering the various possible topics or meanings of the query.
Regret Analysis in Deterministic Reinforcement Learning
We consider Markov Decision Processes (MDPs) with deterministic transitions and study the problem of regret minimization, which is central to the analysis and design of optimal learning algorithms.
Navigating to the Best Policy in Markov Decision Processes
We investigate the classical active pure exploration problem in Markov Decision Processes, where the agent sequentially selects actions and, from the resulting system trajectory, aims at identifying the best policy as fast as possible.
Data-Driven Control and Data-Poisoning attacks in Buildings: the KTH Live-In Lab case study
This work investigates the feasibility of using input-output data-driven control techniques for building control and their susceptibility to data-poisoning techniques.
Poisoning Attacks against Data-Driven Control Methods
This paper investigates poisoning attacks against data-driven control methods.
Minimizing Information Leakage of Abrupt Changes in Stochastic Systems
In contrast to previous work on privacy, we study the problem for an online sequence of data.
Adaptive Sampling for Best Policy Identification in Markov Decision Processes
We then provide a simple and tight upper bound of the sample complexity lower bound, whose corresponding nearly-optimal sample allocation becomes explicit.
Optimal Best-arm Identification in Linear Bandits
We study the problem of best-arm identification with fixed confidence in stochastic linear bandits.
Off-policy Learning for Remote Electrical Tilt Optimization
In this paper, we circumvent these issues by learning an improved policy in an offline manner using existing data collected on real networks.
Predictive Bandits
In each round, the decision maker first decides whether to gather information about the rewards of particular arms (so that their rewards in this round can be predicted).
Finite-time Identification of Stable Linear Systems: Optimality of the Least-Squares Estimator
We present a new finite-time analysis of the estimation error of the Ordinary Least Squares (OLS) estimator for stable linear time-invariant systems.
Distributed Online Optimization with Long-Term Constraints
When the loss functions are strongly convex, we establish improved regret and constraint violation upper bounds in $\mathcal{O}(\log(T))$ and $\mathcal{O}(\sqrt{T\log(T)})$.
Optimal Sampling and Clustering in the Stochastic Block Model
We derive information-theoretical upper bounds on the cluster recovery rate.
Optimal Clustering from Noisy Binary Feedback
The objective is to devise an algorithm with a minimal cluster recovery error rate.
An Optimal Algorithm for Multiplayer Multi-Armed Bandits
We present DPE (Decentralized Parsimonious Exploration), a decentralized algorithm that achieves the same regret as that obtained by an optimal centralized algorithm.
Optimal Attacks on Reinforcement Learning Policies
Finally, we show that from the main agent perspective, the system uncertainties and the attacker can be modeled as a Partially Observable Markov Decision Process.
From self-tuning regulators to reinforcement learning and back again
Machine and reinforcement learning (RL) are increasingly being applied to plan and control the behavior of autonomous systems interacting with the physical world.
Sample Complexity Lower Bounds for Linear System Identification
For controlled systems, our lower bounds are not as explicit as in the case of uncontrolled systems, but could well provide interesting insights into the design of control policy with minimal sample complexity.
Distributed Online Linear Regression
We propose simple and natural distributed regression algorithms, involving, at each node and in each round, a local gradient descent step and a communication and averaging step where nodes aim at aligning their predictors to those of their neighbors.
Learning to Personalize in Appearance-Based Gaze Tracking
We contribute to gaze tracking research by empirically showing that personal variations are well-modeled as a 3-dimensional latent parameter space for each eye.
Exploration in Structured Reinforcement Learning
For Lipschitz MDPs, the bounds are shown not to scale with the sizes $S$ and $A$ of the state and action spaces, i. e., they are smaller than $c\log T$ where $T$ is the time horizon and the constant $c$ only depends on the Lipschitz structure, the span of the bias function, and the minimal action sub-optimality gap.
Minimal Exploration in Structured Stochastic Bandits
This paper introduces and addresses a wide class of stochastic bandit problems where the function mapping the arm to the corresponding reward exhibits some known structural properties.
Fast and Memory Optimal Low-Rank Matrix Approximation
This means that its average mean-square error converges to 0 as $m$ and $n$ grow large (i. e., $\|\hat{M}^{(k)}-M^{(k)} \|_F^2 = o(mn)$ with high probability, where $\hat{M}^{(k)}$ and $M^{(k)}$ denote the output of SLA and the optimal rank $k$ approximation of $M$, respectively).
Optimal Cluster Recovery in the Labeled Stochastic Block Model
We find the set of parameters such that there exists a clustering algorithm with at most $s$ misclassified items in average under the general LSBM and for any $s=o(n)$, which solves one open problem raised in \cite{abbe2015community}.
Cluster-Aided Mobility Predictions
In this paper, we develop cluster-aided predictors that exploit past trajectories collected from all users to predict the next location of a given user.
Streaming, Memory Limited Matrix Completion with Noise
We propose a streaming algorithm which produces an estimate of the original matrix with a vanishing mean square error, uses memory space scaling linearly with the ambient dimension of the matrix, i. e. the memory required to store the output alone, and spends computations as much as the number of non-zero entries of the input matrix.
Combinatorial Bandits Revisited
In the adversarial setting under bandit feedback, we propose \textsc{CombEXP}, an algorithm with the same regret scaling as state-of-the-art algorithms, but with lower computational complexity for some combinatorial problems.
Accurate Community Detection in the Stochastic Block Model via Spectral Algorithms
We consider the problem of community detection in the Stochastic Block Model with a finite number $K$ of communities of sizes linearly growing with the network size $n$.
Social and Information Networks Data Structures and Algorithms
Streaming, Memory Limited Algorithms for Community Detection
The first algorithm is {\it offline}, as it needs to store and keep the assignments of nodes to clusters, and requires a memory that scales linearly with the network size.
Unimodal Bandits without Smoothness
To our knowledge, the SP algorithm constitutes the first sequential arm selection rule that achieves a regret and optimization error scaling as $O(\sqrt{T})$ and $O(1/\sqrt{T})$, respectively, up to a logarithmic factor for non-smooth expected reward functions, as well as for smooth functions with unknown smoothness.
Unimodal Bandits: Regret Lower Bounds and Optimal Algorithms
We also provide a regret upper bound for OSUB in non-stationary environments where the expected rewards smoothly evolve over time.
Lipschitz Bandits: Regret Lower Bounds and Optimal Algorithms
For discrete Lipschitz bandits, we derive asymptotic problem specific lower bounds for the regret satisfied by any algorithm, and propose OSLB and CKL-UCB, two algorithms that efficiently exploit the Lipschitz structure of the problem.
Dynamic Rate and Channel Selection in Cognitive Radio Systems
In turn, the proposed algorithms optimally exploit the inherent structure of the throughput.
Two-Target Algorithms for Infinite-Armed Bandits with Bernoulli Rewards
This two-target algorithm achieves a long-term average regret in $\sqrt{2n}$ for a large parameter $m$ and a known time horizon $n$.
Stochastic Online Shortest Path Routing: The Value of Feedback
The parameters, and hence the optimal path, can only be estimated by routing packets through the network and observing the realized delays.
Spectrum Bandit Optimization
We consider the problem of allocating radio channels to links in a wireless network.
