Search Results for author:
Found 208 papers, 34 papers with code
RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compression
Video encoders optimize compression for human perception by minimizing reconstruction error under bit-rate constraints.
Learning Multiple Initial Solutions to Optimization Problems
Sequentially solving similar optimization problems under strict runtime constraints is essential for many applications, such as robot control, autonomous driving, and portfolio management.
Improving Inverse Folding for Peptide Design with Diversity-regularized Direct Preference Optimization
However, when applied to peptides, these models are prone to generating repetitive sequences that do not fold into the reference structure.
Improved Sample Complexity for Global Convergence of Actor-Critic Algorithms
In this paper, we establish the global convergence of the actor-critic algorithm with a significantly improved sample complexity of $O(\epsilon^{-3})$, advancing beyond the existing local convergence results.
Dual Pricing to Prioritize Renewable Energy and Consumer Preferences in Electricity Markets
Electricity markets currently fail to incorporate preferences of buyers, treating polluting and renewable energy sources as having equal social benefit under a system of uniform clearing prices.
Efficient Fairness-Performance Pareto Front Computation
Due to the complexity of optimisation algorithms in most modern representation learning approaches, for a given method it may be non-trivial to decide whether the obtained fairness-performance curve of the method is optimal, i. e., whether it is close to the true Pareto front for these quantities for the underlying data distribution.
From Glucose Patterns to Health Outcomes: A Generalizable Foundation Model for Continuous Glucose Monitor Data Analysis
Here, we present GluFormer, a generative foundation model for CGM data that learns nuanced glycemic patterns and translates them into predictive representations of metabolic health.
PlaMo: Plan and Move in Rich 3D Physical Environments
Controlling humanoids in complex physically simulated worlds is a long-standing challenge with numerous applications in gaming, simulation, and visual content creation.
RL in Latent MDPs is Tractable: Online Guarantees via Off-Policy Evaluation
Our result builds off a new perspective on the role of off-policy evaluation guarantees and coverage coefficients in LMDPs, a perspective, that has been overlooked in the context of exploration in partially observed environments.
On Bits and Bandits: Quantifying the Regret-Information Trade-off
We study the trade-off between the information an agent accumulates and the regret it suffers.
Tree Search-Based Policy Optimization under Stochastic Execution Delay
We show that given observed delay values, it is sufficient to perform a policy search in the class of Markov policies in order to reach optimal performance, thus extending the deterministic fixed delay case.
On the Global Convergence of Policy Gradient in Average Reward Markov Decision Processes
We present the first finite time global convergence analysis of policy gradient in the context of infinite horizon average reward Markov decision processes (MDPs).
Conservative DDPG -- Pessimistic RL without Ensemble
DDPG is hindered by the overestimation bias problem, wherein its $Q$-estimates tend to overstate the actual $Q$-values.
Exploration-Driven Policy Optimization in RLHF: Theoretical Insights on Efficient Data Utilization
The algorithm is based on the popular Policy Cover-Policy Gradient (PC-PG) algorithm, which assumes knowledge of the reward function.
Improving Token-Based World Models with Parallel Observation Prediction
We incorporate POP in a novel TBWM agent named REM (Retentive Environment Model), showcasing a 15. 4x faster imagination compared to prior TBWMs.
SQT -- std $Q$-target
We implement SQT on top of TD3/TD7 code and test it against the state-of-the-art (SOTA) actor-critic algorithms, DDPG, TD3 and TD7 on seven popular MuJoCo and Bullet tasks.
MinMaxMin $Q$-learning
MinMaxMin $Q$-learning is a novel optimistic Actor-Critic algorithm that addresses the problem of overestimation bias ($Q$-estimations are overestimating the real $Q$-values) inherent in conservative RL algorithms.
Prospective Side Information for Latent MDPs
In such an environment, the latent information remains fixed throughout each episode, since the identity of the user does not change during an interaction.
Solving Non-Rectangular Reward-Robust MDPs via Frequency Regularization
In robust Markov decision processes (RMDPs), it is assumed that the reward and the transition dynamics lie in a given uncertainty set.
Sobolev Space Regularised Pre Density Models
We propose a new approach to non-parametric density estimation that is based on regularizing a Sobolev norm of the density.
Bring Your Own (Non-Robust) Algorithm to Solve Robust MDPs by Estimating The Worst Kernel
Robust Markov Decision Processes (RMDPs) provide a framework for sequential decision-making that is robust to perturbations on the transition kernel.
Representation-Driven Reinforcement Learning
We present a representation-driven framework for reinforcement learning.
CALM: Conditional Adversarial Latent Models for Directable Virtual Characters
In this work, we present Conditional Adversarial Latent Models (CALM), an approach for generating diverse and directable behaviors for user-controlled interactive virtual characters.
Twice Regularized Markov Decision Processes: The Equivalence between Robustness and Regularization
We then generalize regularized MDPs to twice regularized MDPs ($\text{R}^2$ MDPs), i. e., MDPs with $\textit{both}$ value and policy regularization.
Policy Gradient for Rectangular Robust Markov Decision Processes
We provide a closed-form expression for the worst occupation measure.
An Efficient Solution to s-Rectangular Robust Markov Decision Processes
We present an efficient robust value iteration for \texttt{s}-rectangular robust Markov Decision Processes (MDPs) with a time complexity comparable to standard (non-robust) MDPs which is significantly faster than any existing method.
SoftTreeMax: Exponential Variance Reduction in Policy Gradient via Tree Search
We prove that the resulting variance decays exponentially with the planning horizon as a function of the expansion policy.
Towards Deployable RL - What's Broken with RL Research and a Potential Fix
Reinforcement learning (RL) has demonstrated great potential, but is currently full of overhyping and pipe dreams.
DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles
To enable the joint optimization of AV stacks while retaining modularity, we present DiffStack, a differentiable and modular stack for prediction, planning, and control.
Tractable Optimality in Episodic Latent MABs
Then, through a method-of-moments approach, we design a procedure that provably learns a near-optimal policy with $O(\texttt{poly}(A) + \texttt{poly}(M, H)^{\min(M, H)})$ interactions.
Reward-Mixing MDPs with a Few Latent Contexts are Learnable
We consider episodic reinforcement learning in reward-mixing Markov decision processes (RMMDPs): at the beginning of every episode nature randomly picks a latent reward model among $M$ candidates and an agent interacts with the MDP throughout the episode for $H$ time steps.
Policy Gradient for Reinforcement Learning with General Utilities
The policy gradient theorem proves to be a cornerstone in Linear RL due to its elegance and ease of implementability.
SoftTreeMax: Policy Gradient with Tree Search
This allows us to reduce the variance of gradients by three orders of magnitude and to benefit from better sample complexity compared with standard policy gradient.
Actor-Critic based Improper Reinforcement Learning
For the AC-based approach we provide convergence rate guarantees to a stationary point in the basic AC case and to a global optimum in the NAC case.
Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs
As communication protocols evolve, datacenter network utilization increases.
Analysis of Stochastic Processes through Replay Buffers
We provide an analysis of the properties of the sampled process such as stationarity, Markovity and autocorrelation in terms of the properties of the original process.
Reinforcement Learning with a Terminator
We learn the parameters of the TerMDP and leverage the structure of the estimation problem to provide state-wise confidence bounds.
Efficient Policy Iteration for Robust Markov Decision Processes via Regularization
But we don't have a clear understanding to exploit this equivalence, to do policy improvement steps to get the optimal value function or policy.
Efficient Risk-Averse Reinforcement Learning
In risk-averse reinforcement learning (RL), the goal is to optimize some risk measure of the returns.
Optimizing Tensor Network Contraction Using Reinforcement Learning
Quantum Computing (QC) stands to revolutionize computing, but is currently still limited.
Learning Hidden Markov Models When the Locations of Missing Observations are Unknown
However, standard HMM learning algorithms rely crucially on the assumption that the positions of the missing observations \emph{within the observation sequence} are known.
Learning to reason about and to act on physical cascading events
Reasoning and interacting with dynamic environments is a fundamental problem in AI, but it becomes extremely challenging when actions can trigger cascades of cross-dependent events.
Continuous Forecasting via Neural Eigen Decomposition
Neural differential equations predict the derivative of a stochastic process.
Coordinated Attacks against Contextual Bandits: Fundamental Limits and Defense Mechanisms
This parallelization gain is fundamentally altered by the presence of adversarial users: unless there are super-polynomial number of users, we show a lower bound of $\tilde{\Omega}(\min(S, A) \cdot \alpha^2 / \epsilon^2)$ {\it per-user} interactions to learn an $\epsilon$-optimal policy for the good users.
The Geometry of Robust Value Functions
The key of this perspective is to decompose the value space, in a state-wise manner, into unions of hypersurfaces.
Planning and Learning with Adaptive Lookahead
Some of the most powerful reinforcement learning frameworks use planning for action selection.
On Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning
We analyze the limitations of learning from such data with and without external reward, and propose an adjustment of standard imitation learning algorithms to fit this setup.
Twice regularized MDPs and the equivalence between robustness and regularization
We finally generalize regularized MDPs to twice regularized MDPs (R${}^2$ MDPs), i. e., MDPs with $\textit{both}$ value and policy regularization.
Query-Reward Tradeoffs in Multi-Armed Bandits
We consider a stochastic multi-armed bandit setting where reward must be actively queried for it to be observed.
Reinforcement Learning in Reward-Mixing MDPs
We study the problem of learning a near optimal policy for two reward-mixing MDPs.
Continuous-Time Fitted Value Iteration for Robust Policies
Especially for continuous control, solving this differential equation and its extension the Hamilton-Jacobi-Isaacs equation, is important as it yields the optimal policy that achieves the maximum reward on a give task.
Sim and Real: Better Together
Simulation is used extensively in autonomous systems, particularly in robotic manipulation.
Kalman Filter Is All You Need: Optimization Works When Noise Estimation Fails
Determining the noise parameters of a Kalman Filter (KF) has been studied for decades.
Two Regimes of Generalization for Non-Linear Metric Learning
Specifically, we discover and analyze two regimes of behavior of the networks, which are roughly related to the sparsity of the last layer.
Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learning
Cooperative multi-agent reinforcement learning (MARL) faces significant scalability issues due to state and action spaces that are exponentially large in the number of agents.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Improve Agents without Retraining: Parallel Tree Search with Off-Policy Correction
We first discover and analyze a counter-intuitive phenomenon: action selection through TS and a pre-trained value function often leads to lower performance compared to the original pre-trained agent, even when having access to the exact state and reward in future steps.
Robust Value Iteration for Continuous Control Tasks
The adversarial perturbations encourage a optimal policy that is robust to changes in the dynamics.
Value Iteration in Continuous Actions, States and Time
This algorithm enables dynamic programming for continuous states and actions with a known dynamics model.
Better than the Best: Gradient-based Improper Reinforcement Learning for Network Scheduling
We consider the problem of scheduling in constrained queueing networks with a view to minimizing packet delay.
The Fragility of Noise Estimation in Kalman Filter: Optimization Can Handle Model-Misspecification
The Kalman Filter (KF) parameters are traditionally determined by noise estimation, since under the KF assumptions, the state prediction errors are minimized when the parameters correspond to the noise covariance.
Maximum Entropy Reinforcement Learning with Mixture Policies
However, using mixture policies in the Maximum Entropy (MaxEnt) framework is not straightforward.
Uncertainty Estimation Using Riemannian Model~Dynamics for Offline Reinforcement Learning
In this work, we combine parametric and nonparametric methods for uncertainty estimation through a novel latent space based metric.
Action Redundancy in Reinforcement Learning
Maximum Entropy (MaxEnt) reinforcement learning is a powerful learning paradigm which seeks to maximize return under entropy regularization.
Reinforcement Learning for Datacenter Congestion Control
We approach the task of network congestion control in datacenters using Reinforcement Learning (RL).
Improper Reinforcement Learning with Gradient-based Policy Optimization
We consider an improper reinforcement learning setting where a learner is given $M$ base controllers for an unknown Markov decision process, and wishes to combine them optimally to produce a potentially new controller that can outperform each of the base ones.
Online Apprenticeship Learning
Finally, we implement a deep variant of our algorithm which shares some similarities to GAIL \cite{ho2016generative}, but where the discriminator is replaced with the costs learned by the OAL problem.
RL for Latent MDPs: Regret Guarantees and a Lower Bound
In this work, we consider the regret minimization problem for reinforcement learning in latent Markov Decision Processes (LMDP).
Dimension Free Generalization Bounds for Non Linear Metric Learning
In this work we study generalization guarantees for the metric learning problem, where the metric is induced by a neural network type embedding of the data.
Online Limited Memory Neural-Linear Bandits with Likelihood Matching
To alleviate this, we propose a likelihood matching algorithm that is resilient to catastrophic forgetting and is completely online.
Confidence-Budget Matching for Sequential Budgeted Learning
We analyze the performance of CBM based algorithms in different settings and show that they perform well in the presence of adversity in the contexts, initial states, and budgets.
Acting in Delayed Environments with Non-Stationary Markov Policies
We introduce a framework for learning and planning in MDPs where the decision-maker commits actions that are executed with a delay of $m$ steps.
Learning Safe Policies with Cost-sensitive Advantage Estimation
Reinforcement Learning (RL) with safety guarantee is critical for agents performing tasks in risky environments.
Online Limited Memory Neural-Linear Bandits
We study neural-linear bandits for solving problems where both exploration and representation learning play an important role.
The Architectural Implications of Distributed Reinforcement Learning on CPU-GPU Systems
With deep reinforcement learning (RL) methods achieving results that exceed human capabilities in games, robotics, and simulated environments, continued scaling of RL training is crucial to its deployment in solving complex real-world problems.
Detecting Rewards Deterioration in Episodic Reinforcement Learning
In many RL applications, once training ends, it is vital to detect any deterioration in the agent performance as soon as possible.
Controlling Graph Dynamics with Reinforcement Learning and Graph Neural Networks
We consider the problem of controlling a partially-observed dynamic process on a graph by a limited number of interventions.
Drift Detection in Episodic Data: Detect When Your Agent Starts Faltering
The statistical power of the new testing procedure is shown to outperform alternative tests - often by orders of magnitude - for a variety of environment modifications (which cause deterioration in agent performance).
Reinforcement Learning with Trajectory Feedback
The standard feedback model of reinforcement learning requires revealing the reward of every visited state-action pair.
Lenient Regret for Multi-Armed Bandits
Importantly, we show that when the mean of the optimal arm is high enough, the lenient regret of $\epsilon$-TS is bounded by a constant.
Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
For deep neural network accelerators, memory movement is both energetically expensive and can bound computation.
Bandits with Partially Observable Confounded Data
We construct a linear bandit algorithm that takes advantage of the projected information, and prove regret bounds.
Distributional Robustness and Regularization in Reinforcement Learning
Distributionally Robust Optimization (DRO) has enabled to prove the equivalence between robustness and regularization in classification and regression, thus providing an analytical reason why regularization generalizes well in statistical learning.
Exploration-Exploitation in Constrained MDPs
In many sequential decision-making problems, the goal is to optimize a utility function while satisfying a set of constraints on different utilities.
Stealing Black-Box Functionality Using The Deep Neural Tree Architecture
This paper makes a substantial step towards cloning the functionality of black-box models by introducing a Machine learning (ML) architecture named Deep Neural Trees (DNTs).
Optimistic Policy Optimization with Bandit Feedback
To the best of our knowledge, the two results are the first sub-linear regret bounds obtained for policy optimization algorithms with unknown transitions and bandit feedback.
Kalman meets Bellman: Improving Policy Evaluation through Value Tracking
These frameworks can learn uncertainties over the value parameters and exploit them for policy exploration.
Tight Lower Bounds for Combinatorial Multi-Armed Bandits
The Combinatorial Multi-Armed Bandit problem is a sequential decision-making problem in which an agent selects a set of arms on each round, observes feedback for each of these arms and aims to maximize a known reward function of the arms it chose.
Over-the-Air Adversarial Flickering Attacks against Video Recognition Networks
In this work we present a manipulation scheme for fooling video classifiers by introducing a flickering temporal perturbation that in some cases may be unnoticeable by human observers and is implementable in the real world.
Reward Tweaking: Maximizing the Total Reward While Planning for Short Horizons
In reinforcement learning, the discount factor $\gamma$ controls the agent's effective planning horizon.
Never Worse, Mostly Better: Stable Policy Improvement in Deep Reinforcement Learning
In recent years, there has been significant progress in applying deep reinforcement learning (RL) for solving challenging problems across a wide variety of domains.
Language is Power: Representing States Using Natural Language in Reinforcement Learning
Recent advances in reinforcement learning have shown its potential to tackle complex real-life tasks.
Partial Simulation for Imitation Learning
Model-based imitation learning methods require full knowledge of the transition kernel for policy evaluation.
Neural Linear Bandits: Overcoming Catastrophic Forgetting through Likelihood Matching
We study neural-linear bandits for solving problems where both exploration and representation learning play an important role.
Contextual Inverse Reinforcement Learning
In this setting, the reward, which is unknown to the agent, is a function of a static parameter referred to as the context.
Stabilizing Off-Policy Reinforcement Learning with Conservative Policy Gradients
In recent years, advances in deep learning have enabled the application of reinforcement learning algorithms in complex domains.
Online Planning with Lookahead Policies
This is the first work that proves improved sample complexity as a result of {\em increasing} the lookahead horizon in online planning.
Off-Policy Evaluation in Partially Observable Environments
This work studies the problem of batch off-policy evaluation for Reinforcement Learning in partially observable environments.
Adaptive Trust Region Policy Optimization: Global Convergence and Faster Rates for Regularized MDPs
Trust region policy optimization (TRPO) is a popular and empirically successful policy search algorithm in Reinforcement Learning (RL) in which a surrogate problem, that restricts consecutive policies to be 'close' to one another, is iteratively solved.
Practical Risk Measures in Reinforcement Learning
Practical application of Reinforcement Learning (RL) often involves risk considerations.
Finite Sample Analysis Of Dynamic Regression Parameter Learning
The proposed bounds depend on the shape of a certain spectrum related to the system operator, and thus provide the first known explicit geometric parameter of the data that can be used to bound estimation errors.
Topic Modeling via Full Dependence Mixtures
FDMs model second moment under general generative assumptions on the data.
Tight Regret Bounds for Model-Based Reinforcement Learning with Greedy Policies
In this paper, we focus on model-based RL in the finite-state finite-horizon MDP setting and establish that exploring with \emph{greedy policies} -- act by \emph{1-step planning} -- can achieve tight minimax performance in terms of regret, $\tilde{\mathcal{O}}(\sqrt{HSAT})$.
Model-based Reinforcement Learning
reinforcement-learning
+2
Inverse Reinforcement Learning in Contextual MDPs
Most importantly, we show both theoretically and empirically that our algorithms perform zero-shot transfer (generalize to new and unseen contexts).
Action Assembly: Sparse Imitation Learning for Text Based Games with Combinatorial Action Spaces
We propose a computationally efficient algorithm that combines compressed sensing with imitation learning to solve text-based games with combinatorial action spaces.
Distributional Policy Optimization: An Alternative Approach for Continuous Control
We show that optimizing over such sets results in local movement in the action space and thus convergence to sub-optimal solutions.
A Bayesian Approach to Robust Reinforcement Learning
Robust Markov Decision Processes (RMDPs) intend to ensure robustness with respect to changing or adversarial system behavior.
Batch-Size Independent Regret Bounds for the Combinatorial Multi-Armed Bandit Problem
We show that a linear dependence of the regret in the batch size in existing algorithms can be replaced by this smoothness parameter.
Image Matters: Scalable Detection of Offensive and Non-Compliant Content / Logo in Product Images
In this paper, we present a computer vision driven offensive and non-compliant image detection system for extremely large image datasets.
An adaptive stochastic optimization algorithm for resource allocation
This can be recast as a specific stochastic optimization problem where the objective is to maximize the cumulative reward, or equivalently to minimize the regret.
The Natural Language of Actions
We introduce Act2Vec, a general framework for learning context-based action representation for Reinforcement Learning.
Value Propagation for Decentralized Networked Deep Multi-agent Reinforcement Learning
To the best of our knowledge, it is the first MARL algorithm with convergence guarantee in the control, off-policy and non-linear function approximation setting.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
Action Robust Reinforcement Learning and Applications in Continuous Control
In this work we formalize two new criteria of robustness to action uncertainty.
Deep Neural Linear Bandits: Overcoming Catastrophic Forgetting through Likelihood Matching
We study the neural-linear bandit model for solving sequential decision-making problems with high dimensional side information.
Trust Region Value Optimization using Kalman Filtering
However, this approach ignores certain distributional properties of both the errors and value parameters.
Multi Instance Learning For Unbalanced Data
In the context of Multi Instance Learning, we analyze the Single Instance (SI) learning objective.
Exploration Conscious Reinforcement Learning Revisited
We continue and analyze properties of exploration-conscious optimal policies and characterize two general approaches to solve such criteria.
Multiple-Step Greedy Policies in Approximate and Online Reinforcement Learning
Multiple-step lookahead policies have demonstrated high empirical competence in Reinforcement Learning, via the use of Monte Carlo Tree Search or Model Predictive Control.
On-Line Learning of Linear Dynamical Systems: Exponential Forgetting in Kalman Filters
Based on this insight, we devise an on-line algorithm for improper learning of a linear dynamical system (LDS), which considers only a few most recent observations.
Inspiration Learning through Preferences
We denote this setup as \textit{Inspiration Learning} - knowledge transfer between agents that operate in different action spaces.
How to Combine Tree-Search Methods in Reinforcement Learning
Finite-horizon lookahead policies are abundantly used in Reinforcement Learning and demonstrate impressive empirical success.
Learn What Not to Learn: Action Elimination with Deep Reinforcement Learning
Learning how to act when there are many available actions in each state is a challenging task for Reinforcement Learning (RL) agents, especially when many of the actions are redundant or irrelevant.
Multi-user Communication Networks: A Coordinated Multi-armed Bandit Approach
Communication networks shared by many users are a widespread challenge nowadays.
Beyond the One-Step Greedy Approach in Reinforcement Learning
The famous Policy Iteration algorithm alternates between policy improvement and policy evaluation.
A General Framework for Bandit Problems Beyond Cumulative Objectives
Unlike the case of cumulative criteria, in the problems we study here the oracle policy, that knows the problem parameters a priori and is used to "center" the regret, is not trivial.
Multiple-Step Greedy Policies in Online and Approximate Reinforcement Learning
Multiple-step lookahead policies have demonstrated high empirical competence in Reinforcement Learning, via the use of Monte Carlo Tree Search or Model Predictive Control.
Nonlinear Distributional Gradient Temporal-Difference Learning
We devise a distributional variant of gradient temporal-difference (TD) learning.
Distributional Reinforcement Learning
Reinforcement Learning
Interdependent Gibbs Samplers
Gibbs sampling, as a model learning method, is known to produce the most accurate results available in a variety of domains, and is a de facto standard in these domains.
Deep Learning Reconstruction of Ultra-Short Pulses
Ultra-short laser pulses with femtosecond to attosecond pulse duration are the shortest systematic events humans can create.
Soft-Robust Actor-Critic Policy-Gradient
It learns an optimal policy with respect to a distribution over an uncertainty set and stays robust to model uncertainty but avoids the conservativeness of robust strategies.
Train on Validation: Squeezing the Data Lemon
We define the notion of on-average-validation-stable algorithms as one in which using small portions of validation data for training does not overfit the model selection process.
Beyond the One Step Greedy Approach in Reinforcement Learning
The famous Policy Iteration algorithm alternates between policy improvement and policy evaluation.
Learning Robust Options
We present a Robust Options Policy Iteration (ROPI) algorithm with convergence guarantees, which learns options that are robust to model uncertainty.
The Stochastic Firefighter Problem
We devise a simple policy that only vaccinates neighbors of infected nodes and is optimal on regular trees and on general graphs for a sufficiently large budget.
Situationally Aware Options
We learn reusable options in different scenarios in a RoboCup soccer domain (i. e., winning/losing).
End-to-End Differentiable Adversarial Imitation Learning
Generative Adversarial Networks (GANs) have been successfully applied to the problem of policy imitation in a model-free setup.
Multi-objective Bandits: Optimizing the Generalized Gini Index
We study the multi-armed bandit (MAB) problem where the agent receives a vectorial feedback that encodes many possibly competing objectives to be optimized.
Shallow Updates for Deep Reinforcement Learning
In this work we propose a hybrid approach -- the Least Squares Deep Q-Network (LS-DQN), which combines rich feature representations learned by a DRL algorithm with the stability of a linear least squares method.
Finite Sample Analyses for TD(0) with Function Approximation
TD(0) is one of the most commonly used algorithms in reinforcement learning.
Finite Sample Analysis of Two-Timescale Stochastic Approximation with Applications to Reinforcement Learning
Using this, we provide a concentration bound, which is the first such result for a two-timescale SA.
Deep Robust Kalman Filter
The Deep-RoK algorithm is a robust Bayesian method, based on the Extended Kalman Filter (EKF), that accounts for both the uncertainty in the weights of the approximated value function and the uncertainty in the transition probabilities, improving the robustness of the agent.
Online Learning with Many Experts
We study the problem of prediction with expert advice when the number of experts in question may be extremely large or even infinite.
Consistent On-Line Off-Policy Evaluation
The problem of on-line off-policy evaluation (OPE) has been actively studied in the last decade due to its importance both as a stand-alone problem and as a module in a policy improvement scheme.
Rotting Bandits
In the classical MAB problem, a decision maker must choose an arm at each time step, upon which she receives a reward.
Outlier Robust Online Learning
We consider the problem of learning from noisy data in practical settings where the size of data is too large to store on a single machine.
Adaptive Lambda Least-Squares Temporal Difference Learning
Temporal Difference learning or TD($\lambda$) is a fundamental algorithm in the field of reinforcement learning.
Supervised Learning for Optimal Power Flow as a Real-Time Proxy
In this work we design and compare different supervised learning algorithms to compute the cost of Alternating Current Optimal Power Flow (ACOPF).
Model-based Adversarial Imitation Learning
A model-based approach for the problem of adversarial imitation learning.
Adaptive Skills Adaptive Partitions (ASAP)
We introduce the Adaptive Skills, Adaptive Partitions (ASAP) framework that (1) learns skills (i. e., temporally extended actions or options) as well as (2) where to apply them.
Unit Commitment using Nearest Neighbor as a Short-Term Proxy
We devise the Unit Commitment Nearest Neighbor (UCNN) algorithm to be used as a proxy for quickly approximating outcomes of short-term decisions, to make tractable hierarchical long-term assessment and planning for large power systems.
Is a picture worth a thousand words? A Deep Multi-Modal Fusion Architecture for Product Classification in e-commerce
Classifying products into categories precisely and efficiently is a major challenge in modern e-commerce.
Situational Awareness by Risk-Conscious Skills
In addition, the learned risk aware skills are able to mitigate reward-based model misspecification.
A nonparametric sequential test for online randomized experiments
The proposed test does not require knowledge of the underlying probability distribution generating the data.
Bayesian Reinforcement Learning: A Survey
The objective of the paper is to provide a comprehensive survey on Bayesian RL algorithms and their theoretical and empirical properties.
How to Allocate Resources For Features Acquisition?
We study classification problems where features are corrupted by noise and where the magnitude of the noise in each feature is influenced by the resources allocated to its acquisition.
Visualizing Dynamics: from t-SNE to SEMI-MDPs
Deep Reinforcement Learning (DRL) is a trending field of research, showing great promise in many challenging problems such as playing Atari, solving Go and controlling robots.
Deep Reinforcement Learning Discovers Internal Models
Deep Reinforcement Learning (DRL) is a trending field of research, showing great promise in challenging problems such as playing Atari, solving Go and controlling robots.
Bending the Curve: Improving the ROC Curve Through Error Redistribution
Features that hold information about the "difficulty" of the data may be non-discriminative and are therefore disregarded in the classification process.
A Reinforcement Learning System to Encourage Physical Activity in Diabetes Patients
Messages were personalized through a Reinforcement Learning (RL) algorithm which optimized messages to improve each participant's compliance with the activity regimen.
Clustering Time Series and the Surprising Robustness of HMMs
Suppose that we are given a time series where consecutive samples are believed to come from a probabilistic source, that the source changes from time to time and that the total number of sources is fixed.
A Deep Hierarchical Approach to Lifelong Learning in Minecraft
Skill distillation enables the HDRLN to efficiently retain knowledge and therefore scale in lifelong learning, by accumulating knowledge and encapsulating multiple reusable skills into a single distilled network.
Hierarchical Decision Making In Electricity Grid Management
The power grid is a complex and vital system that necessitates careful reliability management.
Adaptive Skills, Adaptive Partitions (ASAP)
We introduce the Adaptive Skills, Adaptive Partitions (ASAP) framework that (1) learns skills (i. e., temporally extended actions or options) as well as (2) where to apply them.
Iterative Hierarchical Optimization for Misspecified Problems (IHOMP)
For complex, high-dimensional Markov Decision Processes (MDPs), it may be necessary to represent the policy with function approximation.
Graying the black box: Understanding DQNs
In recent years there is a growing interest in using deep representations for reinforcement learning.
Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms
As most deep learning algorithms are stochastic (e. g., Stochastic Gradient Descent, Dropout, and Bayes-by-backprop), we revisit the robustness arguments of Xu & Mannor, and introduce a new approach, ensemble robustness, that concerns the robustness of a population of hypotheses.
Online Learning for Adversaries with Memory: Price of Past Mistakes
In this work we extend the notion of learning with memory to the general Online Convex Optimization (OCO) framework, and present two algorithms that attain low regret.
Learn on Source, Refine on Target:A Model Transfer Learning Framework with Random Forests
We propose novel model transfer-learning methods that refine a decision forest model M learned within a "source" domain using a training set sampled from a "target" domain, assumed to be a variation of the source.
Generalized Emphatic Temporal Difference Learning: Bias-Variance Analysis
We consider the off-policy evaluation problem in Markov decision processes with function approximation.
Emphatic TD Bellman Operator is a Contraction
Recently, \citet{SuttonMW15} introduced the emphatic temporal differences (ETD) algorithm for off-policy evaluation in Markov decision processes.
Reinforcement Learning for the Unit Commitment Problem
In this work we solve the day-ahead unit commitment (UC) problem, by formulating it as a Markov decision process (MDP) and finding a low-cost policy for generation scheduling.
Bootstrapping Skills
The monolithic approach to policy representation in Markov Decision Processes (MDPs) looks for a single policy that can be represented as a function from states to actions.
Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach
Our first contribution is to show that a CVaR objective, besides capturing risk sensitivity, has an alternative interpretation as expected cost under worst-case modeling errors, for a given error budget.
Multi-user lax communications: a multi-armed bandit approach
Inspired by cognitive radio networks, we consider a setting where multiple users share several channels modeled as a multi-user multi-armed bandit (MAB) problem.
Overlapping Communities Detection via Measure Space Embedding
We present a new algorithm for community detection.
Overlapping Community Detection by Online Cluster Aggregation
We present a new online algorithm for detecting overlapping communities.
Actively Learning to Attract Followers on Twitter
Twitter, a popular social network, presents great opportunities for on-line machine learning research.
Policy Gradient for Coherent Risk Measures
For static risk measures, our approach is in the spirit of policy gradient algorithms and combines a standard sampling approach with convex programming.
Off-policy evaluation for MDPs with unknown structure
Off-policy learning in dynamic decision problems is essential for providing strong evidence that a new policy is better than the one in use.
Contextual Markov Decision Processes
The objective is to learn a strategy that maximizes the accumulated reward across all contexts.
Implicit Temporal Differences
In reinforcement learning, the TD($\lambda$) algorithm is a fundamental policy evaluation method with an efficient online implementation that is suitable for large-scale problems.
How hard is my MDP?" The distribution-norm to the rescue"
In Reinforcement Learning (RL), state-of-the-art algorithms require a large number of samples per state-action pair to estimate the transition kernel $p$.
Robust Logistic Regression and Classification
We consider logistic regression with arbitrary outliers in the covariate matrix.
Nonstochastic Multi-Armed Bandits with Graph-Structured Feedback
This naturally models several situations where the losses of different actions are related, and knowing the loss of one action provides information on the loss of other actions.