Search Results for author:
Found 22 papers, 4 papers with code
Combining policy gradient and Q-learning
Policy gradient is an efficient technique for improving a policy in a reinforcement learning setting.
The Uncertainty Bellman Equation and Exploration
In this paper we consider a similar \textit{uncertainty} Bellman equation (UBE), which connects the uncertainty at any time-step to the expected uncertainties at subsequent time-steps, thereby extending the potential exploratory benefit of a policy beyond individual time-steps.
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
We motivate 'adversarial risk' as an objective for achieving models robust to worst-case inputs.
Training verified learners with learned verifiers
This paper proposes a new algorithmic framework, predictor-verifier training, to train neural networks that are verifiable, i. e., networks that provably satisfy some desired input-output properties.
Variational Bayesian Reinforcement Learning with Regret Bounds
We show deep connections of this approach to the soft-max and maximum-entropy strands of research in reinforcement learning.
Hamiltonian Descent Methods
Yet, crucially the kinetic gradient map can be designed to incorporate information about the convex conjugate in a fashion that allows for linear convergence on convex functions that may be non-smooth or non-strongly convex.
Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles
We show that increasing the number of parameters in adversarially-trained models increases their robustness, and in particular that ensembling smaller models while adversarially training the entire ensemble as a single model is a more efficient way of spending said budget than simply using a larger single model.
Verification of Non-Linear Specifications for Neural Networks
We show that a number of important properties of interest can be modeled within this class, including conservation of energy in a learned dynamics model of a physical system; semantic consistency of a classifier's output labels under adversarial perturbations and bounding errors in a system that predicts the summation of handwritten digits.
Making Sense of Reinforcement Learning and Probabilistic Inference
Reinforcement learning (RL) combines a control problem with statistical estimation: The system dynamics are not known to the agent, but can be learned through experience.
Matrix games with bandit feedback
We study a version of the classical zero-sum matrix game with unknown payoff matrix and bandit feedback, where the players only observe each others actions and a noisy payoff.
Sample Efficient Reinforcement Learning with REINFORCE
Policy gradient methods are among the most effective methods for large-scale reinforcement learning, and their empirical success has prompted several works that develop the foundation of their global convergence theory.
Solving Mixed Integer Programs Using Neural Networks
Our approach constructs two corresponding neural network-based components, Neural Diving and Neural Branching, to use in a base MIP solver such as SCIP.
Discovering a set of policies for the worst case reward
Our main contribution is a policy iteration algorithm that builds a set of policies in order to maximize the worst-case performance of the resulting SMP on the set of tasks.
Discovering Diverse Nearly Optimal Policies with Successor Features
We propose Diverse Successive Policies, a method for discovering policies that are diverse in the space of Successor Features, while assuring that they are near optimal.
Reward is enough for convex MDPs
Maximising a cumulative reward function that is Markov and stationary, i. e., defined over state-action pairs and independent of time, is sufficient to capture many kinds of goals in a Markov decision process (MDP).
Evaluating Predictive Distributions: Does Bayesian Deep Learning Work?
This paper introduces \textit{The Neural Testbed}, which provides tools for the systematic evaluation of agents that generate such predictions.
The Neural Testbed: Evaluating Joint Predictions
Predictive distributions quantify uncertainties ignored by point estimates.
Variational Bayesian Optimistic Sampling
We derive a set of Bayesian `optimistic' policies which, in the stochastic multi-armed bandit case, includes the Thompson sampling policy.
On the connection between Bregman divergence and value in regularized Markov decision processes
In this short note we derive a relationship between the Bregman divergence from the current policy to the optimal policy and the suboptimality of the current value function in a regularized Markov decision process.
POMRL: No-Regret Learning-to-Plan with Increasing Horizons
We study the problem of planning under model uncertainty in an online meta-reinforcement learning (RL) setting where an agent is presented with a sequence of related tasks with limited interactions per task.
ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPs
Such applications often require to put constraints on the agent's behavior.
Efficient Exploration via Epistemic-Risk-Seeking Policy Optimization
Optimism in the face of uncertainty is a well-known heuristic with theoretical guarantees in the tabular setting, but how best to translate the principle to deep reinforcement learning, which involves online stochastic gradients and deep network function approximators, is not fully understood.
