Search Results for author:
Found 87 papers, 27 papers with code
Learning Natural Language Generation with Truncated Reinforcement Learning
To our knowledge, it is the first approach that successfully learns a language generation policy without pre-training, using only reinforcement learning.
On Imitation in Mean-field Games
We explore the problem of imitation learning (IL) in the context of mean-field games (MFGs), where the goal is to imitate the behavior of a population of agents following a Nash equilibrium policy according to some unknown payoff function.
Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedback
Despite the seeming success of contemporary grounded text generation systems, they often tend to generate factually inconsistent text with respect to their input.
Abstractive Text Summarization
Natural Language Inference
+2
Regularization and Variance-Weighted Regression Achieves Minimax Optimality in Linear MDPs: Theory and Practice
Mirror descent value iteration (MDVI), an abstraction of Kullback-Leibler (KL) and entropy-regularized reinforcement learning (RL), has served as the basis for recent high-performing practical RL algorithms.
Get Back Here: Robust Imitation by Return-to-Distribution Planning
We consider the Imitation Learning (IL) setup where expert data are not collected on the actual deployment environment but on a different version.
SingSong: Generating musical accompaniments from singing
We present SingSong, a system that generates instrumental music to accompany input vocals, potentially offering musicians and non-musicians alike an intuitive new way to create music featuring their own voice.
On the importance of data collection for training general goal-reaching policies
Are the quantity and quality of data truly transformative to the performance of a general controller?
Emergent Communication: Generalization and Overfitting in Lewis Games
Lewis signaling games are a class of simple communication games for simulating the emergence of language.
vec2text with Round-Trip Translations
We propose a simple data augmentation technique based on round-trip translations and show in extensive experiments that the resulting vec2text model surprisingly leads to vector spaces that fulfill our four desired properties and that this model strongly outperforms both standard and denoising auto-encoders.
AudioLM: a Language Modeling Approach to Audio Generation
We introduce AudioLM, a framework for high-quality audio generation with long-term consistency.
Learning Correlated Equilibria in Mean-Field Games
The designs of many large-scale systems today, from traffic routing environments to smart grids, rely on game-theoretic equilibrium concepts.
KL-Entropy-Regularized RL with a Generative Model is Minimax Optimal
In this work, we consider and analyze the sample complexity of model-free reinforcement learning with a generative model.
Learning Mean Field Games: A Survey
Non-cooperative and cooperative games with a very large number of players have many applications but remain generally intractable when the number of players increases.
Scalable Deep Reinforcement Learning Algorithms for Mean Field Games
One limiting factor to further scale up using RL is that existing algorithms to solve MFGs require the mixing of approximated quantities such as strategies or $q$-values.
Lazy-MDPs: Towards Interpretable Reinforcement Learning by Learning When to Act
We deem those states and corresponding actions important since they explain the difference in performance between the default and the new, lazy policy.
RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learning
We introduce RLDS (Reinforcement Learning Datasets), an ecosystem for recording, replaying, manipulating, annotating and sharing data in the context of Sequential Decision Making (SDM) including Reinforcement Learning (RL), Learning from Demonstrations, Offline RL or Imitation Learning.
Continuous Control with Action Quantization from Demonstrations
The proposed approach consists in learning a discretization of continuous action spaces from human demonstrations.
Generalization in Mean Field Games by Learning Master Policies
Mean Field Games (MFGs) can potentially scale multi-agent systems to extremely large populations of agents.
Learning Natural Language Generation from Scratch
This paper introduces TRUncated ReinForcement Learning for Language (TrufLL), an original ap-proach to train conditional language models from scratch by only using reinforcement learning (RL).
Offline Reinforcement Learning as Anti-Exploration
This is the converse of exploration in RL, which favors such actions.
There Is No Turning Back: A Self-Supervised Approach for Reversibility-Aware Reinforcement Learning
We propose to learn to distinguish reversible from irreversible actions for better informed decision-making in Reinforcement Learning (RL).
Concave Utility Reinforcement Learning: the Mean-Field Game Viewpoint
Mean-field Games (MFGs) are a continuous approximation of many-agent RL.
What Matters for Adversarial Imitation Learning?
To tackle this issue, we implement more than 50 of these choices in a generic adversarial imitation learning framework and investigate their impacts in a large-scale study (>500k trained agents) with both synthetic and human-generated demonstrations.
Hyperparameter Selection for Imitation Learning
The vast literature in imitation learning mostly considers this reward function to be available for HP selection, but this is not a realistic setting.
Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulness
Sparse rewards are double-edged training signals in reinforcement learning: easy to design but hard to optimize.
Mean Field Games Flock! The Reinforcement Learning Way
We present a method enabling a large number of agents to learn how to flock, which is a natural behavior observed in large populations of animals.
Relevant Action Matters : Motivating agent with action usefulness
We evaluate RAM on the procedurally-generated environment MiniGrid, against state-of-the-art methods.
Offline Reinforcement Learning with Pseudometric Learning
In the presence of function approximation, and under the assumption of limited coverage of the state-action space of the environment, it is necessary to enforce the policy to visit state-action pairs close to the support of logged transitions.
Scaling up Mean Field Games with Online Mirror Descent
We address scaling up equilibrium computation in Mean Field Games (MFGs) using Online Mirror Descent (OMD).
Adversarially Guided Actor-Critic
Despite definite success in deep reinforcement learning problems, actor-critic algorithms are still confronted with sample inefficiency in complex environments, particularly in tasks where efficient exploration is a bottleneck.
What Matters for On-Policy Deep Actor-Critic Methods? A Large-Scale Study
In recent years, reinforcement learning (RL) has been successfully applied to many different continuous control tasks.
Self-Imitation Advantage Learning
Self-imitation learning is a Reinforcement Learning (RL) method that encourages actions whose returns were higher than expected, which helps in hard exploration and sparse reward problems.
Leverage the Average: an Analysis of KL Regularization in Reinforcement Learning
Recent Reinforcement Learning (RL) algorithms making use of Kullback-Leibler (KL) regularization as a core component have shown outstanding performance.
Learning from Heterogeneous EEG Signals with Differentiable Channel Reordering
We propose CHARM, a method for training a single neural network across inconsistent input channels.
Supervised Seeded Iterated Learning for Interactive Language Learning
Language drift has been one of the major obstacles to train language models through interaction.
A Machine of Few Words -- Interactive Speaker Recognition with Reinforcement Learning
To do so, we cast the speaker recognition task into a sequential decision-making problem that we solve with Reinforcement Learning.
Munchausen Reinforcement Learning
Bootstrapping is a core mechanism in Reinforcement Learning (RL).
Ranked #8 on
Atari Games
on Atari-57
The Monte Carlo Transformer: a stochastic self-attention model for sequence prediction
This paper introduces the Sequential Monte Carlo Transformer, an original approach that naturally captures the observations distribution in a transformer architecture.
Fictitious Play for Mean Field Games: Continuous Time Analysis and Applications
In this paper, we deepen the analysis of continuous time Fictitious Play learning algorithm to the consideration of various finite state Mean Field Game settings (finite horizon, $\gamma$-discounted), allowing in particular for the introduction of an additional common noise.
Show me the Way: Intrinsic Motivation from Demonstrations
Using an inverse RL approach, we show that complex exploration behaviors, reflecting different motivations, can be learnt and efficiently used by RL agents to solve tasks for which exhaustive exploration is prohibitive.
What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study
In recent years, on-policy reinforcement learning (RL) has been successfully applied to many different continuous control tasks.
Primal Wasserstein Imitation Learning
Imitation Learning (IL) methods seek to match the behavior of an agent with that of an expert.
Acme: A Research Framework for Distributed Reinforcement Learning
These implementations serve both as a validation of our design decisions as well as an important contribution to reproducibility in RL research.
Reinforcement Learning
Reinforcement learning (RL) is a general framework for adaptive control, which has proven to be efficient in many domains, e. g., board games, video games or autonomous vehicles.
Leverage the Average: an Analysis of KL Regularization in RL
Recent Reinforcement Learning (RL) algorithms making use of Kullback-Leibler (KL) regularization as a core component have shown outstanding performance.
Countering Language Drift with Seeded Iterated Learning
At each time step, the teacher is created by copying the student agent, before being finetuned to maximize task completion.
HIGhER : Improving instruction following with Hindsight Generation for Experience Replay
Language creates a compact representation of the world and allows the description of unlimited situations and objectives through compositionality.
Momentum in Reinforcement Learning
We adapt the optimization's concept of momentum to reinforcement learning.
On Connections between Constrained Optimization and Reinforcement Learning
In this paper, we draw connections between DP and (constrained) convex optimization.
I'm sorry Dave, I'm afraid I can't do that, Deep Q-learning from forbidden action
Violating constraints thus results in rejected actions or entering in a safe mode driven by an external controller, making RL agents incapable of learning from their mistakes.
Self-Educated Language Agent with Hindsight Experience Replay for Instruction Following
Language creates a compact representation of the world and allows the description of unlimited situations and objectives through compositionality.
Self-Attentional Credit Assignment for Transfer in Reinforcement Learning
The ability to transfer knowledge to novel environments and tasks is a sensible desiderata for general learning agents.
On the Convergence of Model Free Learning in Mean Field Games
In order to design scalable algorithms for systems with a large population of interacting agents (e. g. swarms), this paper focuses on Mean Field MAS, where the number of agents is asymptotically infinite.
MULEX: Disentangling Exploitation from Exploration in Deep RL
An agent learning through interactions should balance its action selection process between probing the environment to discover new rewards and using the information acquired in the past to adopt useful behaviour.
Foolproof Cooperative Learning
We prove that in repeated symmetric games, this algorithm is a learning equilibrium.
Deep Conservative Policy Iteration
Conservative Policy Iteration (CPI) is a founding algorithm of Approximate Dynamic Programming (ADP).
CopyCAT: Taking Control of Neural Policies with Constant Attacks
In this setting, the adversary cannot directly modify the agent's state -- its representation of the environment -- but can only attack the agent's observation -- its perception of the environment.
Towards Consistent Performance on Atari using Expert Demonstrations
Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks such as Atari 2600 games.
Budgeted Reinforcement Learning in Continuous State Space
A Budgeted Markov Decision Process (BMDP) is an extension of a Markov Decision Process to critical applications requiring safety constraints.
A Theory of Regularized Markov Decision Processes
Many recent successful (deep) reinforcement learning algorithms make use of regularization, generally based on entropy or Kullback-Leibler divergence.
Playing the Game of Universal Adversarial Perturbations
We study the problem of learning classifiers robust to universal adversarial perturbations.
Visual Reasoning with Multi-hop Feature Modulation
Recent breakthroughs in computer vision and natural language processing have spurred interest in challenging multi-modal tasks such as visual question-answering and visual dialogue.
Observe and Look Further: Achieving Consistent Performance on Atari
Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks such as Atari 2600 games.
End-to-End Automatic Speech Translation of Audiobooks
We investigate end-to-end speech-to-text translation on a corpus of audiobooks specifically augmented for this task.
Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
We propose a general and model-free approach for Reinforcement Learning (RL) on real robotics with sparse rewards.
LIG-CRIStAL System for the WMT17 Automatic Post-Editing Task
This paper presents the LIG-CRIStAL submission to the shared Automatic Post- Editing task of WMT 2017.
Modulating early visual processing by language
It is commonly assumed that language refers to high-level visual concepts while leaving low-level visual processing unaffected.
Noisy Networks for Exploration
We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to aid efficient exploration.
Ranked #1 on
Atari Games
on Atari 2600 Surround
Observational Learning by Reinforcement Learning
Observational learning is a type of learning that occurs as a function of observing, retaining and possibly replicating or imitating the behaviour of another agent.
Deep Q-learning from Demonstrations
We present an algorithm, Deep Q-learning from Demonstrations (DQfD), that leverages small sets of demonstration data to massively accelerate the learning process even from relatively small amounts of demonstration data and is able to automatically assess the necessary ratio of demonstration data while learning thanks to a prioritized replay mechanism.
End-to-end optimization of goal-driven and visually grounded dialogue systems
End-to-end design of dialogue systems has recently become a popular research topic thanks to powerful tools such as encoder-decoder architectures for sequence-to-sequence learning.
Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation
This paper proposes a first attempt to build an end-to-end speech-to-text translation system, which does not use source language transcription during learning or decoding.
GuessWhat?! Visual object discovery through multi-modal dialogue
Our key contribution is the collection of a large-scale dataset consisting of 150K human-played games with a total of 800K visual question-answer pairs on 66K images.
Is the Bellman residual a bad proxy?
This paper aims at theoretically and empirically comparing two standard optimization criteria for Reinforcement Learning: i) maximization of the mean value and ii) minimization of the Bellman residual.
Difference of Convex Functions Programming Applied to Control with Expert Data
This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data.
MultiVec: a Multilingual and Multilevel Representation Learning Toolkit for NLP
We present MultiVec, a new toolkit for computing continuous representations for text at different granularity levels (word-level or sequences of words).
Difference of Convex Functions Programming for Reinforcement Learning
Controlling this residual allows controlling the distance to the optimal action-value function, and we show that minimizing an empirical norm of the OBR is consistant in the Vapnik sense.
NASTIA: Negotiating Appointment Setting Interface
Appointment scheduling is a hybrid task halfway between slot-filling and negotiation.
DINASTI: Dialogues with a Negotiating Appointment Setting Interface
NASTIA is a reinforcement learning-based system.
Kalman Temporal Differences
Because reinforcement learning suffers from a lack of scalability, online value (and Q-) function approximation has received increasing interest this last decade.
Inverse Reinforcement Learning through Structured Classification
This paper adresses the inverse reinforcement learning (IRL) problem, that is inferring a reward for which a demonstrated expert behavior is optimal.
