Search Results for author: Joelle Pineau

Found 138 papers, 64 papers with code

Do Encoder Representations of Generative Dialogue Models have sufficient summary of the Information about the task ?

1 code implementation SIGDIAL (ACL) 2021 Prasanna Parthasarathi, Joelle Pineau, Sarath Chandar

Predicting the next utterance in dialogue is contingent on encoding of users’ input text to generate appropriate and relevant response in data-driven approaches.

Text Generation

Sometimes We Want Ungrammatical Translations

1 code implementation Findings (EMNLP) 2021 Prasanna Parthasarathi, Koustuv Sinha, Joelle Pineau, Adina Williams

Rapid progress in Neural Machine Translation (NMT) systems over the last few years has focused primarily on improving translation quality, and as a secondary focus, improving robustness to perturbations (e. g. spelling).

Machine Translation Translation

Questions Are All You Need to Train a Dense Passage Retriever

no code implementations21 Jun 2022 Devendra Singh Sachan, Mike Lewis, Dani Yogatama, Luke Zettlemoyer, Joelle Pineau, Manzil Zaheer

We introduce ART, a new corpus-level autoencoding approach for training dense retrieval models that does not require any labeled training data.

Denoising Language Modelling

New Insights on Reducing Abrupt Representation Change in Online Continual Learning

1 code implementation ICLR 2022 Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, Eugene Belilovsky

In this work, we focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream, and new classes must be distinguished from previous ones.

Continual Learning

Estimating causal effects with optimization-based methods: A review and empirical comparison

no code implementations28 Feb 2022 Martin Cousineau, Vedat Verter, Susan A. Murphy, Joelle Pineau

In the absence of randomized controlled and natural experiments, it is necessary to balance the distributions of (observable) covariates of the treated and control groups in order to obtain an unbiased estimate of a causal effect of interest; otherwise, a different effect size may be estimated, and incorrect recommendations may be given.

Causal Inference

Efficient Continual Learning Ensembles in Neural Network Subspaces

no code implementations20 Feb 2022 Thang Doan, Seyed Iman Mirzadeh, Joelle Pineau, Mehrdad Farajtabar

A growing body of research in continual learning focuses on the catastrophic forgetting problem.

Continual Learning

Robust Policy Learning over Multiple Uncertainty Sets

no code implementations14 Feb 2022 Annie Xie, Shagun Sodhani, Chelsea Finn, Joelle Pineau, Amy Zhang

Reinforcement learning (RL) agents need to be robust to variations in safety-critical environments.

A Generalized Bootstrap Target for Value-Learning, Efficiently Combining Value and Feature Predictions

no code implementations5 Jan 2022 Anthony GX-Chen, Veronica Chelu, Blake A. Richards, Joelle Pineau

We illustrate that incorporating predictive knowledge through an $\eta\gamma$-discounted SF model makes more efficient use of sampled experience, compared to either extreme, i. e. bootstrapping entirely on the value function estimate, or bootstrapping on the product of separately estimated successor features and instantaneous reward models.

Block Contextual MDPs for Continual Learning

no code implementations13 Oct 2021 Shagun Sodhani, Franziska Meier, Joelle Pineau, Amy Zhang

In this work, we propose to examine this continual reinforcement learning setting through the block contextual MDP (BC-MDP) framework, which enables us to relax the assumption of stationarity.

Continual Learning Generalization Bounds +1

OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation

1 code implementation21 Jun 2021 Jongmin Lee, Wonseok Jeon, Byung-Jun Lee, Joelle Pineau, Kee-Eung Kim

We consider the offline reinforcement learning (RL) setting where the agent aims to optimize the policy solely from the data without further environment interactions.

Offline RL

Do Encoder Representations of Generative Dialogue Models Encode Sufficient Information about the Task ?

1 code implementation20 Jun 2021 Prasanna Parthasarathi, Joelle Pineau, Sarath Chandar

Predicting the next utterance in dialogue is contingent on encoding of users' input text to generate appropriate and relevant response in data-driven approaches.

Text Generation

A Brief Study on the Effects of Training Generative Dialogue Models with a Semantic loss

1 code implementation SIGDIAL (ACL) 2021 Prasanna Parthasarathi, Mohamed Abdelsalam, Joelle Pineau, Sarath Chandar

Neural models trained for next utterance generation in dialogue task learn to mimic the n-gram sequences in the training set with training objectives like negative log-likelihood (NLL) or cross-entropy.

Language Modelling Semantic Similarity +2

SPeCiaL: Self-Supervised Pretraining for Continual Learning

no code implementations16 Jun 2021 Lucas Caccia, Joelle Pineau

This paper presents SPeCiaL: a method for unsupervised pretraining of representations tailored for continual learning.

Continual Learning Few-Shot Learning

Correcting Momentum in Temporal Difference Learning

1 code implementation7 Jun 2021 Emmanuel Bengio, Joelle Pineau, Doina Precup

A common optimization tool used in deep reinforcement learning is momentum, which consists in accumulating and discounting past gradients, reapplying them at each iteration.

reinforcement-learning

Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPs

no code implementations NeurIPS 2021 Harsh Satija, Philip S. Thomas, Joelle Pineau, Romain Laroche

We study the problem of Safe Policy Improvement (SPI) under constraints in the offline Reinforcement Learning (RL) setting.

Sometimes We Want Translationese

no code implementations15 Apr 2021 Prasanna Parthasarathi, Koustuv Sinha, Joelle Pineau, Adina Williams

Rapid progress in Neural Machine Translation (NMT) systems over the last few years has been driven primarily towards improving translation quality, and as a secondary focus, improved robustness to input perturbations (e. g. spelling and grammatical mistakes).

Machine Translation Translation

Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little

no code implementations EMNLP 2021 Koustuv Sinha, Robin Jia, Dieuwke Hupkes, Joelle Pineau, Adina Williams, Douwe Kiela

A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to represent the syntactic structures prevalent in classical NLP pipelines.

Language Modelling Masked Language Modeling

New Insights on Reducing Abrupt Representation Change in Online Continual Learning

2 code implementations11 Apr 2021 Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, Eugene Belilovsky

In this work, we focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream, and new classes must be distinguished from previous ones.

Continual Learning Metric Learning

Quasi-Equivalence Discovery for Zero-Shot Emergent Communication

no code implementations14 Mar 2021 Kalesha Bullard, Douwe Kiela, Franziska Meier, Joelle Pineau, Jakob Foerster

In contrast, in this work, we present a novel problem setting and the Quasi-Equivalence Discovery (QED) algorithm that allows for zero-shot coordination (ZSC), i. e., discovering protocols that can generalize to independently trained agents.

Resolving Causal Confusion in Reinforcement Learning via Robust Exploration

no code implementations ICLR Workshop SSL-RL 2021 Clare Lyle, Amy Zhang, Minqi Jiang, Joelle Pineau, Yarin Gal

To address this, we present a robust exploration strategy which enables causal hypothesis-testing by interaction with the environment.

reinforcement-learning

Domain Adversarial Reinforcement Learning

no code implementations14 Feb 2021 Bonnie Li, Vincent François-Lavet, Thang Doan, Joelle Pineau

We consider the problem of generalization in reinforcement learning where visual aspects of the observations might differ, e. g. when there are different backgrounds or change in contrast, brightness, etc.

reinforcement-learning

Multi-Task Reinforcement Learning with Context-based Representations

2 code implementations11 Feb 2021 Shagun Sodhani, Amy Zhang, Joelle Pineau

We posit that an efficient approach to knowledge transfer is through the use of multiple context-dependent, composable representations shared across a family of tasks.

Multi-Task Learning reinforcement-learning

Exploring the Limits of Few-Shot Link Prediction in Knowledge Graphs

no code implementations EACL 2021 Dora Jambor, Komal Teru, Joelle Pineau, William L. Hamilton

Real-world knowledge graphs are often characterized by low-frequency relations - a challenge that has prompted an increasing interest in few-shot link prediction methods.

Knowledge Graphs Link Prediction

COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction

1 code implementation13 Jan 2021 Anuroop Sriram, Matthew Muckley, Koustuv Sinha, Farah Shamout, Joelle Pineau, Krzysztof J. Geras, Lea Azour, Yindalon Aphinyanaphongs, Nafissa Yakubova, William Moore

The first is deterioration prediction from a single image, where our model achieves an area under receiver operating characteristic curve (AUC) of 0. 742 for predicting an adverse event within 96 hours (compared to 0. 703 with supervised pretraining) and an AUC of 0. 765 for predicting oxygen requirements greater than 6 L a day at 24 hours (compared to 0. 749 with supervised pretraining).

Representation Learning Self-Supervised Learning

GraphLog: A Benchmark for Measuring Logical Generalization in Graph Neural Networks

1 code implementation1 Jan 2021 Koustuv Sinha, Shagun Sodhani, Joelle Pineau, William L. Hamilton

In this work, we study the logical generalization capabilities of GNNs by designing a benchmark suite grounded in first-order logic.

Continual Learning Knowledge Graphs +1

UnNatural Language Inference

1 code implementation ACL 2021 Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau, Adina Williams

We provide novel evidence that complicates this claim: we find that state-of-the-art Natural Language Inference (NLI) models assign the same labels to permuted examples as they do to the original, i. e. they are largely invariant to random word-order permutations.

Natural Language Inference Natural Language Understanding

Intervention Design for Effective Sim2Real Transfer

no code implementations3 Dec 2020 Melissa Mozifian, Amy Zhang, Joelle Pineau, David Meger

The goal of this work is to address the recent success of domain randomization and data augmentation for the sim2real setting.

Causal Inference Data Augmentation

Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populations

no code implementations29 Oct 2020 Kalesha Bullard, Franziska Meier, Douwe Kiela, Joelle Pineau, Jakob Foerster

Indeed, emergent communication is now a vibrant field of research, with common settings involving discrete cheap-talk channels.

Regularized Inverse Reinforcement Learning

no code implementations ICLR 2021 Wonseok Jeon, Chen-Yang Su, Paul Barde, Thang Doan, Derek Nowrouzezahrai, Joelle Pineau

Inverse Reinforcement Learning (IRL) aims to facilitate a learner's ability to imitate expert behavior by acquiring reward functions that explain the expert's decisions.

reinforcement-learning

Novelty Search in Representational Space for Sample Efficient Exploration

1 code implementation NeurIPS 2020 Ruo Yu Tao, Vincent François-Lavet, Joelle Pineau

We then leverage these intrinsic rewards for sample-efficient exploration with planning routines in representational space for hard exploration tasks with sparse rewards.

Efficient Exploration

Constrained Markov Decision Processes via Backward Value Functions

no code implementations ICML 2020 Harsh Satija, Philip Amortila, Joelle Pineau

In standard RL, the agent is incentivized to explore any behavior as long as it maximizes rewards, but in the real world, undesired behavior can damage either the system or the agent in a way that breaks the learning process itself.

Learning Robust State Abstractions for Hidden-Parameter Block MDPs

2 code implementations ICLR 2021 Amy Zhang, Shagun Sodhani, Khimya Khetarpal, Joelle Pineau

Further, we provide transfer and generalization bounds based on task and state similarity, along with sample complexity bounds that depend on the aggregate number of samples across tasks, rather than the number of tasks, a significant improvement over prior work that use the same environment assumptions.

Generalization Bounds Meta Reinforcement Learning +1

TDprop: Does Jacobi Preconditioning Help Temporal Difference Learning?

no code implementations6 Jul 2020 Joshua Romoff, Peter Henderson, David Kanaa, Emmanuel Bengio, Ahmed Touati, Pierre-Luc Bacon, Joelle Pineau

We investigate whether Jacobi preconditioning, accounting for the bootstrap term in temporal difference (TD) learning, can help boost performance of adaptive optimizers.

Deep interpretability for GWAS

no code implementations3 Jul 2020 Deepak Sharma, Audrey Durand, Marc-André Legault, Louis-Philippe Lemieux Perreault, Audrey Lemaçon, Marie-Pierre Dubé, Joelle Pineau

Genome-Wide Association Studies are typically conducted using linear models to find genetic variants associated with common diseases.

Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization

3 code implementations NeurIPS 2020 Paul Barde, Julien Roy, Wonseok Jeon, Joelle Pineau, Christopher Pal, Derek Nowrouzezahrai

Adversarial Imitation Learning alternates between learning a discriminator -- which tells apart expert's demonstrations from generated ones -- and a generator's policy to produce trajectories that can fool this discriminator.

Imitation Learning reinforcement-learning

Plan2Vec: Unsupervised Representation Learning by Latent Plans

1 code implementation7 May 2020 Ge Yang, Amy Zhang, Ari S. Morcos, Joelle Pineau, Pieter Abbeel, Roberto Calandra

In this paper we introduce plan2vec, an unsupervised representation learning approach that is inspired by reinforcement learning.

Motion Planning reinforcement-learning +1

Automated Personalized Feedback Improves Learning Gains in an Intelligent Tutoring System

no code implementations5 May 2020 Ekaterina Kochmar, Dung Do Vu, Robert Belfer, Varun Gupta, Iulian Vlad Serban, Joelle Pineau

Our model is used in Korbit, a large-scale dialogue-based ITS with thousands of students launched in 2019, and we demonstrate that the personalized feedback leads to considerable improvement in student learning outcomes and in the subjective evaluation of the feedback.

Natural Language Processing

Learning an Unreferenced Metric for Online Dialogue Evaluation

1 code implementation ACL 2020 Koustuv Sinha, Prasanna Parthasarathi, Jasmine Wang, Ryan Lowe, William L. Hamilton, Joelle Pineau

Evaluating the quality of a dialogue interaction between two agents is a difficult task, especially in open-domain chit-chat style dialogue.

Dialogue Evaluation

Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)

no code implementations27 Mar 2020 Joelle Pineau, Philippe Vincent-Lamarre, Koustuv Sinha, Vincent Larivière, Alina Beygelzimer, Florence d'Alché-Buc, Emily Fox, Hugo Larochelle

Reproducibility, that is obtaining similar results as presented in a paper or talk, using the same code and data (when available), is a necessary step to verify the reliability of research findings.

Evaluating Logical Generalization in Graph Neural Networks

1 code implementation ICML Workshop LifelongML 2020 Koustuv Sinha, Shagun Sodhani, Joelle Pineau, William L. Hamilton

Recent research has highlighted the role of relational inductive biases in building learning agents that can generalize and reason in a compositional manner.

Continual Learning Knowledge Graphs +2

Interference and Generalization in Temporal Difference Learning

no code implementations ICML 2020 Emmanuel Bengio, Joelle Pineau, Doina Precup

We study the link between generalization and interference in temporal-difference (TD) learning.

Invariant Causal Prediction for Block MDPs

1 code implementation ICML 2020 Amy Zhang, Clare Lyle, Shagun Sodhani, Angelos Filos, Marta Kwiatkowska, Joelle Pineau, Yarin Gal, Doina Precup

Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges.

Causal Inference Variable Selection

Stable Policy Optimization via Off-Policy Divergence Regularization

1 code implementation9 Mar 2020 Ahmed Touati, Amy Zhang, Joelle Pineau, Pascal Vincent

Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) are among the most successful policy gradient approaches in deep reinforcement learning (RL).

reinforcement-learning

Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic

no code implementations24 Feb 2020 Wonseok Jeon, Paul Barde, Derek Nowrouzezahrai, Joelle Pineau

Multi-agent adversarial inverse reinforcement learning (MA-AIRL) is a recent approach that applies single-agent AIRL to multi-agent problems where we seek to recover both policies for our agents and reward functions that promote expert-like behavior.

reinforcement-learning

On the interaction between supervision and self-play in emergent communication

1 code implementation ICLR 2020 Ryan Lowe, Abhinav Gupta, Jakob Foerster, Douwe Kiela, Joelle Pineau

A promising approach for teaching artificial agents to use natural language involves using human-in-the-loop training.

Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning

2 code implementations31 Jan 2020 Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, Joelle Pineau

Accurate reporting of energy and carbon usage is essential for understanding the potential climate impacts of machine learning research.

reinforcement-learning

No-Press Diplomacy: Modeling Multi-Agent Gameplay

no code implementations NeurIPS 2019 Philip Paquette, Yuchen Lu, Seton Steven Bocco, Max Smith, Satya O.-G., Jonathan K. Kummerfeld, Joelle Pineau, Satinder Singh, Aaron C. Courville

Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal.

Exploiting Spatial Invariance for Scalable Unsupervised Object Tracking

1 code implementation20 Nov 2019 Eric Crawford, Joelle Pineau

The ability to detect and track objects in the visual world is a crucial skill for any intelligent agent, as it is a necessary precursor to any object-level reasoning process.

Object Tracking

Online Learned Continual Compression with Adaptive Quantization Modules

1 code implementation ICML 2020 Lucas Caccia, Eugene Belilovsky, Massimo Caccia, Joelle Pineau

We show how to use discrete auto-encoders to effectively address this challenge and introduce Adaptive Quantization Modules (AQM) to control variation in the compression ability of the module at any given stage of learning.

Continual Learning Quantization

Off-Policy Policy Gradient Algorithms by Constraining the State Distribution Shift

no code implementations16 Nov 2019 Riashat Islam, Komal K. Teru, Deepak Sharma, Joelle Pineau

This data distribution shift between current and past samples can significantly impact the performance of most modern off-policy based policy optimization algorithms.

Continuous Control

Seeded self-play for language learning

no code implementations WS 2019 Abhinav Gupta, Ryan Lowe, Jakob Foerster, Douwe Kiela, Joelle Pineau

Once the meta-learning agent is able to quickly adapt to each population of agents, it can be deployed in new populations, including populations speaking human language.

Imitation Learning Meta-Learning

Benchmarking Batch Deep Reinforcement Learning Algorithms

4 code implementations3 Oct 2019 Scott Fujimoto, Edoardo Conti, Mohammad Ghavamzadeh, Joelle Pineau

Widely-used deep reinforcement learning algorithms have been shown to fail in the batch setting--learning from a fixed data set without interaction with the environment.

Q-Learning reinforcement-learning

Assessing Generalization in TD methods for Deep Reinforcement Learning

no code implementations25 Sep 2019 Emmanuel Bengio, Doina Precup, Joelle Pineau

Current Deep Reinforcement Learning (DRL) methods can exhibit both data inefficiency and brittleness, which seem to indicate that they generalize poorly.

reinforcement-learning

Online Learned Continual Compression with Stacked Quantization Modules

no code implementations25 Sep 2019 Lucas Caccia, Eugene Belilovsky, Massimo Caccia, Joelle Pineau

We first replace the episodic memory used in Experience Replay with SQM, leading to significant gains on standard continual learning benchmarks using a fixed memory budget.

Continual Learning Quantization

Attraction-Repulsion Actor-Critic for Continuous Control Reinforcement Learning

no code implementations17 Sep 2019 Thang Doan, Bogdan Mazoure, Moloud Abdar, Audrey Durand, Joelle Pineau, R. Devon Hjelm

Continuous control tasks in reinforcement learning are important because they provide an important framework for learning in high-dimensional state spaces with deceptive rewards, where the agent can easily become trapped into suboptimal solutions.

Continuous Control reinforcement-learning

No Press Diplomacy: Modeling Multi-Agent Gameplay

1 code implementation4 Sep 2019 Philip Paquette, Yuchen Lu, Steven Bocco, Max O. Smith, Satya Ortiz-Gagne, Jonathan K. Kummerfeld, Satinder Singh, Joelle Pineau, Aaron Courville

Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal.

CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text

5 code implementations IJCNLP 2019 Koustuv Sinha, Shagun Sodhani, Jin Dong, Joelle Pineau, William L. Hamilton

The recent success of natural language understanding (NLU) systems has been troubled by results highlighting the failure of these models to generalize in a systematic and robust way.

Inductive logic programming Natural Language Understanding +2

Learning Causal State Representations of Partially Observable Environments

no code implementations25 Jun 2019 Amy Zhang, Zachary C. Lipton, Luis Pineda, Kamyar Azizzadenesheli, Anima Anandkumar, Laurent Itti, Joelle Pineau, Tommaso Furlanello

In this paper, we propose an algorithm to approximate causal states, which are the coarsest partition of the joint history of actions and observations in partially-observable Markov decision processes (POMDP).

Causal Inference

Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning

1 code implementation NeurIPS 2019 Mahmoud Assran, Joshua Romoff, Nicolas Ballas, Joelle Pineau, Michael Rabbat

We show that we can run several loosely coupled GALA agents in parallel on a single GPU and achieve significantly higher hardware utilization and frame-rates than vanilla A2C at comparable power draws.

reinforcement-learning

Recurrent Value Functions

no code implementations23 May 2019 Pierre Thodoroff, Nishanth Anand, Lucas Caccia, Doina Precup, Joelle Pineau

Despite recent successes in Reinforcement Learning, value-based methods often suffer from high variance hindering performance.

Continuous Control reinforcement-learning

Leveraging exploration in off-policy algorithms via normalizing flows

1 code implementation16 May 2019 Bogdan Mazoure, Thang Doan, Audrey Durand, R. Devon Hjelm, Joelle Pineau

The ability to discover approximately optimal policies in domains with sparse rewards is crucial to applying reinforcement learning (RL) in many real-world scenarios.

Continuous Control

Separating value functions across time-scales

1 code implementation5 Feb 2019 Joshua Romoff, Peter Henderson, Ahmed Touati, Emma Brunskill, Joelle Pineau, Yann Ollivier

In settings where this bias is unacceptable - where the system must optimize for longer horizons at higher discounts - the target of the value function approximator may increase in variance leading to difficulties in learning.

Deep Generative Modeling of LiDAR Data

1 code implementation4 Dec 2018 Lucas Caccia, Herke van Hoof, Aaron Courville, Joelle Pineau

In this work, we show that one can adapt deep generative models for this task by unravelling lidar scans into a 2D point map.

Point Cloud Generation

Natural Environment Benchmarks for Reinforcement Learning

1 code implementation14 Nov 2018 Amy Zhang, Yuxin Wu, Joelle Pineau

While current benchmark reinforcement learning (RL) tasks have been useful to drive progress in the field, they are in many ways poor substitutes for learning with real-world data.

reinforcement-learning

Compositional Language Understanding with Text-based Relational Reasoning

2 code implementations7 Nov 2018 Koustuv Sinha, Shagun Sodhani, William L. Hamilton, Joelle Pineau

Neural networks for natural language reasoning have largely focused on extractive, fact-based question-answering (QA) and common-sense inference.

Common Sense Reasoning Inductive Bias +3

Language GANs Falling Short

1 code implementation ICLR 2020 Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, Laurent Charlin

Generating high-quality text with sufficient diversity is essential for a wide range of Natural Language Generation (NLG) tasks.

Text Generation

Adversarial Gain

no code implementations4 Nov 2018 Peter Henderson, Koustuv Sinha, Rosemary Nan Ke, Joelle Pineau

Adversarial examples can be defined as inputs to a model which induce a mistake - where the model output is different than that of an oracle, perhaps in surprising or malicious ways.

Computer Vision General Classification +1

Temporal Regularization in Markov Decision Process

2 code implementations1 Nov 2018 Pierre Thodoroff, Audrey Durand, Joelle Pineau, Doina Precup

Several applications of Reinforcement Learning suffer from instability due to high variance.

Atari Games reinforcement-learning

TarMAC: Targeted Multi-Agent Communication

no code implementations ICLR 2019 Abhishek Das, Théophile Gervet, Joshua Romoff, Dhruv Batra, Devi Parikh, Michael Rabbat, Joelle Pineau

We propose a targeted communication architecture for multi-agent reinforcement learning, where agents learn both what messages to send and whom to address them to while performing cooperative tasks in partially-observable environments.

Multi-agent Reinforcement Learning

Where Did My Optimum Go?: An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methods

1 code implementation5 Oct 2018 Peter Henderson, Joshua Romoff, Joelle Pineau

We find that adaptive optimizers have a narrow window of effective learning rates, diverging in other cases, and that the effectiveness of momentum varies depending on the properties of the environment.

Continuous Control Policy Gradient Methods

Extending Neural Generative Conversational Model using External Knowledge Sources

no code implementations EMNLP 2018 Prasanna Parthasarathi, Joelle Pineau

The use of connectionist approaches in conversational agents has been progressing rapidly due to the availability of large corpora.

Sequential Coordination of Deep Models for Learning Visual Arithmetic

no code implementations ICLR 2018 Eric Crawford, Guillaume Rabusseau, Joelle Pineau

Achieving machine intelligence requires a smooth integration of perception and reasoning, yet models developed to date tend to specialize in one or the other; sophisticated manipulation of symbols acquired from rich perceptual spaces has so far proved elusive.

Combined Reinforcement Learning via Abstract Representations

1 code implementation12 Sep 2018 Vincent François-Lavet, Yoshua Bengio, Doina Precup, Joelle Pineau

In the quest for efficient and robust reinforcement learning methods, both model-free and model-based approaches offer advantages.

reinforcement-learning Transfer Learning

On-line Adaptative Curriculum Learning for GANs

3 code implementations31 Jul 2018 Thang Doan, Joao Monteiro, Isabela Albuquerque, Bogdan Mazoure, Audrey Durand, Joelle Pineau, R. Devon Hjelm

We argue that less expressive discriminators are smoother and have a general coarse grained view of the modes map, which enforces the generator to cover a wide portion of the data distribution support.

Multi-Armed Bandits Stochastic Optimization

Randomized Value Functions via Multiplicative Normalizing Flows

1 code implementation6 Jun 2018 Ahmed Touati, Harsh Satija, Joshua Romoff, Joelle Pineau, Pascal Vincent

In particular, we augment DQN and DDPG with multiplicative normalizing flows in order to track a rich approximate posterior distribution over the parameters of the value function.

Efficient Exploration

Decoupling Dynamics and Reward for Transfer Learning

1 code implementation27 Apr 2018 Amy Zhang, Harsh Satija, Joelle Pineau

Current reinforcement learning (RL) methods can successfully learn single tasks but often generalize poorly to modest perturbations in task domain or training procedure.

reinforcement-learning Transfer Learning

An inference-based policy gradient method for learning options

no code implementations ICLR 2018 Matthew J. A. Smith, Herke van Hoof, Joelle Pineau

In this work we develop a novel policy gradient method for the automatic learning of policies with options.

Multitask Spectral Learning of Weighted Automata

no code implementations NeurIPS 2017 Guillaume Rabusseau, Borja Balle, Joelle Pineau

We first present a natural notion of relatedness between WFAs by considering to which extent several WFAs can share a common underlying representation.

Ethical Challenges in Data-Driven Dialogue Systems

1 code implementation24 Nov 2017 Peter Henderson, Koustuv Sinha, Nicolas Angelard-Gontier, Nan Rosemary Ke, Genevieve Fried, Ryan Lowe, Joelle Pineau

The use of dialogue systems as a medium for human-machine interaction is an increasingly prevalent paradigm.

reinforcement-learning

ACtuAL: Actor-Critic Under Adversarial Learning

no code implementations13 Nov 2017 Anirudh Goyal, Nan Rosemary Ke, Alex Lamb, R. Devon Hjelm, Chris Pal, Joelle Pineau, Yoshua Bengio

This makes it fundamentally difficult to train GANs with discrete data, as generation in this case typically involves a non-differentiable function.

Language Modelling

On overfitting and asymptotic bias in batch reinforcement learning with partial observability

no code implementations22 Sep 2017 Vincent Francois-Lavet, Guillaume Rabusseau, Joelle Pineau, Damien Ernst, Raphael Fonteneau

This paper provides an analysis of the tradeoff between asymptotic bias (suboptimality with unlimited data) and overfitting (additional suboptimality due to limited data) in the context of reinforcement learning with partial observability.

reinforcement-learning

Deep Reinforcement Learning that Matters

5 code implementations19 Sep 2017 Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, David Meger

In recent years, significant progress has been made in solving challenging problems across various domains using deep reinforcement learning (RL).

reinforcement-learning

Independently Controllable Factors

no code implementations3 Aug 2017 Valentin Thomas, Jules Pondard, Emmanuel Bengio, Marc Sarfati, Philippe Beaudoin, Marie-Jean Meurs, Joelle Pineau, Doina Precup, Yoshua Bengio

It has been postulated that a good representation is one that disentangles the underlying explanatory factors of variation.

Predicting Success in Goal-Driven Human-Human Dialogues

no code implementations WS 2017 Michael Noseworthy, Jackie Chi Kit Cheung, Joelle Pineau

We then propose a turn-based hierarchical neural network model that can be used to predict success without requiring a structured goal definition.

MACA: A Modular Architecture for Conversational Agents

1 code implementation WS 2017 Hoai Phuoc Truong, Prasanna Parthasarathi, Joelle Pineau

We propose a software architecture designed to ease the implementation of dialogue systems.

Independently Controllable Features

no code implementations22 Mar 2017 Emmanuel Bengio, Valentin Thomas, Joelle Pineau, Doina Precup, Yoshua Bengio

Finding features that disentangle the different causes of variation in real data is a difficult task, that has nonetheless received considerable attention in static domains like natural images.

Training End-to-End Dialogue Systems with the Ubuntu Dialogue Corpus

no code implementations1 Jan 2017 Ryan Lowe, Nissan Pow, Iulian Vlad Serban, Laurent Charlin, Chia-Wei Liu, Joelle Pineau

In this paper, we analyze neural network-based dialogue systems trained in an end-to-end manner using an updated version of the recent Ubuntu Dialogue Corpus, a dataset containing almost 1 million multi-turn dialogues, with a total of over 7 million utterances and 100 million words.

Conversation Disentanglement Feature Engineering

Piecewise Latent Variables for Neural Variational Text Processing

2 code implementations EMNLP (ACL) 2017 Iulian V. Serban, Alexander G. Ororbia II, Joelle Pineau, Aaron Courville

Advances in neural variational inference have facilitated the learning of powerful directed graphical models with continuous latent variables, such as variational autoencoders.

Natural Language Processing Text Generation +1

Generative Deep Neural Networks for Dialogue: A Short Review

no code implementations18 Nov 2016 Iulian Vlad Serban, Ryan Lowe, Laurent Charlin, Joelle Pineau

Researchers have recently started investigating deep neural networks for dialogue applications.

Response Generation

Bayesian Reinforcement Learning: A Survey

no code implementations14 Sep 2016 Mohammad Ghavamzadeh, Shie Mannor, Joelle Pineau, Aviv Tamar

The objective of the paper is to provide a comprehensive survey on Bayesian RL algorithms and their theoretical and empirical properties.

Bayesian Inference reinforcement-learning

Learning Robust Features using Deep Learning for Automatic Seizure Detection

1 code implementation31 Jul 2016 Pierre Thodoroff, Joelle Pineau, Andrew Lim

We present and evaluate the capacity of a deep neural network to learn robust features from EEG to automatically detect seizures.

EEG Seizure Detection

A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues

9 code implementations19 May 2016 Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron Courville, Yoshua Bengio

Sequential data often possesses a hierarchical structure with complex dependencies between subsequences, such as found between the utterances in a dialogue.

Response Generation

On the Evaluation of Dialogue Systems with Next Utterance Classification

no code implementations WS 2016 Ryan Lowe, Iulian V. Serban, Mike Noseworthy, Laurent Charlin, Joelle Pineau

An open challenge in constructing dialogue systems is developing methods for automatically learning dialogue strategies from large amounts of unlabelled data.

Classification General Classification

A Survey of Available Corpora for Building Data-Driven Dialogue Systems

4 code implementations17 Dec 2015 Iulian Vlad Serban, Ryan Lowe, Peter Henderson, Laurent Charlin, Joelle Pineau

During the past decade, several areas of speech and language understanding have witnessed substantial breakthroughs from the use of data-driven models.

Transfer Learning

Conditional Computation in Neural Networks for faster models

1 code implementation19 Nov 2015 Emmanuel Bengio, Pierre-Luc Bacon, Joelle Pineau, Doina Precup

In this paper, we use reinforcement learning as a tool to optimize conditional computation policies.

reinforcement-learning

Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

7 code implementations17 Jul 2015 Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, Joelle Pineau

We investigate the task of building open domain, conversational dialogue systems based on large dialogue corpora using generative models.

Word Embeddings

Practical Kernel-Based Reinforcement Learning

no code implementations21 Jul 2014 André M. S. Barreto, Doina Precup, Joelle Pineau

In this paper we introduce an algorithm that turns KBRL into a practical reinforcement learning tool.

reinforcement-learning

Representation as a Service

no code implementations24 Feb 2014 Ouais Alsharif, Philip Bachman, Joelle Pineau

Consider a Machine Learning Service Provider (MLSP) designed to rapidly create highly accurate learners for a never-ending stream of new tasks.

Non-Deterministic Policies in Markovian Decision Processes

no code implementations16 Jan 2014 Mahdi Milani Fard, Joelle Pineau

Although conventional methods in reinforcement learning have proved to be useful in problems concerning sequential decision-making, they cannot be applied in their current form to decision support systems, such as those in medical domains, as they suggest policies that are often highly prescriptive and leave little room for the users input.

Decision Making reinforcement-learning

Online Planning Algorithms for POMDPs

no code implementations15 Jan 2014 Stéphane Ross, Joelle Pineau, Sébastien Paquet, Brahim Chaib-Draa

Partially Observable Markov Decision Processes (POMDPs) provide a rich framework for sequential decision-making under uncertainty in stochastic domains.

Decision Making Decision Making Under Uncertainty

Online Ensemble Learning for Imbalanced Data Streams

no code implementations30 Oct 2013 Boyu Wang, Joelle Pineau

While both cost-sensitive learning and online learning have been studied extensively, the effort in simultaneously dealing with these two issues is limited.

Ensemble Learning online learning

End-to-End Text Recognition with Hybrid HMM Maxout Models

no code implementations7 Oct 2013 Ouais Alsharif, Joelle Pineau

The problem of detecting and recognizing text in natural scenes has proved to be more challenging than its counterpart in documents, with most of the previous work focusing on a single part of the problem.

On-line Reinforcement Learning Using Incremental Kernel-Based Stochastic Factorization

no code implementations NeurIPS 2012 Doina Precup, Joelle Pineau, Andre S. Barreto

The ability to learn a policy for a sequential decision problem with continuous state space using on-line data is a long-standing challenge.

reinforcement-learning

PAC-Bayesian Policy Evaluation for Reinforcement Learning

no code implementations14 Feb 2012 Mahdi Milani Fard, Joelle Pineau, Csaba Szepesvari

PAC-Bayesian methods overcome this problem by providing bounds that hold regardless of the correctness of the prior distribution.

Model Selection reinforcement-learning +1

Reinforcement Learning using Kernel-Based Stochastic Factorization

no code implementations NeurIPS 2011 Andre S. Barreto, Doina Precup, Joelle Pineau

Kernel-based reinforcement-learning (KBRL) is a method for learning a decision policy from a set of sample transitions which stands out for its strong theoretical guarantees.

reinforcement-learning

PAC-Bayesian Model Selection for Reinforcement Learning

no code implementations NeurIPS 2010 Mahdi M. Fard, Joelle Pineau

This paper introduces the first set of PAC-Bayesian bounds for the batch reinforcement learning problem in finite state spaces.

Model Selection reinforcement-learning

Manifold Embeddings for Model-Based Reinforcement Learning under Partial Observability

no code implementations NeurIPS 2009 Keith Bush, Joelle Pineau

Interesting real-world datasets often exhibit nonlinear, noisy, continuous-valued states that are unexplorable, are poorly described by first principles, and are only partially observable.

Model-based Reinforcement Learning reinforcement-learning

MDPs with Non-Deterministic Policies

no code implementations NeurIPS 2008 Mahdi M. Fard, Joelle Pineau

Markov Decision Processes (MDPs) have been extensively studied and used in the context of planning and decision-making, and many methods exist to find the optimal policy for problems modelled as MDPs.

Decision Making

Theoretical Analysis of Heuristic Search Methods for Online POMDPs

no code implementations NeurIPS 2007 Stephane Ross, Joelle Pineau, Brahim Chaib-Draa

The algorithm uses search heuristics based on an error analysis of lookahead search, to guide the online search towards reachable beliefs with the most potential to reduce error.

Cannot find the paper you are looking for? You can Submit a new open access paper.