Search Results for author: Shagun Sodhani

Found 30 papers, 16 papers with code

TorchRL: A data-driven decision-making library for PyTorch

2 code implementations1 Jun 2023 Albert Bou, Matteo Bettini, Sebastian Dittert, Vikash Kumar, Shagun Sodhani, Xiaomeng Yang, Gianni de Fabritiis, Vincent Moens

Striking a balance between integration and modularity is crucial for a machine learning library to be versatile and user-friendly, especially in handling decision and control tasks that involve large development teams and complex, real-world data, and environments.

Decision Making Reinforcement Learning (RL)

Sequence Modeling is a Robust Contender for Offline Reinforcement Learning

1 code implementation23 May 2023 Prajjwal Bhargava, Rohan Chitnis, Alborz Geramifard, Shagun Sodhani, Amy Zhang

Our key findings are: (1) Sequence Modeling requires more data than Q-Learning to learn competitive policies but is more robust; (2) Sequence Modeling is a substantially better choice than both Q-Learning and Imitation Learning in sparse-reward and low-quality data settings; and (3) Sequence Modeling and Imitation Learning are preferable as task horizon increases, or when data is obtained from human demonstrators.

D4RL Imitation Learning +5

VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training

1 code implementation30 Sep 2022 Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, Amy Zhang

Given the inherent cost and scarcity of in-domain, task-specific robot data, learning from large, diverse, offline human videos has emerged as a promising path towards acquiring a generally useful visual representation for control; however, how these human videos can be used for general-purpose reward learning remains an open question.

Offline RL Open-Ended Question Answering +2

The Neural Race Reduction: Dynamics of Abstraction in Gated Networks

no code implementations21 Jul 2022 Andrew M. Saxe, Shagun Sodhani, Sam Lewallen

Our theoretical understanding of deep learning has not kept pace with its empirical success.

An Introduction to Lifelong Supervised Learning

no code implementations10 Jul 2022 Shagun Sodhani, Mojtaba Faramarzi, Sanket Vaibhav Mehta, Pranshu Malviya, Mohamed Abdelsalam, Janarthanan Janarthanan, Sarath Chandar

Following these different classes of learning algorithms, we discuss the commonly used evaluation benchmarks and metrics for lifelong learning (Chapter 6) and wrap up with a discussion of future challenges and important research directions in Chapter 7.

Robust Policy Learning over Multiple Uncertainty Sets

no code implementations14 Feb 2022 Annie Xie, Shagun Sodhani, Chelsea Finn, Joelle Pineau, Amy Zhang

Reinforcement learning (RL) agents need to be robust to variations in safety-critical environments.

Reinforcement Learning (RL)

Block Contextual MDPs for Continual Learning

no code implementations13 Oct 2021 Shagun Sodhani, Franziska Meier, Joelle Pineau, Amy Zhang

In this work, we propose to examine this continual reinforcement learning setting through the block contextual MDP (BC-MDP) framework, which enables us to relax the assumption of stationarity.

Continual Learning Generalization Bounds +2

Multi-Task Reinforcement Learning with Context-based Representations

2 code implementations11 Feb 2021 Shagun Sodhani, Amy Zhang, Joelle Pineau

We posit that an efficient approach to knowledge transfer is through the use of multiple context-dependent, composable representations shared across a family of tasks.

Multi-Task Learning reinforcement-learning +1

GraphLog: A Benchmark for Measuring Logical Generalization in Graph Neural Networks

1 code implementation1 Jan 2021 Koustuv Sinha, Shagun Sodhani, Joelle Pineau, William L. Hamilton

In this work, we study the logical generalization capabilities of GNNs by designing a benchmark suite grounded in first-order logic.

Continual Learning Knowledge Graphs +1

A Closer Look at Codistillation for Distributed Training

no code implementations6 Oct 2020 Shagun Sodhani, Olivier Delalleau, Mahmoud Assran, Koustuv Sinha, Nicolas Ballas, Michael Rabbat

Surprisingly, we find that even at moderate batch sizes, models trained with codistillation can perform as well as models trained with synchronous data-parallel methods, despite using a much weaker synchronization mechanism.

Distributed Computing

Learning Robust State Abstractions for Hidden-Parameter Block MDPs

2 code implementations ICLR 2021 Amy Zhang, Shagun Sodhani, Khimya Khetarpal, Joelle Pineau

Further, we provide transfer and generalization bounds based on task and state similarity, along with sample complexity bounds that depend on the aggregate number of samples across tasks, rather than the number of tasks, a significant improvement over prior work that use the same environment assumptions.

Generalization Bounds Meta Reinforcement Learning +2

Evaluating Logical Generalization in Graph Neural Networks

1 code implementation ICML Workshop LifelongML 2020 Koustuv Sinha, Shagun Sodhani, Joelle Pineau, William L. Hamilton

Recent research has highlighted the role of relational inductive biases in building learning agents that can generalize and reason in a compositional manner.

Continual Learning Knowledge Graphs +2

Invariant Causal Prediction for Block MDPs

1 code implementation ICML 2020 Amy Zhang, Clare Lyle, Shagun Sodhani, Angelos Filos, Marta Kwiatkowska, Joelle Pineau, Yarin Gal, Doina Precup

Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges.

Causal Inference Variable Selection

Recurrent Independent Mechanisms

3 code implementations ICLR 2021 Anirudh Goyal, Alex Lamb, Jordan Hoffmann, Shagun Sodhani, Sergey Levine, Yoshua Bengio, Bernhard Schölkopf

Learning modular structures which reflect the dynamics of the environment can lead to better generalization and robustness to changes which only affect a few of the underlying causes.

CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text

5 code implementations IJCNLP 2019 Koustuv Sinha, Shagun Sodhani, Jin Dong, Joelle Pineau, William L. Hamilton

The recent success of natural language understanding (NLU) systems has been troubled by results highlighting the failure of these models to generalize in a systematic and robust way.

Inductive logic programming Natural Language Understanding +2

Learning Powerful Policies by Using Consistent Dynamics Model

1 code implementation11 Jun 2019 Shagun Sodhani, Anirudh Goyal, Tristan Deleu, Yoshua Bengio, Sergey Levine, Jian Tang

There is enough evidence that humans build a model of the environment, not only by observing the environment but also by interacting with the environment.

Atari Games Model-based Reinforcement Learning +1

Environments for Lifelong Reinforcement Learning

2 code implementations26 Nov 2018 Khimya Khetarpal, Shagun Sodhani, Sarath Chandar, Doina Precup

To achieve general artificial intelligence, reinforcement learning (RL) agents should learn not only to optimize returns for one specific task but also to constantly build more complex skills and scaffold their knowledge about the world, without forgetting what has already been learned.

reinforcement-learning Reinforcement Learning (RL)

Towards Training Recurrent Neural Networks for Lifelong Learning

no code implementations16 Nov 2018 Shagun Sodhani, Sarath Chandar, Yoshua Bengio

Both these models are proposed in the context of feedforward networks and we evaluate the feasibility of using them for recurrent networks.

Compositional Language Understanding with Text-based Relational Reasoning

2 code implementations7 Nov 2018 Koustuv Sinha, Shagun Sodhani, William L. Hamilton, Joelle Pineau

Neural networks for natural language reasoning have largely focused on extractive, fact-based question-answering (QA) and common-sense inference.

Common Sense Reasoning Inductive Bias +3

Spatial Co-location Pattern Mining - A new perspective using Graph Database

1 code implementation21 Oct 2018 Sanket Vaibhav Mehta, Shagun Sodhani, Dhaval Patel

Spatial co-location pattern mining refers to the task of discovering the group of objects or events that co-occur at many places.

Databases Distributed, Parallel, and Cluster Computing

Learning powerful policies and better dynamics models by encouraging consistency

no code implementations27 Sep 2018 Shagun Sodhani, Anirudh Goyal, Tristan Deleu, Yoshua Bengio, Jian Tang

Analogously, we would expect such interaction to be helpful for a learning agent while learning to model the environment dynamics.

Model-based Reinforcement Learning

Memory Augmented Self-Play

1 code implementation28 May 2018 Shagun Sodhani, Vardaan Pahuja

Self-play is an unsupervised training procedure which enables the reinforcement learning agents to explore the environment without requiring any external rewards.

reinforcement-learning Reinforcement Learning (RL)

Reproducibility Report for "Learning To Count Objects In Natural Images For Visual Question Answering"

no code implementations21 May 2018 Shagun Sodhani, Vardaan Pahuja

This is the reproducibility report for the paper "Learning To Count Objects In Natural Images For Visual QuestionAnswering"

Question Answering Visual Question Answering

Survey of Recent Advances in Visual Question Answering

no code implementations24 Sep 2017 Supriya Pandhre, Shagun Sodhani

Visual Question Answering (VQA) presents a unique challenge as it requires the ability to understand and encode the multi-modal inputs - in terms of image processing and natural language processing.

Question Answering Visual Question Answering

Improving Search through A3C Reinforcement Learning based Conversational Agent

no code implementations ICLR 2018 Milan Aggarwal, Aarushi Arora, Shagun Sodhani, Balaji Krishnamurthy

We develop a reinforcement learning based search assistant which can assist users through a set of actions and sequence of interactions to enable them realize their intent.

Q-Learning reinforcement-learning +1

Stack Exchange Tagger

no code implementations13 Dec 2015 Sanket Mehta, Shagun Sodhani

The goal of our project is to develop an accurate tagger for questions posted on Stack Exchange.

Classification General Classification

Cannot find the paper you are looking for? You can Submit a new open access paper.