Search Results for author: Sarath Chandar

Found 79 papers, 42 papers with code

Learning to Navigate in Synthetically Accessible Chemical Space Using Reinforcement Learning

1 code implementation ICML 2020 Sai Krishna Gottipati, Boris Sattarov, Sufeng. Niu, Hao-Ran Wei, Yashaswi Pathak, Shengchao Liu, Simon Blackburn, Karam Thomas, Connor Coley, Jian Tang, Sarath Chandar, Yoshua Bengio

In this work, we propose a novel reinforcement learning (RL) setup for drug discovery that addresses this challenge by embedding the concept of synthetic accessibility directly into the de novo compound design system.

Drug Discovery Navigate +3

Do Encoder Representations of Generative Dialogue Models have sufficient summary of the Information about the task ?

1 code implementation SIGDIAL (ACL) 2021 Prasanna Parthasarathi, Joelle Pineau, Sarath Chandar

Predicting the next utterance in dialogue is contingent on encoding of users’ input text to generate appropriate and relevant response in data-driven approaches.

Text Generation

Exploring Quantization for Efficient Pre-Training of Transformer Language Models

1 code implementation16 Jul 2024 Kamran Chitsaz, Quentin Fournier, Gonçalo Mordido, Sarath Chandar

While quantization has proven to be effective after pre-training and during fine-tuning, applying quantization in Transformers during pre-training has remained largely unexplored at scale for language modeling.

Language Modelling Quantization

Why Don't Prompt-Based Fairness Metrics Correlate?

1 code implementation9 Jun 2024 Abdelrahman Zayed, Goncalo Mordido, Ioana Baldini, Sarath Chandar

CAIRO augments the original prompts of a given fairness metric by using several pre-trained language models and then selects the combination of the augmented prompts that achieves the highest correlation across metrics.

Fairness

A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques

no code implementations7 Jun 2024 Megh Thakkar, Quentin Fournier, Matthew D Riemer, Pin-Yu Chen, Amal Zouaq, Payel Das, Sarath Chandar

Large language models are first pre-trained on trillions of tokens and then instruction-tuned or aligned to specific preferences.

BindGPT: A Scalable Framework for 3D Molecular Design via Language Modeling and Reinforcement Learning

no code implementations6 Jun 2024 Artem Zholus, Maksim Kuznetsov, Roman Schutski, Rim Shayakhmetov, Daniil Polykovskiy, Sarath Chandar, Alex Zhavoronkov

Generating novel active molecules for a given protein is an extremely challenging task for generative models that requires an understanding of the complex physical interactions between the molecule and its environment.

Graph Reconstruction Language Modelling

Interpretability Needs a New Paradigm

no code implementations8 May 2024 Andreas Madsen, Himabindu Lakkaraju, Siva Reddy, Sarath Chandar

At present, interpretability is divided into two paradigms: the intrinsic paradigm, which believes that only models designed to be explained can be explained, and the post-hoc paradigm, which believes that black-box models can be explained.

Sub-goal Distillation: A Method to Improve Small Language Agents

1 code implementation4 May 2024 Maryam Hashemzadeh, Elias Stengel-Eskin, Sarath Chandar, Marc-Alexandre Cote

While Large Language Models (LLMs) have demonstrated significant promise as agents in interactive tasks, their substantial computational requirements and restricted number of calls constrain their practical utility, especially in long-horizon interactive tasks such as decision-making or in scenarios involving continuous ongoing tasks.

Imitation Learning Knowledge Distillation +1

Intelligent Switching for Reset-Free RL

1 code implementation2 May 2024 Darshan Patil, Janarthanan Rajendran, Glen Berseth, Sarath Chandar

In the real world, the strong episode resetting mechanisms that are needed to train agents in simulation are unavailable.

Mastering Memory Tasks with World Models

1 code implementation7 Mar 2024 Mohammad Reza Samsami, Artem Zholus, Janarthanan Rajendran, Sarath Chandar

Through a diverse set of illustrative tasks, we systematically demonstrate that R2I not only establishes a new state-of-the-art for challenging memory and credit assignment RL tasks, such as BSuite and POPGym, but also showcases superhuman performance in the complex memory domain of Memory Maze.

Model-based Reinforcement Learning

Are self-explanations from Large Language Models faithful?

1 code implementation15 Jan 2024 Andreas Madsen, Sarath Chandar, Siva Reddy

For example, if an LLM says a set of words is important for making a prediction, then it should not be able to make its prediction without these words.

counterfactual Faithfulness Critic +4

Fairness-Aware Structured Pruning in Transformers

1 code implementation24 Dec 2023 Abdelrahman Zayed, Goncalo Mordido, Samira Shabanian, Ioana Baldini, Sarath Chandar

The increasing size of large language models (LLMs) has introduced challenges in their training and inference.

Fairness Language Modelling

Language Model-In-The-Loop: Data Optimal Approach to Learn-To-Recommend Actions in Text Games

no code implementations13 Nov 2023 Arjun Vaithilingam Sudhakar, Prasanna Parthasarathi, Janarthanan Rajendran, Sarath Chandar

In this work, we explore and evaluate updating LLM used for candidate recommendation during the learning of the text based game as well to mitigate the reliance on the human annotated gameplays, which are costly to acquire.

Language Modelling text-based games

Self-Influence Guided Data Reweighting for Language Model Pre-training

no code implementations2 Nov 2023 Megh Thakkar, Tolga Bolukbasi, Sriram Ganapathy, Shikhar Vashishth, Sarath Chandar, Partha Talukdar

Once the pre-training corpus has been assembled, all data samples in the corpus are treated with equal importance during LM pre-training.

Language Modelling

EpiK-Eval: Evaluation for Language Models as Epistemic Models

1 code implementation23 Oct 2023 Gabriele Prato, Jerry Huang, Prasannna Parthasarathi, Shagun Sodhani, Sarath Chandar

In the age of artificial intelligence, the role of large language models (LLMs) is becoming increasingly central.

Question Answering

Faithfulness Measurable Masked Language Models

1 code implementation11 Oct 2023 Andreas Madsen, Siva Reddy, Sarath Chandar

Additionally, because the model makes faithfulness cheap to measure, we can optimize explanations towards maximal faithfulness; thus, our model becomes indirectly inherently explainable.

Towards Few-shot Coordination: Revisiting Ad-hoc Teamplay Challenge In the Game of Hanabi

1 code implementation20 Aug 2023 Hadi Nekoei, Xutong Zhao, Janarthanan Rajendran, Miao Liu, Sarath Chandar

In this work, we show empirically that state-of-the-art ZSC algorithms have poor performance when paired with agents trained with different learning methods, and they require millions of interaction samples to adapt to these new partners.

Game of Hanabi Multi-agent Reinforcement Learning +1

Lookbehind-SAM: k steps back, 1 step forward

1 code implementation31 Jul 2023 Gonçalo Mordido, Pranshu Malviya, Aristide Baratin, Sarath Chandar

In this work, we increase the efficiency of the maximization and minimization parts of SAM's objective to achieve a better loss-sharpness trade-off.

Promoting Exploration in Memory-Augmented Adam using Critical Momenta

1 code implementation18 Jul 2023 Pranshu Malviya, Gonçalo Mordido, Aristide Baratin, Reza Babanezhad Harikandeh, Jerry Huang, Simon Lacoste-Julien, Razvan Pascanu, Sarath Chandar

To address this, we propose a new memory-augmented version of Adam that encourages exploration towards flatter minima by incorporating a buffer of critical momentum terms during training.

Image Classification Language Modelling

Thompson sampling for improved exploration in GFlowNets

no code implementations30 Jun 2023 Jarrid Rector-Brooks, Kanika Madan, Moksh Jain, Maksym Korablyov, Cheng-Hao Liu, Sarath Chandar, Nikolay Malkin, Yoshua Bengio

Generative flow networks (GFlowNets) are amortized variational inference algorithms that treat sampling from a distribution over compositional objects as a sequential decision-making problem with a learnable action policy.

Active Learning Decision Making +3

Should We Attend More or Less? Modulating Attention for Fairness

no code implementations22 May 2023 Abdelrahman Zayed, Goncalo Mordido, Samira Shabanian, Sarath Chandar

In this work, we investigate the role of attention, a widely-used technique in current state-of-the-art NLP models, in the propagation of social biases.

Fairness text-classification +1

Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness

1 code implementation20 Nov 2022 Abdelrahman Zayed, Prasanna Parthasarathi, Goncalo Mordido, Hamid Palangi, Samira Shabanian, Sarath Chandar

The fairness achieved by our method surpasses that of data augmentation on three text classification datasets, using no more than half of the examples in the augmented dataset.

counterfactual Data Augmentation +3

SAMSON: Sharpness-Aware Minimization Scaled by Outlier Normalization for Improving DNN Generalization and Robustness

no code implementations18 Nov 2022 Gonçalo Mordido, Sébastien Henwood, Sarath Chandar, François Leduc-Primeau

In this work, we show that applying sharpness-aware training, by optimizing for both the loss value and loss sharpness, significantly improves robustness to noisy hardware at inference time without relying on any assumptions about the target hardware.

PatchBlender: A Motion Prior for Video Transformers

no code implementations11 Nov 2022 Gabriele Prato, Yale Song, Janarthanan Rajendran, R Devon Hjelm, Neel Joshi, Sarath Chandar

We show that our method is successful at enabling vision transformers to encode the temporal component of video data.

Local Structure Matters Most in Most Languages

no code implementations9 Nov 2022 Louis Clouâtre, Prasanna Parthasarathi, Amal Zouaq, Sarath Chandar

In this work, we replicate a study on the importance of local structure, and the relative unimportance of global structure, in a multilingual setting.

Natural Language Understanding

Improving Meta-Learning Generalization with Activation-Based Early-Stopping

1 code implementation3 Aug 2022 Simon Guiroy, Christopher Pal, Gonçalo Mordido, Sarath Chandar

Specifically, we analyze the evolution, during meta-training, of the neural activations at each hidden layer, on a small set of unlabelled support examples from a single task of the target tasks distribution, as this constitutes a minimal and justifiably accessible information from the target problem.

Few-Shot Learning Transfer Learning

An Introduction to Lifelong Supervised Learning

no code implementations10 Jul 2022 Shagun Sodhani, Mojtaba Faramarzi, Sanket Vaibhav Mehta, Pranshu Malviya, Mohamed Abdelsalam, Janarthanan Janarthanan, Sarath Chandar

Following these different classes of learning algorithms, we discuss the commonly used evaluation benchmarks and metrics for lifelong learning (Chapter 6) and wrap up with a discussion of future challenges and important research directions in Chapter 7.

Towards Evaluating Adaptivity of Model-Based Reinforcement Learning Methods

1 code implementation25 Apr 2022 Yi Wan, Ali Rahimi-Kalahroudi, Janarthanan Rajendran, Ida Momennejad, Sarath Chandar, Harm van Seijen

We empirically validate these insights in the case of linear function approximation by demonstrating that a modified version of linear Dyna achieves effective adaptation to local changes.

Model-based Reinforcement Learning reinforcement-learning +1

Improving Sample Efficiency of Value Based Models Using Attention and Vision Transformers

no code implementations1 Feb 2022 Amir Ardalan Kalantari, Mohammad Amini, Sarath Chandar, Doina Precup

Much of recent Deep Reinforcement Learning success is owed to the neural architecture's potential to learn and use effective internal representations of the world.

reinforcement-learning Reinforcement Learning (RL)

An Empirical Investigation of the Role of Pre-training in Lifelong Learning

1 code implementation NeurIPS 2023 Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, Emma Strubell

The lifelong learning paradigm in machine learning is an attractive alternative to the more prominent isolated learning scheme not only due to its resemblance to biological learning but also its potential to reduce energy waste by obviating excessive model re-training.

Continual Learning Image Classification

Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifiers

no code implementations13 Oct 2021 Gabriele Prato, Simon Guiroy, Ethan Caballero, Irina Rish, Sarath Chandar

Empirical science of neural scaling laws is a rapidly growing area of significant importance to the future of machine learning, particularly in the light of recent breakthroughs achieved by large-scale pre-trained models such as GPT-3, CLIP and DALL-e.

Few-Shot Learning Image Classification

Early-Stopping for Meta-Learning: Estimating Generalization from the Activation Dynamics

no code implementations29 Sep 2021 Simon Guiroy, Christopher Pal, Sarath Chandar

To this end, we empirically show that as meta-training progresses, a model's generalization to a target distribution of novel tasks can be estimated by analysing the dynamics of its neural activations.

Few-Shot Learning Transfer Learning

Post-hoc Interpretability for Neural NLP: A Survey

no code implementations10 Aug 2021 Andreas Madsen, Siva Reddy, Sarath Chandar

Neural networks for NLP are becoming increasingly complex and widespread, and there is a growing concern if these models are responsible to use.

Local Structure Matters Most: Perturbation Study in NLU

no code implementations Findings (ACL) 2022 Louis Clouatre, Prasanna Parthasarathi, Amal Zouaq, Sarath Chandar

Recent research analyzing the sensitivity of natural language understanding models to word-order perturbations has shown that neural models are surprisingly insensitive to the order of words.

Natural Language Understanding Position

Do Encoder Representations of Generative Dialogue Models Encode Sufficient Information about the Task ?

1 code implementation20 Jun 2021 Prasanna Parthasarathi, Joelle Pineau, Sarath Chandar

Predicting the next utterance in dialogue is contingent on encoding of users' input text to generate appropriate and relevant response in data-driven approaches.

Text Generation

Memory Augmented Optimizers for Deep Learning

2 code implementations ICLR 2022 Paul-Aymeric McRae, Prasanna Parthasarathi, Mahmoud Assran, Sarath Chandar

Popular approaches for minimizing loss in data-driven learning often involve an abstraction or an explicit retention of the history of gradients for efficient parameter updates.

A Brief Study on the Effects of Training Generative Dialogue Models with a Semantic loss

1 code implementation SIGDIAL (ACL) 2021 Prasanna Parthasarathi, Mohamed Abdelsalam, Joelle Pineau, Sarath Chandar

Neural models trained for next utterance generation in dialogue task learn to mimic the n-gram sequences in the training set with training objectives like negative log-likelihood (NLL) or cross-entropy.

Diversity Language Modelling +4

TAG: Task-based Accumulated Gradients for Lifelong learning

1 code implementation11 May 2021 Pranshu Malviya, Balaraman Ravindran, Sarath Chandar

We also show that our method performs better than several state-of-the-art methods in lifelong learning on complex datasets with a large number of tasks.

Continual Learning

A Survey of Data Augmentation Approaches for NLP

1 code implementation Findings (ACL) 2021 Steven Y. Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, Eduard Hovy

In this paper, we present a comprehensive and unifying survey of data augmentation for NLP by summarizing the literature in a structured manner.

Data Augmentation

Out-of-Distribution Classification and Clustering

no code implementations1 Jan 2021 Gabriele Prato, Sarath Chandar

This includes left out classes from the same dataset, as well as entire datasets never trained on.

Classification Clustering +2

Maximum Reward Formulation In Reinforcement Learning

1 code implementation8 Oct 2020 Sai Krishna Gottipati, Yashaswi Pathak, Rohan Nuttall, Sahir, Raviteja Chunduru, Ahmed Touati, Sriram Ganapathi Subramanian, Matthew E. Taylor, Sarath Chandar

Reinforcement learning (RL) algorithms typically deal with maximizing the expected cumulative return (discounted or undiscounted, finite or infinite horizon).

Drug Discovery reinforcement-learning +1

Slot Contrastive Networks: A Contrastive Approach for Representing Objects

no code implementations18 Jul 2020 Evan Racah, Sarath Chandar

Unsupervised extraction of objects from low-level visual data is an important goal for further progress in machine learning.

Atari Games

The LoCA Regret: A Consistent Metric to Evaluate Model-Based Behavior in Reinforcement Learning

2 code implementations NeurIPS 2020 Harm van Seijen, Hadi Nekoei, Evan Racah, Sarath Chandar

For example, the common single-task sample-efficiency metric conflates improvements due to model-based learning with various other aspects, such as representation learning, making it difficult to assess true progress on model-based RL.

Model-based Reinforcement Learning Reinforcement Learning (RL) +1

Chaotic Continual Learning

no code implementations ICML Workshop LifelongML 2020 Touraj Laleh, Mojtaba Faramarzi, Irina Rish, Sarath Chandar

Most proposed approaches for this issue try to compensate for the effects of parameter updates in the batch incremental setup in which the training model visits a lot of samples for several epochs.

Continual Learning

PatchUp: A Feature-Space Block-Level Regularization Technique for Convolutional Neural Networks

1 code implementation14 Jun 2020 Mojtaba Faramarzi, Mohammad Amini, Akilesh Badrinaaraayanan, Vikas Verma, Sarath Chandar

Our approach improves the robustness of CNN models against the manifold intrusion problem that may occur in other state-of-the-art mixing approaches.

Towards Lossless Encoding of Sentences

1 code implementation ACL 2019 Gabriele Prato, Mathieu Duchesneau, Sarath Chandar, Alain Tapp

A lot of work has been done in the field of image compression via machine learning, but not much attention has been given to the compression of natural language.

Image Compression Sentence +2

Structure Learning for Neural Module Networks

no code implementations WS 2019 Vardaan Pahuja, Jie Fu, Sarath Chandar, Christopher J. Pal

In current formulations of such networks only the parameters of the neural modules and/or the order of their execution is learned.

Question Answering Visual Question Answering

Environments for Lifelong Reinforcement Learning

2 code implementations26 Nov 2018 Khimya Khetarpal, Shagun Sodhani, Sarath Chandar, Doina Precup

To achieve general artificial intelligence, reinforcement learning (RL) agents should learn not only to optimize returns for one specific task but also to constantly build more complex skills and scaffold their knowledge about the world, without forgetting what has already been learned.

reinforcement-learning Reinforcement Learning (RL)

Towards Training Recurrent Neural Networks for Lifelong Learning

no code implementations16 Nov 2018 Shagun Sodhani, Sarath Chandar, Yoshua Bengio

Both these models are proposed in the context of feedforward networks and we evaluate the feasibility of using them for recurrent networks.

Language Expansion In Text-Based Games

no code implementations17 May 2018 Ghulam Ahmed Ansari, Sagar J P, Sarath Chandar, Balaraman Ravindran

Text-based games are suitable test-beds for designing agents that can learn by interaction with the environment in the form of natural language text.

reinforcement-learning Reinforcement Learning (RL) +1

Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph

1 code implementation31 Jan 2018 Amrita Saha, Vardaan Pahuja, Mitesh M. Khapra, Karthik Sankaranarayanan, Sarath Chandar

Further, unlike existing large scale QA datasets which contain simple questions that can be answered from a single tuple, the questions in our dialogs require a larger subgraph of the KG.

Knowledge Graphs Question Answering

Memory Augmented Neural Networks with Wormhole Connections

no code implementations30 Jan 2017 Caglar Gulcehre, Sarath Chandar, Yoshua Bengio

We use discrete addressing for read/write operations which helps to substantially to reduce the vanishing gradient problem with very long sequences.

GuessWhat?! Visual object discovery through multi-modal dialogue

4 code implementations CVPR 2017 Harm de Vries, Florian Strub, Sarath Chandar, Olivier Pietquin, Hugo Larochelle, Aaron Courville

Our key contribution is the collection of a large-scale dataset consisting of 150K human-played games with a total of 800K visual question-answer pairs on 66K images.

Object Object Discovery

Dynamic Neural Turing Machine with Soft and Hard Addressing Schemes

no code implementations30 Jun 2016 Caglar Gulcehre, Sarath Chandar, Kyunghyun Cho, Yoshua Bengio

We investigate the mechanisms and effects of learning to read and write into a memory through experiments on Facebook bAbI tasks using both a feedforward and GRUcontroller.

Natural Language Inference Question Answering

A Correlational Encoder Decoder Architecture for Pivot Based Sequence Generation

no code implementations COLING 2016 Amrita Saha, Mitesh M. Khapra, Sarath Chandar, Janarthanan Rajendran, Kyunghyun Cho

However, there is no parallel training data available between X and Y but, training data is available between X & Z and Z & Y (as is often the case in many real world applications).

Decoder Transliteration

Hierarchical Memory Networks

no code implementations24 May 2016 Sarath Chandar, Sungjin Ahn, Hugo Larochelle, Pascal Vincent, Gerald Tesauro, Yoshua Bengio

In this paper, we explore a form of hierarchical memory network, which can be considered as a hybrid between hard and soft attention memory networks.

Hard Attention Question Answering

Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning

1 code implementation NAACL 2016 Janarthanan Rajendran, Mitesh M. Khapra, Sarath Chandar, Balaraman Ravindran

In this work, we address a real-world scenario where no direct parallel data is available between two views of interest (say, $V_1$ and $V_2$) but parallel data is available between each of these views and a pivot view ($V_3$).

Document Classification Representation Learning +2

TSEB: More Efficient Thompson Sampling for Policy Learning

no code implementations10 Oct 2015 P. Prasanna, Sarath Chandar, Balaraman Ravindran

In this paper, we propose TSEB, a Thompson Sampling based algorithm with adaptive exploration bonus that aims to solve the problem with tighter PAC guarantees, while being cautious on the regret as well.

Thompson Sampling

Reasoning about Linguistic Regularities in Word Embeddings using Matrix Manifolds

no code implementations28 Jul 2015 Sridhar Mahadevan, Sarath Chandar

In this paper, we introduce a new approach to capture analogies in continuous word representations, based on modeling not just individual word vectors, but rather the subspaces spanned by groups of words.

Word Embeddings

Clustering is Efficient for Approximate Maximum Inner Product Search

no code implementations21 Jul 2015 Alex Auvolat, Sarath Chandar, Pascal Vincent, Hugo Larochelle, Yoshua Bengio

Efficient Maximum Inner Product Search (MIPS) is an important task that has a wide applicability in recommendation systems and classification with a large number of classes.

Clustering Recommendation Systems +2

Correlational Neural Networks

2 code implementations27 Apr 2015 Sarath Chandar, Mitesh M. Khapra, Hugo Larochelle, Balaraman Ravindran

CCA based approaches learn a joint representation by maximizing correlation of the views when projected to the common subspace.

Representation Learning Transfer Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.