Search Results for author: Rahul Ramesh

Found 16 papers, 8 papers with code

Prospective Learning: Learning for a Dynamic Future

1 code implementation31 Oct 2024 Ashwin De Silva, Rahul Ramesh, Rubing Yang, Siyu Yu, Joshua T Vogelstein, Pratik Chaudhari

We show that standard ERM as done in PAC learning, without incorporating time, can result in failure to learn when distributions are dynamic.

PAC learning

Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing

no code implementations22 Oct 2024 Kento Nishi, Maya Okawa, Rahul Ramesh, Mikail Khona, Hidenori Tanaka, Ekdeep Singh Lubana

We call this phenomenon representation shattering and demonstrate that it results in degradation of factual recall and reasoning performance more broadly.

knowledge editing Mamba

Many Perception Tasks are Highly Redundant Functions of their Input Data

no code implementations18 Jul 2024 Rahul Ramesh, Anthony Bisulco, Ronald W. DiTullio, Linran Wei, Vijay Balasubramanian, Kostas Daniilidis, Pratik Chaudhari

We show that many perception tasks, from visual recognition, semantic segmentation, optical flow, depth estimation to vocalization discrimination, are highly redundant functions of their input data.

Depth Estimation Optical Flow Estimation +1

Towards an Understanding of Stepwise Inference in Transformers: A Synthetic Graph Navigation Model

no code implementations12 Feb 2024 Mikail Khona, Maya Okawa, Jan Hula, Rahul Ramesh, Kento Nishi, Robert Dick, Ekdeep Singh Lubana, Hidenori Tanaka

Stepwise inference protocols, such as scratchpads and chain-of-thought, help language models solve complex problems by decomposing them into a sequence of simpler subproblems.

Diversity

Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks

1 code implementation21 Nov 2023 Rahul Ramesh, Ekdeep Singh Lubana, Mikail Khona, Robert P. Dick, Hidenori Tanaka

Transformers trained on huge text corpora exhibit a remarkable set of capabilities, e. g., performing basic arithmetic.

A picture of the space of typical learnable tasks

2 code implementations31 Oct 2022 Rahul Ramesh, Jialin Mao, Itay Griniasty, Rubing Yang, Han Kheng Teoh, Mark Transtrum, James P. Sethna, Pratik Chaudhari

We develop information geometric techniques to understand the representations learned by deep networks when they are trained on different tasks using supervised, meta-, semi-supervised and contrastive learning.

Contrastive Learning Meta-Learning +1

The Value of Out-of-Distribution Data

1 code implementation23 Aug 2022 Ashwin De Silva, Rahul Ramesh, Carey E. Priebe, Pratik Chaudhari, Joshua T. Vogelstein

In this work, we show a counter-intuitive phenomenon: the generalization error of a task can be a non-monotonic function of the number of OOD samples.

Data Augmentation Hyperparameter Optimization

Deep Reference Priors: What is the best way to pretrain a model?

2 code implementations pproximateinference AABI Symposium 2022 Yansong Gao, Rahul Ramesh, Pratik Chaudhari

Such priors enable the task to maximally affect the Bayesian posterior, e. g., reference priors depend upon the number of samples available for learning the task and for very small sample sizes, the prior puts more probability mass on low-complexity models in the hypothesis space.

image-classification Semi-Supervised Image Classification +1

Model Zoo: A Growing Brain That Learns Continually

no code implementations ICLR 2022 Rahul Ramesh, Pratik Chaudhari

This paper argues that continual learning methods can benefit by splitting the capacity of the learner across multiple models.

Continual Learning Learning Theory +1

Model Zoo: A Growing "Brain" That Learns Continually

2 code implementations6 Jun 2021 Rahul Ramesh, Pratik Chaudhari

We use statistical learning theory and experimental analysis to show how multiple tasks can interact with each other in a non-trivial fashion when a single model is trained on them.

Continual Learning Learning Theory +1

Option Encoder: A Framework for Discovering a Policy Basis in Reinforcement Learning

no code implementations9 Sep 2019 Arjun Manoharan, Rahul Ramesh, Balaraman Ravindran

Option discovery and skill acquisition frameworks are integral to the functioning of a Hierarchically organized Reinforcement learning agent.

reinforcement-learning Reinforcement Learning +1

Successor Options: An Option Discovery Framework for Reinforcement Learning

1 code implementation14 May 2019 Rahul Ramesh, Manan Tomar, Balaraman Ravindran

This work adopts a complementary approach, where we attempt to discover options that navigate to landmark states.

Navigate reinforcement-learning +2

Cannot find the paper you are looking for? You can Submit a new open access paper.