Search Results for author:
Found 16 papers, 8 papers with code
Prospective Learning: Learning for a Dynamic Future
We show that standard ERM as done in PAC learning, without incorporating time, can result in failure to learn when distributions are dynamic.
Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing
We call this phenomenon representation shattering and demonstrate that it results in degradation of factual recall and reasoning performance more broadly.
Many Perception Tasks are Highly Redundant Functions of their Input Data
We show that many perception tasks, from visual recognition, semantic segmentation, optical flow, depth estimation to vocalization discrimination, are highly redundant functions of their input data.
Towards an Understanding of Stepwise Inference in Transformers: A Synthetic Graph Navigation Model
Stepwise inference protocols, such as scratchpads and chain-of-thought, help language models solve complex problems by decomposing them into a sequence of simpler subproblems.
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks
Transformers trained on huge text corpora exhibit a remarkable set of capabilities, e. g., performing basic arithmetic.
The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold
We develop information-geometric techniques to analyze the trajectories of the predictions of deep networks during training.
A picture of the space of typical learnable tasks
We develop information geometric techniques to understand the representations learned by deep networks when they are trained on different tasks using supervised, meta-, semi-supervised and contrastive learning.
The Value of Out-of-Distribution Data
In this work, we show a counter-intuitive phenomenon: the generalization error of a task can be a non-monotonic function of the number of OOD samples.
Deep Reference Priors: What is the best way to pretrain a model?
Such priors enable the task to maximally affect the Bayesian posterior, e. g., reference priors depend upon the number of samples available for learning the task and for very small sample sizes, the prior puts more probability mass on low-complexity models in the hypothesis space.
image-classification
Semi-Supervised Image Classification
+1
Prospective Learning: Principled Extrapolation to the Future
We conjecture that certain sequences of tasks are not retrospectively learnable (in which the data distribution is fixed), but are prospectively learnable (in which distributions may be dynamic), suggesting that prospective learning is more difficult in kind than retrospective learning.
Model Zoo: A Growing Brain That Learns Continually
This paper argues that continual learning methods can benefit by splitting the capacity of the learner across multiple models.
Model Zoo: A Growing "Brain" That Learns Continually
We use statistical learning theory and experimental analysis to show how multiple tasks can interact with each other in a non-trivial fashion when a single model is trained on them.
Ranked #1 on
Continual Learning
on Rotated MNIST
Option Encoder: A Framework for Discovering a Policy Basis in Reinforcement Learning
Option discovery and skill acquisition frameworks are integral to the functioning of a Hierarchically organized Reinforcement learning agent.
Successor Options: An Option Discovery Framework for Reinforcement Learning
This work adopts a complementary approach, where we attempt to discover options that navigate to landmark states.
FigureNet: A Deep Learning model for Question-Answering on Scientific Plots
Deep Learning has managed to push boundaries in a wide variety of tasks.
Learning to Factor Policies and Action-Value Functions: Factored Action Space Representations for Deep Reinforcement learning
Deep Reinforcement Learning (DRL) methods have performed well in an increasing numbering of high-dimensional visual decision making domains.
