Search Results for author:
Found 21 papers, 14 papers with code
Estimating Q(s,s') with Deterministic Dynamics Gradients
In this paper, we introduce a novel form of a value function, $Q(s, s')$, that expresses the utility of transitioning from a state $s$ to a neighboring state $s'$ and then acting optimally thereafter.
Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5?
We introduce and study the problem of adversarial arithmetic, which provides a simple yet challenging testbed for language model alignment.
Character-Aware Models Improve Visual Text Rendering
In the text-only domain, we find that character-aware models provide large gains on a novel spelling task (WikiSpell).
Extremely Simple Activation Shaping for Out-of-Distribution Detection
The separation between training and deployment of machine learning models implies that not all scenarios encountered in deployment can be anticipated during training, and therefore relying solely on advancements in training has its limits.
What does a platypus look like? Generating customized prompts for zero-shot image classification
Unlike traditional classification models, open-vocabulary models classify among any arbitrary set of categories specified with natural language during inference.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
When less is more: Simplifying inputs aids neural network understanding
To answer these questions, we need a clear measure of input simplicity (or inversely, complexity), an optimization objective that correlates with simplification, and a framework to incorporate such objective into training and inference.
Natural Adversarial Objects
Moreover, by comparing a variety of object detection architectures, we find that better performance on MSCOCO validation set does not necessarily translate to better performance on NAO, suggesting that robustness cannot be simply achieved by training a more accurate model.
Ranked #1 on
Object Detection
on NAO
Exploring unfairness in Integrated Gradients based attribution methods
Integrated Gradients has axioms derived from this heritage with the implication of a similar rigorous, intuitive notion of fairness.
Language Models are Few-shot Multilingual Learners
General-purpose language models have demonstrated impressive capabilities, performing on par with state-of-the-art approaches on a range of downstream natural language processing (NLP) tasks and benchmarks when inferring instructions from very few examples.
When does loss-based prioritization fail?
Each example is propagated forward and backward through the network the same amount of times, independent of how much the example contributes to the learning protocol.
Why is Pruning at Initialization Immune to Reinitializing and Shuffling?
Recent studies assessing the efficacy of pruning neural networks methods uncovered a surprising finding: when conducting ablation studies on existing pruning-at-initialization methods, namely SNIP, GraSP, SynFlow, and magnitude pruning, performances of these methods remain unchanged and sometimes even improve when randomly shuffling the mask positions within each layer (Layerwise Shuffling) or sampling new initial weight values (Reinit), while keeping pruning masks the same.
Supermasks in Superposition
We present the Supermasks in Superposition (SupSup) model, capable of sequentially learning thousands of tasks without catastrophic forgetting.
Estimating Q(s,s') with Deep Deterministic Dynamics Gradients
In this paper, we introduce a novel form of value function, $Q(s, s')$, that expresses the utility of transitioning from a state $s$ to a neighboring state $s'$ and then acting optimally thereafter.
Plug and Play Language Models: A Simple Approach to Controlled Text Generation
Large transformer-based language models (LMs) trained on huge text corpora have shown unparalleled generation capabilities.
LCA: Loss Change Allocation for Neural Network Training
We propose a new window into training called Loss Change Allocation (LCA), in which credit for changes to the network loss is conservatively partitioned to the parameters.
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask
The recent "Lottery Ticket Hypothesis" paper by Frankle & Carbin showed that a simple approach to creating sparse networks (keeping the large weights) results in models that are trainable from scratch, but only when starting from the same initial weights.
An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agents
We lessen this friction, by (1) training several algorithms at scale and releasing trained models, (2) integrating with a previous Deep RL model release, and (3) releasing code that makes it easy for anyone to load, visualize, and analyze such models.
Faster Neural Networks Straight from JPEG
In this paper we modify \libjpeg to produce DCT coefficients directly, modify a ResNet-50 network to accommodate the differently sized and strided input, and evaluate performance on ImageNet.
An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution
In this paper we show a striking counterexample to this intuition via the seemingly trivial coordinate transform problem, which simply requires learning a mapping between coordinates in (x, y) Cartesian space and one-hot pixel space.
Ranked #833 on
Image Classification
on ImageNet
Measuring the Intrinsic Dimension of Objective Landscapes
A byproduct of this construction is a simple approach for compressing networks, in some cases by more than 100 times.
