Search Results for author:
Found 20 papers, 9 papers with code
PaLM 2 Technical Report
Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM.
Gradient-Based Automated Iterative Recovery for Parameter-Efficient Tuning
In this paper, we explore the use of TracIn to improve model performance in the parameter-efficient tuning (PET) setting.
FAVOR#: Sharp Attention Kernel Approximations via New Classes of Positive Random Features
The problem of efficient approximation of a linear operator induced by the Gaussian or softmax kernel is often addressed using random features (RFs) which yield an unbiased approximation of the operator's result.
Augmentation with Projection: Towards an Effective and Efficient Data Augmentation Paradigm for Distillation
However, these data augmentation methods either potentially cause shifts in decision boundaries (representation interpolation), are not expressive enough (token replacement), or introduce too much computational overhead (augmentation with models).
DETR++: Taming Your Multi-Scale Detection Transformer
Convolutional Neural Networks (CNN) have dominated the field of detection ever since the success of AlexNet in ImageNet classification [12].
Chefs' Random Tables: Non-Trigonometric Random Features
We introduce chefs' random tables (CRTs), a new class of non-trigonometric random features (RFs) to approximate Gaussian and softmax kernels.
Towards Tracing Factual Knowledge in Language Models Back to the Training Data
In this paper, we propose the problem of fact tracing: identifying which training examples taught an LM to generate a particular factual assertion.
First is Better Than Last for Language Data Influence
However, we observe that since the activation connected to the last layer of weights contains "shared logic", the data influenced calculated via the last layer weights prone to a ``cancellation effect'', where the data influence of different examples have large magnitude that contradicts each other.
Threading the Needle of On and Off-Manifold Value Functions for Shapley Explanations
We formalize the desiderata of value functions that respect both the model and the data manifold in a set of axioms and are robust to perturbation on off-manifold regions, and show that there exists a unique value function that satisfies these axioms, which we term the Joint Baseline value function, and the resulting Shapley value the Joint Baseline Shapley (JBshap), and validate the effectiveness of JBshap in experiments.
EncT5: A Framework for Fine-tuning T5 as Non-autoregressive Models
Pre-trained encoder-decoder transformer architectures have become increasingly popular recently with the advent of T5 models.
Leveraging redundancy in attention with Reuse Transformers
Pairwise dot product-based attention allows Transformers to exchange information between tokens in an input-dependent way, and is key to their success across diverse applications in language and vision.
Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensembles
This observation leads to two challenging tasks: (1) unsupervised accuracy estimation, which aims to estimate the accuracy of a pre-trained classifier on a set of unlabeled test inputs; (2) error detection, which aims to identify mis-classified test inputs.
The Penalty Imposed by Ablated Data Augmentation
There is a set of data augmentation techniques that ablate parts of the input at random.
Estimating Training Data Influence by Tracing Gradient Descent
We introduce a method called TracIn that computes the influence of a training example on a prediction made by the model.
Incorporating Priors with Feature Attribution on Text Classification
Feature attribution methods, proposed recently, help users interpret the predictions of complex models.
Differentially-Private "Draw and Discard" Machine Learning
In this work, we propose a novel framework for privacy-preserving client-distributed machine learning.
Handling Homographs in Neural Machine Translation
Homographs, words with different meanings but the same surface form, have long caused difficulty for machine translation systems, as it is difficult to select the correct translation based on the context.
Learning Character-level Compositionality with Visual Features
Previous work has modeled the compositionality of words by creating character-level models of meaning, reducing problems of sparsity for rare words.
