Search Results for author:
Found 22 papers, 12 papers with code
Evaluating Machine Accuracy on ImageNet
We perform an in-depth evaluation of human accuracy on the ImageNet dataset.
Training Language Models to Self-Correct via Reinforcement Learning
In particular, we observe that training via SFT falls prey to either a distribution mismatch between mistakes made by the data-collection policy and the model's own responses, or to behavior collapse, where learning implicitly prefers only a certain mode of correction behavior that is often not effective at self-correction on test problems.
Multi-Agent Reachability Calibration with Conformal Prediction
We investigate methods to provide safety assurances for autonomous agents that incorporate predictions of other, uncontrolled agents' behavior into their own trajectory planning.
Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios
To our knowledge, this is the first application of a combined imitation and reinforcement learning approach in autonomous driving that utilizes large amounts of real-world human driving data.
CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationships
Finally, we release the causal agent labels (at https://github. com/google-research/causal-agents) as an additional attribute to WOMD and the robustness benchmarks to aid the community in building more reliable and safe deep-learning models for motion forecasting.
When does dough become a bagel? Analyzing the remaining mistakes on ImageNet
To help contextualize progress on ImageNet and provide a more meaningful evaluation for today's state-of-the-art models, we manually review and categorize every remaining mistake that a few top models make in order to provide insight into the long-tail of errors on one of the most benchmarked datasets in computer vision.
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
The conventional recipe for maximizing model accuracy is to (1) train multiple models with various hyperparameters and (2) pick the individual model which performs best on a held-out validation set, discarding the remainder.
Ranked #1 on
Image Classification
on ImageNet V2
(using extra training data)
Spectral Bias in Practice: The Role of Function Frequency in Generalization
We also explore the connections between function frequency and image frequency and find that spectral bias is sensitive to the low frequencies prevalent in natural images.
Scene Transformer: A unified architecture for predicting future trajectories of multiple agents
In this work, we formulate a model for predicting the behavior of all agents jointly, producing consistent futures that account for interactions between agents.
Robust fine-tuning of zero-shot models
Compared to standard fine-tuning, WiSE-FT provides large accuracy improvements under distribution shift, while preserving high accuracy on the target distribution.
Ranked #12 on
Image Classification
on ObjectNet
(using extra training data)
The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning
Although machine learning models typically experience a drop in performance on out-of-distribution data, accuracies on in- versus out-of-distribution data are widely observed to follow a single linear trend when evaluated across a testbed of models.
Scene Transformer: A unified architecture for predicting multiple agent trajectories
In this work, we formulate a model for predicting the behavior of all agents jointly, producing consistent futures that account for interactions between agents.
AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation
We extend semi-supervised learning to the problem of domain adaptation to learn significantly higher-accuracy models that train on one data distribution and test on a different one.
Semi-supervised Domain Adaptation
Unsupervised Domain Adaptation
Pseudo-labeling for Scalable 3D Object Detection
To safely deploy autonomous vehicles, onboard perception systems must work reliably at high accuracy across a diverse set of environments and geographies.
Mitigating Bias in Calibration Error Estimation
We find that binning-based estimators with bins of equal mass (number of instances) have lower bias than estimators with bins of equal width.
A Meta-Analysis of Overfitting in Machine Learning
By systematically comparing the public ranking with the final ranking, we assess how much participants adapted to the holdout set over the course of a competition.
Do Image Classifiers Generalize Across Time?
Additionally, we evaluate three detection models and show that natural perturbations induce both classification as well as localization errors, leading to a median drop in detection mAP of 14 points.
A Systematic Framework for Natural Perturbations from Videos
We introduce a systematic framework for quantifying the robustness of classifiers to naturally occurring perturbations of images found in videos.
Do ImageNet Classifiers Generalize to ImageNet?
We evaluate a broad range of models and find accuracy drops of 3% - 15% on CIFAR-10 and 11% - 14% on ImageNet.
Do CIFAR-10 Classifiers Generalize to CIFAR-10?
Although we ensure that the new test set is as close to the original data distribution as possible, we find a large drop in accuracy (4% to 10%) for a broad range of deep learning models.
The Marginal Value of Adaptive Gradient Methods in Machine Learning
Adaptive optimization methods, which perform local optimization with a metric constructed from the history of iterates, are becoming increasingly popular for training deep neural networks.
Large Scale Kernel Learning using Block Coordinate Descent
We demonstrate that distributed block coordinate descent can quickly solve kernel regression and classification problems with millions of data points.
