Search Results
The State of Sparsity in Deep Neural Networks
google-research/google-research
•
•
We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to-German, and ResNet-50 trained on ImageNet.
Predicting the Generalization Gap in Deep Networks with Margin Distributions
google-research/google-research
•
•
ICLR 2019
In this paper, we propose such a measure, and conduct extensive empirical studies on how well it can predict the generalization gap.
Behavior Regularized Offline Reinforcement Learning
google-research/google-research
•
•
In reinforcement learning (RL) research, it is common to assume access to direct online interactions with the environment.
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
google-research/google-research
•
•
NeurIPS 2019
Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful estimates of their predictive {\em uncertainty}.
Tight Dimensionality Reduction for Sketching Low Degree Polynomial Kernels
google-research/google-research
•
•
NeurIPS 2019
We give a new analysis of this sketch, providing nearly optimal bounds.
Momentum-Based Variance Reduction in Non-Convex SGD
google-research/google-research
•
•
NeurIPS 2019
Variance reduction has emerged in recent years as a strong competitor to stochastic gradient descent in non-convex problems, providing the first algorithms to improve upon the converge rate of stochastic gradient descent for finding first-order critical points.
Saccader: Improving Accuracy of Hard Attention Models for Vision
google-research/google-research
•
•
NeurIPS 2019
Although deep convolutional neural networks achieve state-of-the-art performance across nearly all image classification tasks, their decisions are difficult to interpret.
Milking CowMask for Semi-Supervised Image Classification
google-research/google-research
•
•
Using it to provide perturbations for semi-supervised consistency regularization, we achieve a state-of-the-art result on ImageNet with 10% labeled data, with a top-5 error of 8. 76% and top-1 error of 26. 06%.
Dataset Distillation with Infinitely Wide Convolutional Networks
google-research/google-research
•
•
NeurIPS 2021
The effectiveness of machine learning algorithms arises from being able to extract useful features from large amounts of data.
Meta Back-translation
google-research/google-research
•
•
ICLR 2021
Back-translation is an effective strategy to improve the performance of Neural Machine Translation~(NMT) by generating pseudo-parallel data.
