Search Results for author:
Found 18 papers, 11 papers with code
Neglected Hessian component explains mysteries in Sharpness regularization
We find that regularizing feature exploitation but not feature exploration yields performance similar to gradient penalties.
Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment
We present the NeurIPS 2021 consistency experiment, a larger-scale variant of the 2014 NeurIPS experiment in which 10% of conference submissions were reviewed by two independent committees to quantify the randomness in the review process.
SAM operates far from home: eigenvalue regularization as a dynamical phenomenon
We show that in a simplified setting, SAM dynamically induces a stabilization related to the edge of stability (EOS) phenomenon observed in large learning rate gradient descent.
How do Authors' Perceptions of their Papers Compare with Co-authors' Perceptions and Peer-review Decisions?
In a top-tier computer science conference (NeurIPS 2021) with more than 23, 000 submitting authors and 9, 000 submitted papers, we survey the authors on three questions: (i) their predicted probability of acceptance for each of their papers, (ii) their perceived ranking of their own papers based on scientific contribution, and (iii) the change in their perception about their own papers after seeing the reviews.
MetaInit: Initializing learning by learning to initialize
In particular, we find that this approach outperforms normalization for networks without skip connections on CIFAR-10 and can scale to Resnet-50 models on Imagenet.
Simple and Effective Noisy Channel Modeling for Neural Machine Translation
Previous work on neural noisy channel modeling relied on latent variable models that incrementally process the source and target sentence.
Pay Less Attention with Lightweight and Dynamic Convolutions
We predict separate convolution kernels based solely on the current time-step in order to determine the importance of context elements.
Ranked #1 on
Machine Translation
on WMT 2017 English-Chinese
Fixup Initialization: Residual Learning Without Normalization
Normalization layers are a staple in state-of-the-art deep neural network architectures.
Ranked #8 on
Image Classification
on SVHN
mixup: Beyond Empirical Risk Minimization
We also find that mixup reduces the memorization of corrupt labels, increases the robustness to adversarial examples, and stabilizes the training of generative adversarial networks.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
Deal or No Deal? End-to-End Learning for Negotiation Dialogues
Much of human dialogue occurs in semi-cooperative settings, where agents with different goals attempt to agree on common decisions.
Convolutional Sequence to Sequence Learning
The prevalent approach to sequence to sequence learning maps an input sequence to a variable length output sequence via recurrent neural networks.
Language Modeling with Gated Convolutional Networks
The pre-dominant approach to language modeling to date is based on recurrent neural networks.
Ranked #20 on
Language Modelling
on One Billion Word
A Convolutional Encoder Model for Neural Machine Translation
The prevalent approach to neural machine translation relies on bi-directional LSTMs to encode the source sentence.
Ranked #7 on
Machine Translation
on IWSLT2015 German-English
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Predicting distributions with Linearizing Belief Networks
Contrary to a classical neural network, a belief network can predict more than the expected value of the output $Y$ given the input $X$.
Equilibrated adaptive learning rates for non-convex optimization
Parameter-specific adaptive learning rate methods are computationally efficient ways to reduce the ill-conditioning problems encountered when training large deep networks.
On the saddle point problem for non-convex optimization
Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such minimizations, and it is often thought that a main source of difficulty for the ability of these local methods to find the global minimum is the proliferation of local minima with much higher error than the global minimum.
Zero-Shot Learning for Semantic Utterance Classification
We propose a novel zero-shot learning method for semantic utterance classification (SUC).
