Search Results for author:
Found 22 papers, 7 papers with code
No One Representation to Rule Them All: Overlapping Features of Training Methods
Despite being able to capture a range of features of the data, high accuracy models trained with supervision tend to make similar predictions.
Robustmix: Improving Robustness by Regularizing the Frequency Bias of Deep Nets
Deep networks have achieved impressive results on a range of well curated benchmark datasets.
Auxiliary Task Update Decomposition: The Good, The Bad and The Neutral
In this case, careful consideration is needed to select tasks and model parameterizations such that updates from the auxiliary tasks actually help the primary task.
Continental-Scale Building Detection from High Resolution Satellite Imagery
Identifying the locations and footprints of buildings is vital for many practical and scientific purposes.
Deconstructing the Regularization of BatchNorm
Surprisingly, this simple mechanism matches the improvement of $0. 8\%$ of the more complex Dropout regularization for the state-of-the-art Efficientnet-B8 model on Imagenet.
Temperature check: theory and practice for training models with softmax-cross-entropy losses
In this work we develop a theory of early learning for models trained with softmax-cross-entropy loss and show that the learning dynamics depend crucially on the inverse-temperature $\beta$ as well as the magnitude of the logits at initialization, $||\beta{\bf z}||_{2}$.
Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win
Sparse Neural Networks (NNs) can match the generalization of dense NNs using a fraction of the compute/storage for inference, and also have the potential to enable efficient training.
Robust and On-the-fly Dataset Denoising for Image Classification
We address this problem by reasoning counterfactually about the loss distribution of examples with uniform random labels had they were trained with the real examples, and use this information to remove noisy examples from the training set.
Ranked #34 on
Image Classification
on mini WebVision 1.0
What Do Compressed Deep Neural Networks Forget?
However, this measure of performance conceals significant differences in how different classes and images are impacted by model compression techniques.
Selective Brain Damage: Measuring the Disparate Impact of Model Pruning
Neural network pruning techniques have demonstrated it is possible to remove the majority of weights in a network with surprisingly little degradation to top-1 test set accuracy.
Better Generalization with On-the-fly Dataset Denoising
Memorization in over-parameterized neural networks can severely hurt generalization in the presence of mislabeled examples.
On the Pitfalls of Measuring Emergent Communication
How do we know if communication is emerging in a multi-agent system?
Strategies for Structuring Story Generation
Writers generally rely on plans or sketches to write long stories, but most current language models generate word by word from left to right.
Hierarchical Neural Story Generation
We explore story generation: creative systems that can build coherent and fluent passages of text about a topic.
Deal or No Deal? End-to-End Learning of Negotiation Dialogues
Much of human dialogue occurs in semi-cooperative settings, where agents with different goals attempt to agree on common decisions.
Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
In particular, we present a case that links the two observations: small and large batch gradient descent appear to converge to different basins of attraction but we show that they are in fact connected through their flat region and so belong to the same basin.
Tackling Over-pruning in Variational Autoencoders
The so-called epitomes of this model are groups of mutually exclusive latent factors that compete to explain the data.
Parseval Networks: Improving Robustness to Adversarial Examples
We introduce Parseval networks, a form of deep neural networks in which the Lipschitz constant of linear, convolutional and aggregation layers is constrained to be smaller than 1.
EmoNets: Multimodal deep learning approaches for emotion recognition in video
The task of the emotion recognition in the wild (EmotiW) Challenge is to assign one of seven emotions to short video clips extracted from Hollywood style movies.
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such minimizations, and it is often thought that a main source of difficulty for these local methods to find the global minimum is the proliferation of local minima with much higher error than the global minimum.
Stochastic Ratio Matching of RBMs for Sparse High-Dimensional Inputs
Sparse high-dimensional data vectors are common in many application domains where a very large number of rarely non-zero features can be devised.
Better Mixing via Deep Representations
It has previously been hypothesized, and supported with some experimental evidence, that deeper representations, when well trained, tend to do a better job at disentangling the underlying factors of variation.
