Search Results for author:
Found 40 papers, 27 papers with code
NASA Neural Articulated Shape Approximation
Efficient representation of articulated objects such as human bodies is an important problem in computer vision and graphics.
The Forward-Forward Algorithm: Some Preliminary Investigations
The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth further investigation.
Meta-Learning Fast Weight Language Models
Dynamic evaluation of language models (LMs) adapts model parameters at test time using gradient information from previous tokens and substantially improves LM performance.
Gaussian-Bernoulli RBMs Without Tears
We revisit the challenging problem of training Gaussian-Bernoulli restricted Boltzmann machines (GRBMs), introducing two innovations.
A Generalist Framework for Panoptic Segmentation of Images and Videos
Panoptic segmentation assigns semantic and instance ID labels to every pixel of an image.
Scaling Forward Gradient With Local Losses
Forward gradient learning computes a noisy directional gradient and is a biologically plausible alternative to backprop for learning deep neural networks.
Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning
The main idea behind our approach is to first represent the discrete data as binary bits, and then train a continuous diffusion model to model these bits as real numbers which we call analog bits.
Ranked #5 on
Image Captioning
on COCO
A Unified Sequence Interface for Vision Tasks
Despite that, by formulating the output of each task as a sequence of discrete tokens with a unified interface, we show that one can train a neural network with a single model architecture and loss function on all these tasks, with no task-specific customization.
Robust and Efficient Medical Imaging with Self-Supervision
These results suggest that REMEDIS can significantly accelerate the life-cycle of medical imaging AI development thereby presenting an important step forward for medical imaging AI to deliver broad impact.
Pix2seq: A Language Modeling Framework for Object Detection
We present Pix2Seq, a simple and generic framework for object detection.
Ranked #61 on
Object Detection
on COCO minival
(using extra training data)
How to represent part-whole hierarchies in a neural network
Instead, it presents a single idea about representation which allows advances made by several different groups to be combined into an imaginary system called GLOM.
Canonical Capsules: Self-Supervised Capsules in Canonical Pose
We propose a self-supervised capsule architecture for 3D point clouds.
Teaching with Commentaries
We find that commentaries can improve training speed and/or performance, and provide insights about the dataset and training process.
Big Self-Supervised Models are Strong Semi-Supervised Learners
The proposed semi-supervised learning algorithm can be summarized in three steps: unsupervised pretraining of a big ResNet model using SimCLRv2, supervised fine-tuning on a few labeled examples, and distillation with unlabeled examples for refining and transferring the task-specific knowledge.
Self-Supervised Image Classification
Semi-Supervised Image Classification
Neural Additive Models: Interpretable Machine Learning with Neural Nets
They perform similarly to existing state-of-the-art generalized additive models in accuracy, but are more flexible because they are based on neural nets instead of boosted trees.
Imputer: Sequence Modelling via Imputation and Dynamic Programming
This paper presents the Imputer, a neural sequence model that generates output sequences iteratively via imputations.
Deflecting Adversarial Attacks
There has been an ongoing cycle where stronger defenses against adversarial attacks are subsequently broken by a more advanced defense-aware attack.
A Simple Framework for Contrastive Learning of Visual Representations
This paper presents SimCLR: a simple framework for contrastive learning of visual representations.
Ranked #4 on
Contrastive Learning
on imagenet-1k
Contrastive Learning
Self-Supervised Image Classification
+3
Subclass Distillation
By training a small "student" model to match these probabilities, it is possible to transfer most of the generalization ability of the teacher to the student, often producing a much better small model than directly training the student on the training data.
NASA: Neural Articulated Shape Approximation
Efficient representation of articulated objects such as human bodies is an important problem in computer vision and graphics.
CvxNet: Learnable Convex Decomposition
We introduce a network architecture to represent a low dimensional family of convexes.
Lookahead Optimizer: k steps forward, 1 step back
The vast majority of successful deep neural networks are trained using variants of stochastic gradient descent (SGD) algorithms.
Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions
Then, we diagnose the adversarial examples for CapsNets and find that the success of the reconstructive attack is highly related to the visual similarity between the source and target class.
When Does Label Smoothing Help?
The generalization and learning speed of a multi-class neural network can often be significantly improved by using soft targets that are a weighted average of the hard targets and the uniform distribution over labels.
Cerberus: A Multi-headed Derenderer
To generalize to novel visual scenes with new viewpoints and new object poses, a visual system needs representations of the shapes of the parts of an object that are invariant to changes in viewpoint or pose.
Similarity of Neural Network Representations Revisited
We introduce a similarity index that measures the relationship between representational similarity matrices and does not suffer from this limitation.
Analyzing and Improving Representations with the Soft Nearest Neighbor Loss
We explore and expand the $\textit{Soft Nearest Neighbor Loss}$ to measure the $\textit{entanglement}$ of class manifolds in representation space: i. e., how close pairs of points from the same class are relative to pairs of points from different classes.
DARCCC: Detecting Adversaries by Reconstruction from Class Conditional Capsules
In addition to being trained to classify images, the capsule model is trained to reconstruct the images from the pose parameters and identity of the correct top-level capsule.
Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search
We introduce Picturebook, a large-scale lookup operation to ground language via {`}snapshots{'} of our physical world accessed through image search.
Distilling a Neural Network Into a Soft Decision Tree
They excel when the input data is high dimensional, the relationship between the input and the output is complicated, and the number of labeled training examples is large.
Regularizing Neural Networks by Penalizing Confident Output Distributions
We systematically explore regularizing neural networks by penalizing low entropy output distributions.
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
In this work, we address these challenges and finally realize the promise of conditional computation, achieving greater than 1000x improvements in model capacity with only minor losses in computational efficiency on modern GPU clusters.
Ranked #14 on
Language Modelling
on One Billion Word
Using Fast Weights to Attend to the Recent Past
Until recently, research on artificial neural networks was largely restricted to systems with only two types of variable: Neural activities that represent the current or recent input and weights that learn to capture regularities among inputs, outputs and payoffs.
Distilling the Knowledge in a Neural Network
A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.
Ranked #4 on
Knowledge Distillation
on ImageNet
Grammar as a Foreign Language
Syntactic constituency parsing is a fundamental problem in natural language processing and has been the subject of intensive research and engineering for decades.
Ranked #22 on
Constituency Parsing
on Penn Treebank
Dropout: A Simple Way to Prevent Neural Networks from Overfitting
The key idea is to randomly drop units (along with their connections) from the neural network during training.
On the importance of initialization and momentum in deep learning
Deep and recurrent neural networks (DNNs and RNNs respectively) are powerful models that were considered to be almost impossible to train using stochastic gradient descent with momentum.
Speech Recognition with Deep Recurrent Neural Networks
Recurrent neural networks (RNNs) are a powerful model for sequential data.
Ranked #18 on
Speech Recognition
on TIMIT
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Most current speech recognition systems use hidden Markov models (HMMs) to deal with the temporal variability of speech and Gaussian mixture models to determine how well each state of each HMM fits a frame or a short window of frames of coefficients that represents the acoustic input.
Restricted Boltzmann Machines for Collaborative Filtering
Most of the existing approaches to collaborative filtering cannot handle very large data sets.
