Search Results for author:
Found 34 papers, 16 papers with code
Testing GLOM's ability to infer wholes from ambiguous parts
The GLOM architecture proposed by Hinton [2021] is a recurrent neural network for parsing an image into a hierarchy of wholes and parts.
Unsupervised part representation by Flow Capsules
Capsule networks aim to parse images into a hierarchy of objects, parts and relations.
Stacked Capsule Autoencoders
In the second stage, SCAE predicts parameters of a few object capsules, which are then used to reconstruct part poses.
Ranked #3 on
Unsupervised MNIST
on MNIST
Learning Sparse Networks Using Targeted Dropout
Before computing the gradients for each weight update, targeted dropout stochastically selects a set of units or weights to be dropped using a simple self-reinforcing sparsity criterion and then computes the gradients for the remaining weights.
Targeted Dropout
Neural networks are extremely flexible models due to their large number of parameters, which is beneficial for learning, but also highly redundant.
Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures
Here we present results on scaling up biologically motivated models of deep learning on datasets which need deep networks with appropriate architectures to achieve good performance.
Large scale distributed neural network training through online distillation
Two neural networks trained on disjoint subsets of the data can share knowledge by encouraging each model to agree with the predictions the other model would have made.
Matrix capsules with EM routing
A capsule in one layer votes for the pose matrix of many different capsules in the layer above by multiplying its own pose matrix by trainable viewpoint-invariant transformation matrices that could learn to represent part-whole relationships.
Ranked #4 on
Image Classification
on smallNORB
Dynamic Routing Between Capsules
We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters.
Ranked #1 on
Image Classification
on MultiMNIST
Who Said What: Modeling Individual Labelers Improves Classification
We also show that our method performs better than competing algorithms by Welinder and Perona (2010), and by Mnih and Hinton (2012).
Layer Normalization
One way to reduce the training time is to normalize the activities of the neurons.
Attend, Infer, Repeat: Fast Scene Understanding with Generative Models
We present a framework for efficient inference in structured image models that explicitly reason about objects.
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
Learning long term dependencies in recurrent networks is difficult due to vanishing and exploding gradients.
Ranked #27 on
Sequential Image Classification
on Sequential MNIST
Modeling Documents with Deep Boltzmann Machines
We introduce a Deep Boltzmann Machine model suitable for modeling and extracting latent semantic representations from a large unstructured collection of documents.
A Better Way to Pretrain Deep Boltzmann Machines
We describe how the pre-training algorithm for Deep Boltzmann Machines (DBMs) is related to the pre-training algorithm for Deep Belief Networks and we show that under certain conditions, the pre-training procedure improves the variational lower bound of a two-hidden-layer DBM.
ImageNet Classification with Deep Convolutional Neural Networks
We trained a large, deep convolutional neural network to classify the 1. 3 million high-resolution images in the LSVRC-2010 ImageNet training set into the 1000 different classes.
Ranked #4 on
Graph Classification
on HIV-fMRI-77
Improving neural networks by preventing co-adaptation of feature detectors
When a large feedforward neural network is trained on a small training set, it typically performs poorly on held-out test data.
Ranked #217 on
Image Classification
on CIFAR-10
Learning to combine foveal glimpses with a third-order Boltzmann machine
We describe a model based on a Boltzmann machine with third-order connections that can learn how to accumulate information about a shape over several fixations.
Gated Softmax Classification
We describe a log-bilinear" model that computes class probabilities by combining an input vector multiplicatively with a vector of binary latent variables.
Generating more realistic images using gated MRF's
Probabilistic models of natural images are usually evaluated by measuring performance on rather indirect tasks, such as denoising and inpainting.
Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine
Straightforward application of Deep Belief Nets (DBNs) to acoustic modeling produces a rich distributed representation of speech data that is useful for recognition and yields impressive results on the speaker-independent TIMIT phone recognition task.
Zero-shot Learning with Semantic Output Codes
To achieve this, we define the notion of a semantic output code classifier (SOC) which utilizes a knowledge base of semantic properties of $Y$ to extrapolate to novel classes.
Replicated Softmax: an Undirected Topic Model
Each member of the family models the probability distribution of documents of a specific length as a product of topic-specific distributions rather than as a mixture and this gives much better generalization than Latent Dirichlet Allocation for modeling the log probabilities of held-out documents.
3D Object Recognition with Deep Belief Nets
Our model achieves 6. 5% error on the test set, which is close to the best published result for NORB (5. 9%) using a convolutional neural net that has built-in knowledge of translation invariance.
Using matrices to model symbolic relationship
We describe a way of learning matrix representations of objects and relationships.
A Scalable Hierarchical Distributed Language Model
Neural probabilistic language models (NPLMs) have been shown to be competitive with and occasionally superior to the widely-used n-gram language models.
The Recurrent Temporal Restricted Boltzmann Machine
The Temporal Restricted Boltzmann Machine (TRBM) is a probabilistic model for sequences that is able to successfully model (i. e., generate nice-looking samples of) several very high dimensional sequences, such as motion capture data and the pixels of low resolution videos of balls bouncing in a box.
Implicit Mixtures of Restricted Boltzmann Machines
We present a mixture model whose components are Restricted Boltzmann Machines (RBMs).
Modeling image patches with a directed hierarchy of Markov random fields
We describe an efficient learning procedure for multilayer generative models that combine the best aspects of Markov random fields and deep, directed belief nets.
Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes
We show how to use unlabeled data and a deep belief net (DBN) to learn a good covariance kernel for a Gaussian process.
Reducing the Dimensionality of Data with Neural Networks
High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors.
A fast learning algorithm for deep belief nets
Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.
Stochastic Neighbor Embedding
We describe a probabilistic approach to the task of placing objects, de- scribed by high-dimensional vectors or by pairwise dissimilarities, in a low-dimensional space in a way that preserves neighbor identities.
Learning Internal Representations by Error Propagation
The rule, called the generalized delta rule, is a simple scheme for implementing a gradient descent method for finding weights that minimize the sum squared error of the system's performance.
