You need to log in to edit.

You can create a new account if you don't have one.

Or, discuss a change on Slack.

You can create a new account if you don't have one.

Or, discuss a change on Slack.

no code implementations • 27 Nov 2020 • Sara Sabour, Andrea Tagliasacchi, Soroosh Yazdani, Geoffrey E. Hinton, David J. Fleet

Capsule networks aim to parse images into a hierarchy of objects, parts and relations.

12 code implementations • NeurIPS 2019 • Adam R. Kosiorek, Sara Sabour, Yee Whye Teh, Geoffrey E. Hinton

In the second stage, SCAE predicts parameters of a few object capsules, which are then used to reconstruct part poses.

Ranked #2 on Unsupervised MNIST on MNIST

2 code implementations • 31 May 2019 • Aidan N. Gomez, Ivan Zhang, Siddhartha Rao Kamalakara, Divyam Madaan, Kevin Swersky, Yarin Gal, Geoffrey E. Hinton

Before computing the gradients for each weight update, targeted dropout stochastically selects a set of units or weights to be dropped using a simple self-reinforcing sparsity criterion and then computes the gradients for the remaining weights.

1 code implementation • NIPS Workshop CDNNRIA 2018 • Aidan N. Gomez, Ivan Zhang, Kevin Swersky, Yarin Gal, Geoffrey E. Hinton

Neural networks are extremely flexible models due to their large number of parameters, which is beneficial for learning, but also highly redundant.

1 code implementation • NeurIPS 2018 • Sergey Bartunov, Adam Santoro, Blake A. Richards, Luke Marris, Geoffrey E. Hinton, Timothy Lillicrap

Here we present results on scaling up biologically motivated models of deep learning on datasets which need deep networks with appropriate architectures to achieve good performance.

no code implementations • ICLR 2018 • Rohan Anil, Gabriel Pereyra, Alexandre Passos, Robert Ormandi, George E. Dahl, Geoffrey E. Hinton

Two neural networks trained on disjoint subsets of the data can share knowledge by encouraging each model to agree with the predictions the other model would have made.

1 code implementation • ICLR 2018 • Geoffrey E. Hinton, Sara Sabour, Nicholas Frosst

A capsule in one layer votes for the pose matrix of many different capsules in the layer above by multiplying its own pose matrix by trainable viewpoint-invariant transformation matrices that could learn to represent part-whole relationships.

Ranked #3 on Image Classification on smallNORB

71 code implementations • NeurIPS 2017 • Sara Sabour, Nicholas Frosst, Geoffrey E. Hinton

We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters.

Ranked #1 on Image Classification on MultiMNIST

1 code implementation • 26 Mar 2017 • Melody Y. Guan, Varun Gulshan, Andrew M. Dai, Geoffrey E. Hinton

We also show that our method performs better than competing algorithms by Welinder and Perona (2010), and by Mnih and Hinton (2012).

28 code implementations • 21 Jul 2016 • Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton

One way to reduce the training time is to normalize the activities of the neurons.

2 code implementations • NeurIPS 2016 • S. M. Ali Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, David Szepesvari, Koray Kavukcuoglu, Geoffrey E. Hinton

We present a framework for efficient inference in structured image models that explicitly reason about objects.

4 code implementations • 3 Apr 2015 • Quoc V. Le, Navdeep Jaitly, Geoffrey E. Hinton

Learning long term dependencies in recurrent networks is difficult due to vanishing and exploding gradients.

Ranked #23 on Sequential Image Classification on Sequential MNIST

no code implementations • 26 Sep 2013 • Nitish Srivastava, Ruslan R. Salakhutdinov, Geoffrey E. Hinton

We introduce a Deep Boltzmann Machine model suitable for modeling and extracting latent semantic representations from a large unstructured collection of documents.

10 code implementations • NeurIPS 2012 • Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton

We trained a large, deep convolutional neural network to classify the 1. 3 million high-resolution images in the LSVRC-2010 ImageNet training set into the 1000 different classes.

Ranked #4 on Graph Classification on HIV-fMRI-77

no code implementations • NeurIPS 2012 • Geoffrey E. Hinton, Ruslan R. Salakhutdinov

We describe how the pre-training algorithm for Deep Boltzmann Machines (DBMs) is related to the pre-training algorithm for Deep Belief Networks and we show that under certain conditions, the pre-training procedure improves the variational lower bound of a two-hidden-layer DBM.

9 code implementations • 3 Jul 2012 • Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, Ruslan R. Salakhutdinov

When a large feedforward neural network is trained on a small training set, it typically performs poorly on held-out test data.

Ranked #178 on Image Classification on CIFAR-10

no code implementations • NeurIPS 2010 • George Dahl, Marc'Aurelio Ranzato, Abdel-rahman Mohamed, Geoffrey E. Hinton

Straightforward application of Deep Belief Nets (DBNs) to acoustic modeling produces a rich distributed representation of speech data that is useful for recognition and yields impressive results on the speaker-independent TIMIT phone recognition task.

no code implementations • NeurIPS 2010 • Roland Memisevic, Christopher Zach, Marc Pollefeys, Geoffrey E. Hinton

We describe a log-bilinear" model that computes class probabilities by combining an input vector multiplicatively with a vector of binary latent variables.

no code implementations • NeurIPS 2010 • Marc'Aurelio Ranzato, Volodymyr Mnih, Geoffrey E. Hinton

Probabilistic models of natural images are usually evaluated by measuring performance on rather indirect tasks, such as denoising and inpainting.

no code implementations • NeurIPS 2010 • Hugo Larochelle, Geoffrey E. Hinton

We describe a model based on a Boltzmann machine with third-order connections that can learn how to accumulate information about a shape over several fixations.

no code implementations • NeurIPS 2009 • Mark Palatucci, Dean Pomerleau, Geoffrey E. Hinton, Tom M. Mitchell

To achieve this, we define the notion of a semantic output code classifier (SOC) which utilizes a knowledge base of semantic properties of $Y$ to extrapolate to novel classes.

no code implementations • NeurIPS 2009 • Geoffrey E. Hinton, Ruslan R. Salakhutdinov

Each member of the family models the probability distribution of documents of a specific length as a product of topic-specific distributions rather than as a mixture and this gives much better generalization than Latent Dirichlet Allocation for modeling the log probabilities of held-out documents.

no code implementations • NeurIPS 2009 • Vinod Nair, Geoffrey E. Hinton

Our model achieves 6. 5% error on the test set, which is close to the best published result for NORB (5. 9%) using a convolutional neural net that has built-in knowledge of translation invariance.

no code implementations • NeurIPS 2008 • Andriy Mnih, Geoffrey E. Hinton

Neural probabilistic language models (NPLMs) have been shown to be competitive with and occasionally superior to the widely-used n-gram language models.

no code implementations • NeurIPS 2008 • Vinod Nair, Geoffrey E. Hinton

We present a mixture model whose components are Restricted Boltzmann Machines (RBMs).

no code implementations • NeurIPS 2008 • Ilya Sutskever, Geoffrey E. Hinton

We describe a way of learning matrix representations of objects and relationships.

no code implementations • NeurIPS 2008 • Ilya Sutskever, Geoffrey E. Hinton, Graham W. Taylor

The Temporal Restricted Boltzmann Machine (TRBM) is a probabilistic model for sequences that is able to successfully model (i. e., generate nice-looking samples of) several very high dimensional sequences, such as motion capture data and the pixels of low resolution videos of balls bouncing in a box.

no code implementations • NeurIPS 2007 • Simon Osindero, Geoffrey E. Hinton

We describe an efficient learning procedure for multilayer generative models that combine the best aspects of Markov random fields and deep, directed belief nets.

no code implementations • NeurIPS 2007 • Geoffrey E. Hinton, Ruslan R. Salakhutdinov

We show how to use unlabeled data and a deep belief net (DBN) to learn a good covariance kernel for a Gaussian process.

1 code implementation • Science 2006 • Geoffrey E. Hinton, R. R. Salakhutdinov

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors.

1 code implementation • Neural Computation 2006 • Geoffrey E. Hinton, Simon Osindero, Yee-Whye Teh

Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory.

1 code implementation • 20 Feb 1986 • David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams

The rule, called the generalized delta rule, is a simple scheme for implementing a gradient descent method for finding weights that minimize the sum squared error of the system's performance.

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.