Search Results for author:
Found 12 papers, 4 papers with code
Language Modeling on a SpiNNaker 2 Neuromorphic Chip
In this work, we demonstrate the first-ever implementation of a language model on a neuromorphic device - specifically the SpiNNaker 2 chip - based on a recently published event-based architecture called the EGRU.
Activity Sparsity Complements Weight Sparsity for Efficient RNN Inference
Yet, sparse activations, while omnipresent in both biological neural networks and deep learning systems, have not been fully utilized as a compression technique in deep learning.
Beyond Weights: Deep learning in Spiking Neural Networks with pure synaptic-delay training
We show that training ONLY the delays in feed-forward spiking networks using backpropagation can achieve performance comparable to the more conventional weight training.
Block-local learning with probabilistic latent representations
In addition, back-propagation relies on the transpose of forward weight matrices to compute updates, introducing a weight transport problem across the network.
Efficient Real Time Recurrent Learning through combined activity and parameter sparsity
Real-Time Recurrent Learning (RTRL) allows online learning, and the growth of required memory is independent of sequence length.
Efficient recurrent architectures through activity sparsity and sparse back-propagation through time
However, there is still a need to bridge the gap between what RNNs are capable of in terms of efficiency and performance and real-world application requirements.
Ranked #2 on
Gesture Recognition
on DVS128 Gesture
(using extra training data)
Exploring hyper-parameter spaces of neuroscience models on high performance computers with Learning to Learn
Neuroscience models commonly have a high number of degrees of freedom and only specific regions within the parameter space are able to produce dynamics of interest.
Embodied Synaptic Plasticity with Online Reinforcement learning
We demonstrate this framework to evaluate Synaptic Plasticity with Online REinforcement learning (SPORE), a reward-learning rule based on synaptic sampling, on two visuomotor tasks: reaching and lane following.
Reservoirs learn to learn
We wondered whether the performance of liquid state machines can be improved if the recurrent weights are chosen with a purpose, rather than randomly.
Eligibility traces provide a data-inspired alternative to backpropagation through time
Learning in recurrent neural networks (RNNs) is most often implemented by gradient descent using backpropagation through time (BPTT), but BPTT does not model accurately how the brain learns.
Long short-term memory and learning-to-learn in networks of spiking neurons
Recurrent networks of spiking neurons (RSNNs) underlie the astounding computing and learning capabilities of the brain.
Ranked #22 on
Speech Recognition
on TIMIT
Pattern representation and recognition with accelerated analog neuromorphic systems
Despite being originally inspired by the central nervous system, artificial neural networks have diverged from their biological archetypes as they have been remodeled to fit particular tasks.
