Search Results for author:
Found 17 papers, 5 papers with code
Weight Sparsity Complements Activity Sparsity in Neuromorphic Language Models
Activity and parameter sparsity are two standard methods of making neural networks computationally more efficient.
Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Models
While prior methods can process up to a few thousand time steps, our model, based on modern recurrent deep state-space models, scales to event streams of millions of events for both training and inference. We leverage their stable parameterization for learning long-range dependencies, parallelizability along the sequence dimension, and their ability to integrate asynchronous events effectively to scale them up to long event streams. We further augment these with novel event-centric techniques enabling our model to match or beat the state-of-the-art performance on several event stream benchmarks.
Ranked #1 on
Audio Classification
on SSC
Neuromorphic hardware for sustainable AI data centers
In this article, we try to analyze the underlying reasons for this and derive requirements and guidelines to promote neuromorphic systems for efficient and sustainable cloud computing: We first review currently available neuromorphic hardware systems and collect examples where neuromorphic solutions excel conventional AI processing on CPUs and GPUs.
Language Modeling on a SpiNNaker 2 Neuromorphic Chip
In this work, we demonstrate the first-ever implementation of a language model on a neuromorphic device - specifically the SpiNNaker 2 chip - based on a recently published event-based architecture called the EGRU.
Block-local learning with probabilistic latent representations
In addition, back-propagation relies on the transpose of forward weight matrices to compute updates, introducing a weight transport problem across the network.
Efficient recurrent architectures through activity sparsity and sparse back-propagation through time
However, there is still a need to bridge the gap between what RNNs are capable of in terms of efficiency and performance and real-world application requirements.
Ranked #3 on
Gesture Recognition
on DVS128 Gesture
(using extra training data)
Continual Learning with Memory Cascades
This general formulation allows us to use the model also for online learning where no knowledge about task switching times is given to the network.
A synapse-centric account of the free energy principle
The free energy principle (FEP) is a mathematical framework that describes how biological systems self-organize and survive in their environment.
Embodied Synaptic Plasticity with Online Reinforcement learning
We demonstrate this framework to evaluate Synaptic Plasticity with Online REinforcement learning (SPORE), a reward-learning rule based on synaptic sampling, on two visuomotor tasks: reaching and lane following.
Structural plasticity on an accelerated analog neuromorphic hardware system
In computational neuroscience, as well as in machine learning, neuromorphic devices promise an accelerated and scalable alternative to neural network simulations.
Attention on Abstract Visual Reasoning
Our proposed hybrid model, represents an alternative on learning abstract relations using self-attention and demonstrates that the Transformer network is also well suited for abstract visual reasoning.
Efficient Reward-Based Structural Plasticity on a SpiNNaker 2 Prototype
Advances in neuroscience uncover the mechanisms employed by the brain to efficiently solve complex learning tasks with very limited resources.
Deep Rewiring: Training very sparse deep networks
Neuromorphic hardware tends to pose limits on the connectivity of deep networks that one can run on them.
A dynamic connectome supports the emergence of stable computational function of neural circuits through reward-based learning
These data are inconsistent with common models for network plasticity, and raise the questions how neural circuits can maintain a stable computational function in spite of these continuously ongoing processes, and what functional uses these ongoing processes might have.
CaMKII activation supports reward-based neural network optimization through Hamiltonian sampling
Our theoretical analysis shows that stochastic search could in principle even attain optimal network configurations by emulating one of the most well-known nonlinear optimization methods, simulated annealing.
Synaptic Sampling: A Bayesian Approach to Neural Network Plasticity and Rewiring
We reexamine in this article the conceptual and mathematical framework for understanding the organization of plasticity in spiking neural networks.
Network Plasticity as Bayesian Inference
General results from statistical learning theory suggest to understand not only brain computations, but also brain plasticity as probabilistic inference.
