Search Results for author:
Found 41 papers, 17 papers with code
A Grid Cell-Inspired Structured Vector Algebra for Cognitive Maps
The entorhinal-hippocampal formation is the mammalian brain's navigation system, encoding both physical and abstract spaces via grid cells.
A Truly Sparse and General Implementation of Gradient-Based Synaptic Plasticity
We demonstrate the alignment of our gradients with respect to gradient backpropagation on an synthetic task where e-prop gradients are exact, as well as audio speech classification benchmarks.
Optimal Gradient Checkpointing for Sparse and Recurrent Architectures using Off-Chip Memory
Recurrent neural networks (RNNs) are valued for their computational efficiency and reduced memory requirements on tasks involving long sequence lengths but require high memory-processor bandwidth to train.
Zero-Shot Temporal Resolution Domain Adaptation for Spiking Neural Networks
However, SNN model parameters are sensitive to temporal resolution, leading to significant performance drops when the temporal resolution of target data at the edge is not the same with that of the pre-deployment source data used for training, especially when fine-tuning is not possible at the edge.
Unsupervised Learning of Spatio-Temporal Patterns in Spiking Neuronal Networks
The ability to predict future events or patterns based on previous experience is crucial for many applications such as traffic control, weather forecasting, or supply chain management.
On-Chip Learning via Transformer In-Context Learning
Autoregressive decoder-only transformers have become key components for scalable sequence processing and generation models.
Analog In-Memory Computing Attention Mechanism for Fast and Energy-Efficient Large Language Models
However, GPU-stored projections must be loaded into SRAM for each new generation step, causing latency and energy bottlenecks.
SNNAX -- Spiking Neural Networks in JAX
Spiking Neural Networks (SNNs) simulators are essential tools to prototype biologically inspired models and neuromorphic hardware architectures and predict their performance.
Emulating Brain-like Rapid Learning in Neuromorphic Edge Computing
In this work, we emulate the multiple stages of learning with digital neuromorphic technology that simulates the neural and synaptic processes of the brain using two stages of learning.
Optimizing Automatic Differentiation with Deep Reinforcement Learning
In this paper, we present a novel method to optimize the number of necessary multiplications for Jacobian computation by leveraging deep reinforcement learning (RL) and a concept called cross-country elimination while still computing the exact Jacobian.
Distributed Representations Enable Robust Multi-Timescale Symbolic Computation in Neuromorphic Hardware
To address this, we describe a single-shot weight learning scheme to embed robust multi-timescale dynamics into attractor-based RSNNs, by exploiting the properties of high-dimensional distributed representations.
A Hybrid SNN-ANN Network for Event-based Object Detection with Spatial and Temporal Attention
Event cameras offer high temporal resolution and dynamic range with minimal motion blur, making them promising for object detection tasks.
Harnessing Manycore Processors with Distributed Memory for Accelerated Training of Sparse and Recurrent Models
To overcome this limitation, we explore sparse and recurrent model training on a massively parallel multiple instruction multiple data (MIMD) architecture with distributed local memory.
Design Principles for Lifelong Learning AI Accelerators
Lifelong learning - an agent's ability to learn throughout its lifetime - is a hallmark of biological learning systems and a central challenge for artificial intelligence (AI).
Understanding and Improving Optimization in Predictive Coding Networks
Backpropagation (BP), the standard learning algorithm for artificial neural networks, is often considered biologically implausible.
NeuroBench: A Framework for Benchmarking Neuromorphic Computing Algorithms and Systems
To address these shortcomings, we present NeuroBench: a benchmark framework for neuromorphic computing algorithms and systems.
Online Transformers with Spiking Neurons for Fast Prosthetic Hand Control
However, the self-attention mechanism often used in Transformers requires large time windows for each computation step and thus makes them less suitable for online signal processing compared to Recurrent Neural Networks (RNNs).
A Theoretical Framework for Inference Learning
Our novel implementation considerably improves the stability of IL across learning rates, which is consistent with our theory, as a key property of implicit SGD is its stability.
Policy Distillation with Selective Input Gradient Regularization for Efficient Interpretability
Although deep Reinforcement Learning (RL) has proven successful in a wide range of tasks, one challenge it faces is interpretability when applied to real-world problems.
Meta-learning Spiking Neural Networks with Surrogate Gradient Descent
In this work, we demonstrate gradient-based meta-learning in SNNs using the surrogate gradient method that approximates the spiking threshold function for gradient estimations.
Encoding Event-Based Gesture Data With a Hybrid SNN Guided Variational Auto-encoder
Our novel approach consists of an event-based guided Variational Autoencoder (VAE) which encodes event-based data sensed by a Dynamic Vision Sensor (DVS) into a latent space representation suitable to compute the similarity of mid-air gesture data.
Training Spiking Neural Networks Using Lessons From Deep Learning
This paper serves as a tutorial and perspective showing how to apply the lessons learnt from several decades of research in deep learning, gradient descent, backpropagation and neuroscience to biologically plausible spiking neural neural networks.
Tightening the Biological Constraints on Gradient-Based Predictive Coding
This finding suggests that this gradient-based PC model may be useful for understanding how the brain solves the credit assignment problem.
Hessian Aware Quantization of Spiking Neural Networks
To address this challenge, we present a simplified neuron model that reduces the number of state variables by 4-fold while still being compatible with gradient based training.
Encoding Event-Based Data With a Hybrid SNN Guided Variational Auto-encoder in Neuromorphic Hardware
We also implement the encoder component of the model on neuromorphic hardware and discuss the potential for our algorithm to enable real-time learning from real-world event data.
Neural Sampling Machine with Stochastic Synapse allows Brain-like Learning and Inference
We experimentally show that the inherent stochastic switching of the selector element between the insulator and metallic state introduces a multiplicative stochastic noise within the synapses of NSM that samples the conductance states of the FeFET, both during learning and inference.
Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation
To address this issue, we propose a two-stage RL agent that first learns a latent unified state representation (LUSR) which is consistent across multiple domains in the first stage, and then do RL training in one source domain based on LUSR in the second stage.
Online Few-shot Gesture Learning on a Neuromorphic Processor
We present the Surrogate-gradient Online Error-triggered Learning (SOEL) system for online few-shot learning on neuromorphic processors.
Inherent Weight Normalization in Stochastic Neural Networks
Multiplicative stochasticity such as Dropout improves the robustness and generalizability of deep neural networks.
On-chip Few-shot Learning with Surrogate Gradient Descent on a Neuromorphic Processor
Recent work suggests that synaptic plasticity dynamics in biological models of neurons and neuromorphic hardware are compatible with gradient-based learning (Neftci et al., 2019).
Embodied Neuromorphic Vision with Event-Driven Random Backpropagation
In this setup, visual information is actively sensed by a DVS mounted on a robotic head performing microsaccadic eye movements.
Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
A relatively smaller body of work, however, discusses similarities between learning dynamics employed in deep artificial neural networks and synaptic plasticity in spiking neural networks.
Contrastive Hebbian Learning with Random Feedback Weights
It operates in two phases, the forward (or free) phase, where the data are fed to the network, and a backward (or clamped) phase, where the target signals are clamped to the output layer of the network and the feedback signals are transformed through the transpose synaptic weight matrices.
Neural and Synaptic Array Transceiver: A Brain-Inspired Computing Framework for Embedded Learning
Embedded, continual learning for autonomous and adaptive behavior is a key application of neuromorphic hardware.
Neuromorphic Deep Learning Machines
Building on these results, we demonstrate an event-driven random BP (eRBP) rule that uses an error-modulated synaptic plasticity for learning deep representations in neuromorphic computing hardware.
Training a Probabilistic Graphical Model with Resistive Switching Electronic Synapses
Current large scale implementations of deep learning and data mining require thousands of processors, massive amounts of off-chip memory, and consume gigajoules of energy.
Forward Table-Based Presynaptic Event-Triggered Spike-Timing-Dependent Plasticity
We present a novel method for realizing both causal and acausal weight updates using only forward lookup access of the synaptic connectivity table, permitting memory-efficient implementation.
TrueHappiness: Neuromorphic Emotion Recognition on TrueNorth
We present an approach to constructing a neuromorphic device that responds to language input by producing neuron spikes in proportion to the strength of the appropriate positive or negative emotional response.
Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware
We find that short synaptic delays are sufficient to implement the dynamical (temporal) aspect of the RNN in the question classification task.
Learning Non-deterministic Representations with Energy-based Ensembles
These mappings are encoded in a distribution over a (possibly infinite) collection of models.
Event-Driven Contrastive Divergence for Spiking Neuromorphic Systems
However the traditional RBM architecture and the commonly used training algorithm known as Contrastive Divergence (CD) are based on discrete updates and exact arithmetics which do not directly map onto a dynamical neural substrate.
