Search Results for author:
Found 13 papers, 6 papers with code
QKFormer: Hierarchical Spiking Transformer using Q-K Attention
ii) We incorporate the hierarchical structure, which significantly benefits the performance of both the brain and artificial neural networks, into spiking transformers to obtain multi-scale spiking representation.
Enhancing EEG-to-Text Decoding through Transferable Representations from Pre-trained Contrastive EEG-Text Masked Autoencoder
Reconstructing natural language from non-invasive electroencephalography (EEG) holds great promise as a language decoding technology for brain-computer interfaces (BCIs).
Deep recurrent spiking neural networks capture both static and dynamic representations of the visual cortex under movie stimuli
Taken together, our work is the first to apply deep recurrent SNNs to model the mouse visual cortex under movie stimuli and we establish that these networks are competent to capture both static and dynamic representations and make contributions to understanding the movie information processing mechanisms of the visual cortex.
Auto-Spikformer: Spikformer Architecture Search
The integration of self-attention mechanisms into Spiking Neural Networks (SNNs) has garnered considerable interest in the realm of advanced deep learning, primarily due to their biological properties.
Temporal Contrastive Learning for Spiking Neural Networks
However, in this work, we find the method above is not ideal for the SNNs training as it omits the temporal dynamics of SNNs and degrades the performance quickly with the decrease of inference time steps.
Enhancing the Performance of Transformer-based Spiking Neural Networks by SNN-optimized Downsampling with Precise Gradient Backpropagation
In this paper, we propose ConvBN-MaxPooling-LIF (CML), an SNN-optimized downsampling with precise gradient backpropagation.
Binary stochasticity enabled highly efficient neuromorphic deep learning achieves better-than-software accuracy
We propose a binary stochastic learning algorithm that modifies all elementary neural network operations, by introducing (i) stochastic binarization of both the forwarding signals and the activation function derivatives, (ii) signed binarization of the backpropagating errors, and (iii) step-wised weight updates.
Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies
Vanilla spiking neurons in Spiking Neural Networks (SNNs) use charge-fire-reset neuronal dynamics, which can only be simulated serially and can hardly learn long-time dependencies.
Spikingformer: Spike-driven Residual Learning for Transformer-based Spiking Neural Network
Based on this residual design, we develop Spikingformer, a pure transformer-based spiking neural network.
Deep Spiking Neural Networks with High Representation Similarity Model Visual Pathways of Macaque and Mouse
However, they highly simplify the computational properties of neurons compared to their biological counterparts.
The style transformer with common knowledge optimization for image-text retrieval
The main module is the common knowledge adaptor (CKA) with both the style embedding extractor (SEE) and the common knowledge optimization (CKO) modules.
A Unified Framework for Soft Threshold Pruning
In this work, we reformulate soft threshold pruning as an implicit optimization problem solved using the Iterative Shrinkage-Thresholding Algorithm (ISTA), a classic method from the fields of sparse recovery and compressed sensing.
Mining Data from the Congressional Record
We propose a data storage and analysis method for using the US Congressional record as a policy analysis tool.
