Search Results for author: Rui-Jie Zhu

Found 18 papers, 10 papers with code

ARFlow: Autogressive Flow with Hybrid Linear Attention

no code implementations27 Jan 2025 Mude Hui, Rui-Jie Zhu, Songlin Yang, Yu Zhang, ZiRui Wang, Yuyin Zhou, Jason Eshraghian, Cihang Xie

Flow models are effective at progressively generating realistic images, but they generally struggle to capture long-range dependencies during the generation process as they compress all the information from previous time steps into a single corrupted image.

Computational Efficiency Denoising

Quantized Spike-driven Transformer

1 code implementation23 Jan 2025 Xuerui Qiu, Jieyuan Zhang, Wenjie Wei, Honglin Cao, Junsheng Guo, Rui-Jie Zhu, Yimeng Shan, Yang Yang, Malu Zhang, Haizhou Li

To mitigate this issue, we take inspiration from mutual information entropy and propose a bi-level optimization strategy to rectify the information distribution in Q-SDSA.

Quantization

Future-Guided Learning: A Predictive Approach To Enhance Time-Series Forecasting

no code implementations19 Oct 2024 Skye Gunasekaran, Assel Kembay, Hugo Ladret, Rui-Jie Zhu, Laurent Perrinet, Omid Kavehei, Jason Eshraghian

By incorporating a predictive feedback mechanism that adapts to data distribution drift, Future-Guided Learning offers a promising avenue for advancing time-series forecasting with deep learning.

Seizure prediction Time Series +1

Reducing Data Bottlenecks in Distributed, Heterogeneous Neural Networks

no code implementations12 Oct 2024 Ruhai Lin, Rui-Jie Zhu, Jason K. Eshraghian

Through this research, we can determine the trade-off between data transfer volume and model performance, enabling the identification of a balanced point that achieves good performance while minimizing data transfer volume.

Inner-Probe: Discovering Copyright-related Data Generation in LLM Architecture

no code implementations6 Oct 2024 Qichao Ma, Rui-Jie Zhu, Peiye Liu, Renye Yan, Fahong Zhang, Ling Liang, Meng Li, Zhaofei Yu, Zongwei Wang, Yimao Cai, Tiejun Huang

Thus, InnerProbe performs sub-dataset contribution analysis using a lightweight LSTM-based network trained on MHA results in a supervised manner.

Contrastive Learning Prompt Engineering +2

When Spiking neural networks meet temporal attention image decoding and adaptive spiking neuron

1 code implementation5 Jun 2024 Xuerui Qiu, Zheng Luan, Zhaorui Wang, Rui-Jie Zhu

Furthermore, our ALIF neuron model achieves remarkable classification accuracy on MNIST (99. 78\%) and CIFAR-10 (93. 89\%) datasets, demonstrating the effectiveness of learning adaptive thresholds for spiking neurons.

Scalable MatMul-free Language Modeling

1 code implementation4 Jun 2024 Rui-Jie Zhu, Yu Zhang, Ethan Sifferman, Tyler Sheaves, Yiqiao Wang, Dustin Richmond, Peng Zhou, Jason K. Eshraghian

Our experiments show that our proposed MatMul-free models achieve performance on-par with state-of-the-art Transformers that require far more memory during inference at a scale up to at least 2. 7B parameters.

Language Modeling Language Modelling

Autonomous Driving with Spiking Neural Networks

1 code implementation30 May 2024 Rui-Jie Zhu, Ziqing Wang, Leilani Gilpin, Jason K. Eshraghian

Autonomous driving demands an integrated approach that encompasses perception, prediction, and planning, all while operating under strict energy constraints to enhance scalability and environmental sustainability.

Autonomous Driving Prediction

Advancing Spiking Neural Networks towards Multiscale Spatiotemporal Interaction Learning

no code implementations22 May 2024 Yimeng Shan, Malu Zhang, Rui-Jie Zhu, Xuerui Qiu, Jason K. Eshraghian, Haicheng Qu

To address this issue, we have designed a Spiking Multiscale Attention (SMA) module that captures multiscale spatiotemporal interaction information.

Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology

no code implementations2 Dec 2023 Souvik Kundu, Rui-Jie Zhu, Akhilesh Jaiswal, Peter A. Beerel

Neuromorphic computing and, in particular, spiking neural networks (SNNs) have become an attractive alternative to deep neural networks for a broad range of signal processing applications, processing static and/or temporal inputs from different sensory modalities, including audio and vision sensors.

SynA-ResNet: Spike-driven ResNet Achieved through OR Residual Connection

1 code implementation11 Nov 2023 Yimeng Shan, Xuerui Qiu, Rui-Jie Zhu, Jason K. Eshraghian, Malu Zhang, Haicheng Qu

As the demand for heightened performance in SNNs surges, the trend towards training deeper networks becomes imperative, while residual learning stands as a pivotal method for training deep neural networks.

Quantization

Both Efficiency and Effectiveness! A Large Scale Pre-ranking Framework in Search System

no code implementations5 Apr 2023 Qihang Zhao, Rui-Jie Zhu, Liu Yang, He Yongming, Bo Zhou, Luo Cheng

In the realm of search systems, multi-stage cascade architecture is a prevalent method, typically consisting of sequential modules such as matching, pre-ranking, and ranking.

feature selection

SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks

1 code implementation27 Feb 2023 Rui-Jie Zhu, Qihang Zhao, Guoqi Li, Jason K. Eshraghian

As a result, their performance lags behind modern deep learning, and we are yet to see the effectiveness of SNNs in language generation.

Language Modeling Language Modelling +1

TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks

1 code implementation21 Jun 2022 Rui-Jie Zhu, Malu Zhang, Qihang Zhao, Haoyu Deng, Yule Duan, Liang-Jian Deng

Given the critical role of attention mechanisms in enhancing neural network performance, the integration of SNNs and attention mechanisms exhibits potential to deliver energy-efficient and high-performance computing paradigms.

Image Classification Image Generation

SIT: A Bionic and Non-Linear Neuron for Spiking Neural Network

no code implementations30 Mar 2022 Cheng Jin, Rui-Jie Zhu, Xiao Wu, Liang-Jian Deng

Spiking Neural Networks (SNNs) have piqued researchers' interest because of their capacity to process temporal information and low power consumption.

Image Classification

Cannot find the paper you are looking for? You can Submit a new open access paper.