Search Results for author: Yonghong Tian

Found 120 papers, 65 papers with code

SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence

1 code implementation25 Oct 2023 Wei Fang, Yanqi Chen, Jianhao Ding, Zhaofei Yu, Timothée Masquelier, Ding Chen, Liwei Huang, Huihui Zhou, Guoqi Li, Yonghong Tian

Spiking neural networks (SNNs) aim to realize brain-inspired intelligence on neuromorphic chips with high energy efficiency by introducing neural dynamics and spike properties.

Code Generation

HiFi-123: Towards High-fidelity One Image to 3D Content Generation

no code implementations10 Oct 2023 Wangbo Yu, Li Yuan, Yan-Pei Cao, Xiangjun Gao, Xiaoyu Li, Long Quan, Ying Shan, Yonghong Tian

Recent advances in text-to-image diffusion models have enabled 3D generation from a single image.

Image to 3D

Knowledge Prompt-tuning for Sequential Recommendation

1 code implementation14 Aug 2023 Jianyang Zhai, Xiawu Zheng, Chang-Dong Wang, Hui Li, Yonghong Tian

Pre-trained language models (PLMs) have demonstrated strong performance in sequential recommendation (SR), which are utilized to extract general knowledge.

General Knowledge Sequential Recommendation

SODFormer: Streaming Object Detection with Transformer Using Events and Frames

1 code implementation8 Aug 2023 Dianze Li, Jianing Li, Yonghong Tian

Then, we design a spatiotemporal Transformer architecture to detect objects via an end-to-end sequence prediction problem, where the novel temporal Transformer module leverages rich temporal cues from two visual streams to improve the detection performance.

object-detection Object Detection

SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition

1 code implementation8 Aug 2023 Xiao Wang, Zongzhen Wu, Yao Rong, Lin Zhu, Bo Jiang, Jin Tang, Yonghong Tian

Secondly, they adopt either Spiking Neural Networks (SNN) for energy-efficient recognition with suboptimal results, or Artificial Neural Networks (ANN) for energy-intensive, high-performance recognition.

Learning Sparse Neural Networks with Identity Layers

no code implementations14 Jul 2023 Mingjian Ni, Guangyao Chen, Xiawu Zheng, Peixi Peng, Li Yuan, Yonghong Tian

Applying such theory, we propose a plug-and-play CKA-based Sparsity Regularization for sparse network training, dubbed CKA-SR, which utilizes CKA to reduce feature similarity between layers and increase network sparsity.

Dual Adaptive Representation Alignment for Cross-domain Few-shot Learning

1 code implementation18 Jun 2023 Yifan Zhao, Tong Zhang, Jia Li, Yonghong Tian

Recent progress in this setting assumes that the base knowledge and novel query samples are distributed in the same domains, which are usually infeasible for realistic applications.

cross-domain few-shot learning

Population-Based Evolutionary Gaming for Unsupervised Person Re-identification

no code implementations8 Jun 2023 Yunpeng Zhai, Peixi Peng, Mengxi Jia, Shiyong Li, Weiqiang Chen, Xuesong Gao, Yonghong Tian

Extensive experiments demonstrate that (1) CRS approximately measures the performance of models without labeled samples; (2) and PEG produces new state-of-the-art accuracy for person re-identification, indicating the great potential of population-based network cooperative training for unsupervised learning.

Knowledge Distillation Unsupervised Person Re-Identification

Point-Voxel Absorbing Graph Representation Learning for Event Stream based Recognition

1 code implementation8 Jun 2023 Bo Jiang, Chengguo Yuan, Xiao Wang, Zhimin Bao, Lin Zhu, Yonghong Tian, Jin Tang

To address these issues, we propose a novel dual point-voxel absorbing graph representation learning for event stream data representation.

Event data classification Graph Representation Learning

Deep recurrent spiking neural networks capture both static and dynamic representations of the visual cortex under movie stimuli

no code implementations2 Jun 2023 Liwei Huang, Zhengyu Ma, Huihui Zhou, Yonghong Tian

Taken together, our work is the first to apply deep recurrent SNNs to model the mouse visual cortex under movie stimuli and we establish that these networks are competent to capture both static and dynamic representations and make contributions to understanding the movie information processing mechanisms of the visual cortex.

Action Recognition Image Classification +2

Auto-Spikformer: Spikformer Architecture Search

no code implementations1 Jun 2023 Kaiwei Che, Zhaokun Zhou, Zhengyu Ma, Wei Fang, Yanqi Chen, Shuaijie Shen, Li Yuan, Yonghong Tian

The integration of self-attention mechanisms into Spiking Neural Networks (SNNs) has garnered considerable interest in the realm of advanced deep learning, primarily due to their biological properties.

Temporal Contrastive Learning for Spiking Neural Networks

no code implementations23 May 2023 Haonan Qiu, Zeyin Song, Yanqi Chen, Munan Ning, Wei Fang, Tao Sun, Zhengyu Ma, Li Yuan, Yonghong Tian

However, in this work, we find the method above is not ideal for the SNNs training as it omits the temporal dynamics of SNNs and degrades the performance quickly with the decrease of inference time steps.

Contrastive Learning

Album Storytelling with Iterative Story-aware Captioning and Large Language Models

no code implementations22 May 2023 Munan Ning, Yujia Xie, Dongdong Chen, Zeyin Song, Lu Yuan, Yonghong Tian, Qixiang Ye, Li Yuan

One natural approach is to use caption models to describe each photo in the album, and then use LLMs to summarize and rewrite the generated captions into an engaging story.

Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies

1 code implementation25 Apr 2023 Wei Fang, Zhaofei Yu, Zhaokun Zhou, Ding Chen, Yanqi Chen, Zhengyu Ma, Timothée Masquelier, Yonghong Tian

Vanilla spiking neurons in Spiking Neural Networks (SNNs) use charge-fire-reset neuronal dynamics, which can only be simulated serially and can hardly learn long-time dependencies.

Event data classification Image Classification

Spikingformer: Spike-driven Residual Learning for Transformer-based Spiking Neural Network

1 code implementation24 Apr 2023 Chenlin Zhou, Liutao Yu, Zhaokun Zhou, Zhengyu Ma, Han Zhang, Huihui Zhou, Yonghong Tian

Based on this residual design, we develop Spikingformer, a pure transformer-based spiking neural network.

Picking Up Quantization Steps for Compressed Image Classification

1 code implementation21 Apr 2023 Li Ma, Peixi Peng, Guangyao Chen, Yifan Zhao, Siwei Dong, Yonghong Tian

The sensitivity of deep neural networks to compressed images hinders their usage in many real applications, which means classification networks may fail just after taking a screenshot and saving it as a compressed file.

Classification Image Classification +1

Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning

1 code implementation CVPR 2023 Zeyin Song, Yifan Zhao, Yujun Shi, Peixi Peng, Li Yuan, Yonghong Tian

However, in this work, we find that the CE loss is not ideal for the base session training as it suffers poor class separation in terms of representations, which further degrades generalization to novel classes.

Contrastive Learning Few-Shot Class-Incremental Learning +1

Deep Spiking Neural Networks with High Representation Similarity Model Visual Pathways of Macaque and Mouse

1 code implementation9 Mar 2023 Liwei Huang, Zhengyu Ma, Liutao Yu, Huihui Zhou, Yonghong Tian

However, they highly simplify the computational properties of neurons compared to their biological counterparts.

A Unified Framework for Soft Threshold Pruning

1 code implementation25 Feb 2023 Yanqi Chen, Zhengyu Ma, Wei Fang, Xiawu Zheng, Zhaofei Yu, Yonghong Tian

In this work, we reformulate soft threshold pruning as an implicit optimization problem solved using the Iterative Shrinkage-Thresholding Algorithm (ISTA), a classic method from the fields of sparse recovery and compressed sensing.


Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey

1 code implementation20 Feb 2023 Xiao Wang, Guangyao Chen, Guangwu Qian, Pengcheng Gao, Xiao-Yong Wei, YaoWei Wang, Yonghong Tian, Wen Gao

With the urgent demand for generalized deep models, many pre-trained big models are proposed, such as BERT, ViT, GPT, etc.

Training Full Spike Neural Networks via Auxiliary Accumulation Pathway

2 code implementations27 Jan 2023 Guangyao Chen, Peixi Peng, Guoqi Li, Yonghong Tian

The accumulation in AAP could compensate for the information loss during the forward and backward of full spike propagation, and facilitate the training of the FSNN.

Parsing Objects at a Finer Granularity: A Survey

no code implementations28 Dec 2022 Yifan Zhao, Jia Li, Yonghong Tian

Fine-grained visual parsing, including fine-grained part segmentation and fine-grained object recognition, has attracted considerable critical attention due to its importance in many real-world applications, e. g., agriculture, remote sensing, and space technologies.

Object Recognition Segmentation

Part-guided Relational Transformers for Fine-grained Visual Recognition

1 code implementation28 Dec 2022 Yifan Zhao, Jia Li, Xiaowu Chen, Yonghong Tian

This framework, namely PArt-guided Relational Transformers (PART), is proposed to learn the discriminative part features with an automatic part discovery module, and to explore the intrinsic correlations with a feature transformation module by adapting the Transformer models from the field of natural language processing.

Fine-Grained Image Classification Fine-Grained Visual Recognition +1

Universal Object Detection with Large Vision Model

1 code implementation19 Dec 2022 Feng Lin, Wenze Hu, YaoWei Wang, Yonghong Tian, Guangming Lu, Fanglin Chen, Yong Xu, Xiaoyu Wang

In this study, our focus is on a specific challenge: the large-scale, multi-domain universal object detection problem, which contributes to the broader goal of achieving a universal vision system.

object-detection Object Detection

Shadow Removal by High-Quality Shadow Synthesis

1 code implementation8 Dec 2022 Yunshan Zhong, Lizhou You, Yuxin Zhang, Fei Chao, Yonghong Tian, Rongrong Ji

Specifically, the encoder extracts the shadow feature of a region identity which is then paired with another region identity to serve as the generator input to synthesize a pseudo image.

Image Generation Shadow Removal +1

Event-based Monocular Dense Depth Estimation with Recurrent Transformers

no code implementations6 Dec 2022 Xu Liu, Jianing Li, Xiaopeng Fan, Yonghong Tian

Event cameras, offering high temporal resolutions and high dynamic ranges, have brought a new perspective to address common challenges (e. g., motion blur and low light) in monocular depth estimation.

Event-based vision Monocular Depth Estimation

Meta Architecture for Point Cloud Analysis

1 code implementation CVPR 2023 Haojia Lin, Xiawu Zheng, Lijiang Li, Fei Chao, Shanshan Wang, Yan Wang, Yonghong Tian, Rongrong Ji

However, the lack of a unified framework to interpret those networks makes any systematic comparison, contrast, or analysis challenging, and practically limits healthy development of the field.

Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric

1 code implementation20 Nov 2022 Chuanming Tang, Xiao Wang, Ju Huang, Bo Jiang, Lin Zhu, Jianlin Zhang, YaoWei Wang, Yonghong Tian

In this paper, we propose a single-stage backbone network for Color-Event Unified Tracking (CEUTrack), which achieves the above functions simultaneously.

Object Localization Object Tracking

HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors

2 code implementations17 Nov 2022 Xiao Wang, Zongzhen Wu, Bo Jiang, Zhimin Bao, Lin Zhu, Guoqi Li, YaoWei Wang, Yonghong Tian

The main streams of human activity recognition (HAR) algorithms are developed based on RGB cameras which are suffered from illumination, fast motion, privacy-preserving, and large energy consumption.

Activity Prediction Human Activity Recognition +1

Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets Self-similarity

no code implementations2 Nov 2022 Yi Chang, Yun Guo, Yuntong Ye, Changfeng Yu, Lin Zhu, XiLe Zhao, Luxin Yan, Yonghong Tian

In addition, considering that the existing real rain datasets are of low quality, either small scale or downloaded from the internet, we collect a real large-scale dataset under various rainy kinds of weather that contains high-resolution rainy images.

Contrastive Learning Rain Removal

SNN-SC: A Spiking Semantic Communication Framework for Feature Transmission

1 code implementation13 Oct 2022 Mengyang Wang, Jiahui Li, Mengyao Ma, Xiaopeng Fan, Yonghong Tian

However, most of the existing SC works only transmit analog information on the AWGN channel and cannot be directly used for digital channels.

Classification Quantization

Spikformer: When Spiking Neural Network Meets Transformer

1 code implementation29 Sep 2022 Zhaokun Zhou, Yuesheng Zhu, Chao He, YaoWei Wang, Shuicheng Yan, Yonghong Tian, Li Yuan

Spikformer (66. 3M parameters) with comparable size to SEW-ResNet-152 (60. 2M, 69. 26%) can achieve 74. 81% top1 accuracy on ImageNet using 4 time steps, which is the state-of-the-art in directly trained SNNs models.

Image Classification

Attention Spiking Neural Networks

no code implementations28 Sep 2022 Man Yao, Guangshe Zhao, Hengyu Zhang, Yifan Hu, Lei Deng, Yonghong Tian, Bo Xu, Guoqi Li

On ImageNet-1K, we achieve top-1 accuracy of 75. 92% and 77. 08% on single/4-step Res-SNN-104, which are state-of-the-art results in SNNs.

Action Recognition Image Classification

Fine-Grained Object Classification via Self-Supervised Pose Alignment

2 code implementations CVPR 2022 Xuhui Yang, YaoWei Wang, Ke Chen, Yong Xu, Yonghong Tian

Semantic patterns of fine-grained objects are determined by subtle appearance difference of local parts, which thus inspires a number of part-based methods.

Classification Representation Learning

Training-free Transformer Architecture Search

1 code implementation CVPR 2022 Qinqin Zhou, Kekai Sheng, Xiawu Zheng, Ke Li, Xing Sun, Yonghong Tian, Jie Chen, Rongrong Ji

Recently, Vision Transformer (ViT) has achieved remarkable success in several computer vision tasks.

Masked Autoencoders for Point Cloud Self-supervised Learning

1 code implementation13 Mar 2022 Yatian Pang, Wenxiao Wang, Francis E. H. Tay, Wei Liu, Yonghong Tian, Li Yuan

Then, a standard Transformer based autoencoder, with an asymmetric design and a shifting mask tokens operation, learns high-level latent features from unmasked point patches, aiming to reconstruct the masked point patches.

3D Part Segmentation Few-Shot 3D Point Cloud Classification +2

Annotation Efficient Person Re-Identification with Diverse Cluster-Based Pair Selection

no code implementations10 Mar 2022 Lantian Xue, Yixiong Zou, Peixi Peng, Yonghong Tian, Tiejun Huang

To solve this problem, we propose the Annotation Efficient Person Re-Identification method to select image pairs from an alternative pair set according to the fallibility and diversity of pairs, and train the Re-ID model based on the annotation.

Clustering Person Re-Identification

PowerGear: Early-Stage Power Estimation in FPGA HLS via Heterogeneous Edge-Centric GNNs

1 code implementation25 Jan 2022 Zhe Lin, Zike Yuan, Jieru Zhao, Wei zhang, Hui Wang, Yonghong Tian

Specifically, in the graph construction flow, we introduce buffer insertion, datapath merging, graph trimming and feature annotation techniques to transform HLS designs into graph-structured data, which encode both intra-operation micro-architectures and inter-operation interconnects annotated with switching activities.

graph construction Graph Learning

Event-based Video Reconstruction via Potential-assisted Spiking Neural Network

1 code implementation CVPR 2022 Lin Zhu, Xiao Wang, Yi Chang, Jianing Li, Tiejun Huang, Yonghong Tian

We propose a novel Event-based Video reconstruction framework based on a fully Spiking Neural Network (EVSNN), which utilizes Leaky-Integrate-and-Fire (LIF) neuron and Membrane Potential (MP) neuron.

Image Reconstruction Video Reconstruction

1000x Faster Camera and Machine Vision with Ordinary Devices

no code implementations23 Jan 2022 Tiejun Huang, Yajing Zheng, Zhaofei Yu, Rui Chen, Yuan Li, Ruiqin Xiong, Lei Ma, Junwei Zhao, Siwei Dong, Lin Zhu, Jianing Li, Shanshan Jia, Yihua Fu, Boxin Shi, Si Wu, Yonghong Tian

By treating vidar as spike trains in biological vision, we have further developed a spiking neural network-based machine vision system that combines the speed of the machine and the mechanism of biological vision, achieving high-speed object detection and tracking 1, 000x faster than human vision.

object-detection Object Detection

Deep Reinforcement Learning with Spiking Q-learning

no code implementations21 Jan 2022 Ding Chen, Peixi Peng, Tiejun Huang, Yonghong Tian

With the help of special neuromorphic hardware, spiking neural networks (SNNs) are expected to realize artificial intelligence with less energy consumption.

Atari Games Q-Learning +2

Neural Architecture Search With Representation Mutual Information

1 code implementation CVPR 2022 Xiawu Zheng, Xiang Fei, Lei Zhang, Chenglin Wu, Fei Chao, Jianzhuang Liu, Wei Zeng, Yonghong Tian, Rongrong Ji

Building upon RMI, we further propose a new search algorithm termed RMI-NAS, facilitating with a theorem to guarantee the global optimal of the searched architecture.

Neural Architecture Search

IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization

1 code implementation CVPR 2022 Yunshan Zhong, Mingbao Lin, Gongrui Nan, Jianzhuang Liu, Baochang Zhang, Yonghong Tian, Rongrong Ji

In this paper, we observe an interesting phenomenon of intra-class heterogeneity in real data and show that existing methods fail to retain this property in their synthetic images, which causes a limited performance increase.


Optimized Separable Convolution: Yet Another Efficient Convolution Operator

no code implementations29 Sep 2021 Tao Wei, Yonghong Tian, YaoWei Wang, Yun Liang, Chang Wen Chen

In this research, we propose a novel and principled operator called optimized separable convolution by optimal design for the internal number of groups and kernel sizes for general separable convolutions can achieve the complexity of O(C^{\frac{3}{2}}K).

Heterogeneous Relational Complement for Vehicle Re-identification

1 code implementation ICCV 2021 Jiajian Zhao, Yifan Zhao, Jia Li, Ke Yan, Yonghong Tian

The crucial problem in vehicle re-identification is to find the same vehicle identity when reviewing this object from cross-view cameras, which sets a higher demand for learning viewpoint-invariant representations.

Vehicle Re-Identification

Amplitude-Phase Recombination: Rethinking Robustness of Convolutional Neural Networks in Frequency Domain

1 code implementation ICCV 2021 Guangyao Chen, Peixi Peng, Li Ma, Jia Li, Lin Du, Yonghong Tian

This observation leads to more explanations of the CNN's generalization behaviors in both robustness to common perturbations and out-of-distribution detection, and motivates a new perspective on data augmentation designed by re-combing the phase spectrum of the current image and the amplitude spectrum of the distracter image.

Adversarial Attack Data Augmentation +2

An Information Theory-inspired Strategy for Automatic Network Pruning

1 code implementation19 Aug 2021 Xiawu Zheng, Yuexiao Ma, Teng Xi, Gang Zhang, Errui Ding, Yuchao Li, Jie Chen, Yonghong Tian, Rongrong Ji

This practically limits the application of model compression when the model needs to be deployed on a wide range of devices.

AutoML Model Compression +1

VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows

2 code implementations11 Aug 2021 Xiao Wang, Jianing Li, Lin Zhu, Zhipeng Zhang, Zhe Chen, Xin Li, YaoWei Wang, Yonghong Tian, Feng Wu

Different from visible cameras which record intensity images frame by frame, the biologically inspired event camera produces a stream of asynchronous and sparse events with much lower latency.

Object Tracking

MFGNet: Dynamic Modality-Aware Filter Generation for RGB-T Tracking

2 code implementations22 Jul 2021 Xiao Wang, Xiujun Shu, Shiliang Zhang, Bo Jiang, YaoWei Wang, Yonghong Tian, Feng Wu

The visible and thermal filters will be used to conduct a dynamic convolutional operation on their corresponding input feature maps respectively.

Rgb-T Tracking

High-Speed Image Reconstruction Through Short-Term Plasticity for Spiking Cameras

no code implementations CVPR 2021 Yajing Zheng, Lingxiao Zheng, Zhaofei Yu, Boxin Shi, Yonghong Tian, Tiejun Huang

Mimicking the sampling mechanism of the fovea, a retina-inspired camera, named spiking camera, is developed to record the external information with a sampling rate of 40, 000 Hz, and outputs asynchronous binary spike streams.

Image Reconstruction Vocal Bursts Intensity Prediction

Tracking by Joint Local and Global Search: A Target-aware Attention based Approach

1 code implementation9 Jun 2021 Xiao Wang, Jin Tang, Bin Luo, YaoWei Wang, Yonghong Tian, Feng Wu

In this paper, we propose a novel and general target-aware attention mechanism (termed TANet) and integrate it with tracking-by-detection framework to conduct joint local and global search for robust tracking.

Object Tracking

1xN Pattern for Pruning Convolutional Neural Networks

1 code implementation31 May 2021 Mingbao Lin, Yuxin Zhang, Yuchao Li, Bohong Chen, Fei Chao, Mengdi Wang, Shen Li, Yonghong Tian, Rongrong Ji

We also provide a workflow of filter rearrangement that first rearranges the weight matrix in the output channel dimension to derive more influential blocks for accuracy improvements and then applies similar rearrangement to the next-layer weights in the input channel dimension to ensure correct convolutional operations.

Network Pruning

Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networks

1 code implementation25 May 2021 Jianhao Ding, Zhaofei Yu, Yonghong Tian, Tiejun Huang

We show that the inference time can be reduced by optimizing the upper bound of the fit curve in the revised ANN to achieve fast inference.

Pruning of Deep Spiking Neural Networks through Gradient Rewiring

1 code implementation11 May 2021 Yanqi Chen, Zhaofei Yu, Wei Fang, Tiejun Huang, Yonghong Tian

Our key innovation is to redefine the gradient to a new synaptic parameter, allowing better exploration of network structures by taking full advantage of the competition between pruning and regrowth of connections.

Carrying out CNN Channel Pruning in a White Box

1 code implementation24 Apr 2021 Yuxin Zhang, Mingbao Lin, Chia-Wen Lin, Jie Chen, Feiyue Huang, Yongjian Wu, Yonghong Tian, Rongrong Ji

Specifically, to model the contribution of each channel to differentiating categories, we develop a class-wise mask for each channel, implemented in a dynamic training manner w. r. t.

Image Classification

Dynamic Attention guided Multi-Trajectory Analysis for Single Object Tracking

1 code implementation30 Mar 2021 Xiao Wang, Zhe Chen, Jin Tang, Bin Luo, YaoWei Wang, Yonghong Tian, Feng Wu

In this paper, we propose to introduce more dynamics by devising a dynamic attention-guided multi-trajectory tracking strategy.

Object Tracking

Distilling a Powerful Student Model via Online Knowledge Distillation

1 code implementation26 Mar 2021 Shaojie Li, Mingbao Lin, Yan Wang, Yongjian Wu, Yonghong Tian, Ling Shao, Rongrong Ji

Besides, a self-distillation module is adopted to convert the feature map of deeper layers into a shallower one.

Knowledge Distillation

Adversarial Reciprocal Points Learning for Open Set Recognition

1 code implementation1 Mar 2021 Guangyao Chen, Peixi Peng, Xiangqian Wang, Yonghong Tian

Then, an adversarial margin constraint is proposed to reduce the open space risk by limiting the latent open space constructed by reciprocal points.

General Classification Open Set Learning

Collaborative Intelligence: Challenges and Opportunities

no code implementations13 Feb 2021 Ivan V. Bajić, Weisi Lin, Yonghong Tian

This paper presents an overview of the emerging area of collaborative intelligence (CI).

Feature Compression

Deep Residual Learning in Spiking Neural Networks

1 code implementation NeurIPS 2021 Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, Yonghong Tian

Previous Spiking ResNet mimics the standard residual block in ANNs and simply replaces ReLU activation layers with spiking neurons, which suffers the degradation problem and can hardly implement residual learning.

Rethinking Convolution: Towards an Optimal Efficiency

no code implementations1 Jan 2021 Tao Wei, Yonghong Tian, Chang Wen Chen

In this research, we propose a novel operator called \emph{optimal separable convolution} which can be calculated at $O(C^{\frac{3}{2}}KHW)$ by optimal design for the internal number of groups and kernel sizes for general separable convolutions.

NeuSpike-Net: High Speed Video Reconstruction via Bio-Inspired Neuromorphic Cameras

no code implementations ICCV 2021 Lin Zhu, Jianing Li, Xiao Wang, Tiejun Huang, Yonghong Tian

In this paper, we propose a NeuSpike-Net to learn both the high dynamic range and high motion sensitivity of DVS and the full texture sampling of spike camera to achieve high-speed and high dynamic image reconstruction.

Image Reconstruction Video Reconstruction +1

Fast Class-wise Updating for Online Hashing

no code implementations1 Dec 2020 Mingbao Lin, Rongrong Ji, Xiaoshuai Sun, Baochang Zhang, Feiyue Huang, Yonghong Tian, DaCheng Tao

To achieve fast online adaptivity, a class-wise updating method is developed to decompose the binary code learning and alternatively renew the hash functions in a class-wise fashion, which well addresses the burden on large amounts of training batches.

Annotation-Efficient Untrimmed Video Action Recognition

no code implementations30 Nov 2020 Yixiong Zou, Shanghang Zhang, Guangyao Chen, Yonghong Tian, Kurt Keutzer, José M. F. Moura

In this paper, we target a new problem, Annotation-Efficient Video Recognition, to reduce the requirement of annotations for both large amount of samples and the action location.

Action Recognition Contrastive Learning +3

Learning Open Set Network with Discriminative Reciprocal Points

1 code implementation ECCV 2020 Guangyao Chen, Limeng Qiao, Yemin Shi, Peixi Peng, Jia Li, Tiejun Huang, ShiLiang Pu, Yonghong Tian

In this process, one of the key challenges is to reduce the risk of generalizing the inherent characteristics of numerous unknown samples learned from a small amount of known data.

Open Set Learning

Intrinsic Relationship Reasoning for Small Object Detection

no code implementations2 Sep 2020 Kui Fu, Jia Li, Lin Ma, Kai Mu, Yonghong Tian

In this paper, we propose a novel context reasoning approach for small object detection which models and infers the intrinsic semantic and spatial layout relationships between objects.

object-detection Small Object Detection

Cooperative Bi-path Metric for Few-shot Learning

1 code implementation10 Aug 2020 Zeyuan Wang, Yifan Zhao, Jia Li, Yonghong Tian

Given base classes with sufficient labeled samples, the target of few-shot classification is to recognize unlabeled samples of novel classes with only a few labeled samples.

Classification Few-Shot Learning +1

Revisiting Mid-Level Patterns for Cross-Domain Few-Shot Recognition

no code implementations7 Aug 2020 Yixiong Zou, Shanghang Zhang, JianPeng Yu, Yonghong Tian, José M. F. Moura

To solve this problem, cross-domain FSL (CDFSL) is proposed very recently to transfer knowledge from general-domain base classes to special-domain novel classes.

cross-domain few-shot learning

Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks

1 code implementation ICCV 2021 Wei Fang, Zhaofei Yu, Yanqi Chen, Timothee Masquelier, Tiejun Huang, Yonghong Tian

In this paper, we take inspiration from the observation that membrane-related parameters are different across brain regions, and propose a training algorithm that is capable of learning not only the synaptic weights but also the membrane time constants of SNNs.

Image Classification

Multiple Expert Brainstorming for Domain Adaptive Person Re-identification

2 code implementations ECCV 2020 Yunpeng Zhai, Qixiang Ye, Shijian Lu, Mengxi Jia, Rongrong Ji, Yonghong Tian

Often the best performing deep neural models are ensembles of multiple base-level networks, nevertheless, ensemble learning with respect to domain adaptive person re-ID remains unexplored.

Domain Adaptive Person Re-Identification Ensemble Learning +1

SEKD: Self-Evolving Keypoint Detection and Description

1 code implementation9 Jun 2020 Yafei Song, Ling Cai, Jia Li, Yonghong Tian, Mingyang Li

Researchers have attempted utilizing deep neural network (DNN) to learn novel local features from images inspired by its recent successes on a variety of vision tasks.

Homography Estimation Keypoint Detection

Rethinking Performance Estimation in Neural Architecture Search

1 code implementation CVPR 2020 Xiawu Zheng, Rongrong Ji, Qiang Wang, Qixiang Ye, Zhenguo Li, Yonghong Tian, Qi Tian

In this paper, we provide a novel yet systematic rethinking of PE in a resource constrained regime, termed budgeted PE (BPE), which precisely and effectively estimates the performance of an architecture sampled from an architecture space.

Neural Architecture Search

Compositional Few-Shot Recognition with Primitive Discovery and Enhancing

no code implementations12 May 2020 Yixiong Zou, Shanghang Zhang, Ke Chen, Yonghong Tian, Yao-Wei Wang, José M. F. Moura

Inspired by such capability of humans, to imitate humans' ability of learning visual primitives and composing primitives to recognize novel classes, we propose an approach to FSL to learn a feature representation composed of important primitives, which is jointly trained with two parts, i. e. primitive discovery and primitive enhancing.

Few-Shot Image Classification Few-Shot Learning +1

Self-Guided Adaptation: Progressive Representation Alignment for Domain Adaptive Object Detection

no code implementations19 Mar 2020 Zongxian Li, Qixiang Ye, Chong Zhang, Jingjing Liu, Shijian Lu, Yonghong Tian

In this work, we propose a Self-Guided Adaptation (SGA) model, target at aligning feature representation and transferring object detection models across domains while considering the instantaneous alignment difficulty.

object-detection Object Detection +1

HRank: Filter Pruning using High-Rank Feature Map

2 code implementations CVPR 2020 Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, Ling Shao

The principle behind our pruning is that low-rank feature maps contain less information, and thus pruned results can be easily reproduced.

Network Pruning Vocal Bursts Intensity Prediction

Channel Pruning via Automatic Structure Search

1 code implementation23 Jan 2020 Mingbao Lin, Rongrong Ji, Yuxin Zhang, Baochang Zhang, Yongjian Wu, Yonghong Tian

In this paper, we propose a new channel pruning method based on artificial bee colony algorithm (ABC), dubbed as ABCPruner, which aims to efficiently find optimal pruned structure, i. e., channel number in each layer, rather than selecting "important" channels as previous works did.

Filter Sketch for Network Pruning

1 code implementation23 Jan 2020 Mingbao Lin, Liujuan Cao, Shaojie Li, Qixiang Ye, Yonghong Tian, Jianzhuang Liu, Qi Tian, Rongrong Ji

Our approach, referred to as FilterSketch, encodes the second-order information of pre-trained weights, which enables the representation capacity of pruned networks to be recovered with a simple fine-tuning procedure.

Network Pruning

Salient Object Detection with Purificatory Mechanism and Structural Similarity Loss

1 code implementation18 Dec 2019 Jia Li, Jinming Su, Changqun Xia, Mingcan Ma, Yonghong Tian

Through these two attentions, we use the Purificatory Mechanism to impose strict weights with different regions of the whole salient objects and purify results from hard-to-distinguish regions, thus accurately predicting the locations and details of salient objects.

object-detection RGB Salient Object Detection +1

Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning

no code implementations ICCV 2019 Limeng Qiao, Yemin Shi, Jia Li, Yao-Wei Wang, Tiejun Huang, Yonghong Tian

By solving the problem with its closed-form solution on the fly with the setup of transduction, our approach efficiently tailors an episodic-wise metric for each task to adapt all features from a shared task-agnostic embedding space into a more discriminative task-specific metric space.

Few-Shot Learning Metric Learning

Exploring Reciprocal Attention for Salient Object Detection by Cooperative Learning

no code implementations18 Sep 2019 Changqun Xia, Jia Li, Jinming Su, Yonghong Tian

Typically, objects with the same semantics are not always prominent in images containing different backgrounds.

Multi-Task Learning object-detection +2

Distortion-adaptive Salient Object Detection in 360$^\circ$ Omnidirectional Images

no code implementations11 Sep 2019 Jia Li, Jinming Su, Changqun Xia, Yonghong Tian

Moreover, benchmarking results of the proposed baseline approach and other methods on 360$^\circ$ SOD dataset show the proposed dataset is very challenging, which also validate the usefulness of the proposed dataset and approach to boost the development of SOD on 360$^\circ$ omnidirectional scenes.

Benchmarking object-detection +2

Learning Local Feature Descriptor with Motion Attribute for Vision-based Localization

no code implementations3 Aug 2019 Yafei Song, Di Zhu, Jia Li, Yonghong Tian, Mingyang Li

For better performance, the features used for open-loop localization are required to be short-term globally static, and the ones used for re-localization or loop closure detection need to be long-term static.

Loop Closure Detection

A Retina-inspired Sampling Method for Visual Texture Reconstruction

no code implementations20 Jul 2019 Lin Zhu, Siwei Dong, Tiejun Huang, Yonghong Tian

Conventional frame-based camera is not able to meet the demand of rapid reaction for real-time applications, while the emerging dynamic vision sensor (DVS) can realize high speed capturing for moving objects.

P-ODN: Prototype based Open Deep Network for Open Set Recognition

no code implementations6 May 2019 Yu Shu, Yemin Shi, Yao-Wei Wang, Tiejun Huang, Yonghong Tian

Predictors for new categories are added to the classification layer to "open" the deep neural networks to incorporate new categories dynamically.

Open Set Learning

Reconstruction of Natural Visual Scenes from Neural Spikes with Deep Neural Networks

no code implementations30 Apr 2019 Yichen Zhang, Shanshan Jia, Yajing Zheng, Zhaofei Yu, Yonghong Tian, Siwei Ma, Tiejun Huang, Jian. K. Liu

The SID is an end-to-end decoder with one end as neural spikes and the other end as images, which can be trained directly such that visual scenes are reconstructed from spikes in a highly accurate fashion.

Probabilistic Inference of Binary Markov Random Fields in Spiking Neural Networks through Mean-field Approximation

no code implementations22 Feb 2019 Yajing Zheng, Shanshan Jia, Zhaofei Yu, Tiejun Huang, Jian. K. Liu, Yonghong Tian

Recent studies have suggested that the cognitive process of the human brain is realized as probabilistic inference and can be further modeled by probabilistic graphical models like Markov random fields.

Image Denoising valid

2D LiDAR Map Prediction via Estimating Motion Flow with GRU

no code implementations19 Feb 2019 Yafei Song, Yonghong Tian, Gang Wang, Mingyang Li

To tackle this problem, we resort to the motion flow between adjacent maps, as motion flow is a powerful tool to process and analyze the dynamic data, which is named optical flow in video processing.

Optical Flow Estimation Representation Learning

ODN: Opening the Deep Network for Open-set Action Recognition

no code implementations23 Jan 2019 Yu Shu, Yemin Shi, Yao-Wei Wang, Yixiong Zou, Qingsheng Yuan, Yonghong Tian

Most of the existing action recognition works hold the \textit{closed-set} assumption that all action categories are known beforehand while deep networks can be well trained for these categories.

Open Set Action Recognition Temporal Action Localization

Selectivity or Invariance: Boundary-aware Salient Object Detection

no code implementations ICCV 2019 Jinming Su, Jia Li, Yu Zhang, Changqun Xia, Yonghong Tian

In this network, the feature selectivity at boundaries is enhanced by incorporating a boundary localization stream, while the feature invariance at interiors is guaranteed with a complex interior perception stream.

object-detection RGB Salient Object Detection +1

Greedy Hash: Towards Fast Optimization for Accurate Hash Coding in CNN

2 code implementations NeurIPS 2018 Shupeng Su, Chao Zhang, Kai Han, Yonghong Tian

To convert the input into binary code, hashing algorithm has been widely used for approximate nearest neighbor search on large-scale image sets due to its computation and storage efficiency.

Deep Hashing

Model-guided Multi-path Knowledge Aggregation for Aerial Saliency Prediction

no code implementations14 Nov 2018 Kui Fu, Jia Li, Yu Zhang, Hongze Shen, Yonghong Tian

After that, the visual saliency knowledge encoded in the most representative paths is selected and aggregated to improve the capability of MM-Net in predicting spatial saliency in aerial scenarios.

Aerial Video Saliency Prediction Transfer Learning +1

Revealing Fine Structures of the Retinal Receptive Field by Deep Learning Networks

no code implementations6 Nov 2018 Qi Yan, Yajing Zheng, Shanshan Jia, Yichen Zhang, Zhaofei Yu, Feng Chen, Yonghong Tian, Tiejun Huang, Jian. K. Liu

When a deep CNN with many layers is used for the visual system, it is not easy to compare the structure components of CNNs with possible neuroscience underpinnings due to highly complex circuits from the retina to higher visual cortex.

Transfer Learning

Neural System Identification with Spike-triggered Non-negative Matrix Factorization

no code implementations12 Aug 2018 Shanshan Jia, Zhaofei Yu, Arno Onken, Yonghong Tian, Tiejun Huang, Jian. K. Liu

Furthermore, we show that STNMF can separate spikes of a ganglion cell into a few subsets of spikes where each subset is contributed by one presynaptic bipolar cell.

Winner-Take-All as Basic Probabilistic Inference Unit of Neuronal Circuits

no code implementations2 Aug 2018 Zhaofei Yu, Yonghong Tian, Tiejun Huang, Jian. K. Liu

Taken together, our results suggest that the WTA circuit could be seen as the minimal inference unit of neuronal circuits.

Bayesian Inference

Hierarchical Deep Co-segmentation of Primary Objects in Aerial Videos

no code implementations27 Jun 2018 Jia Li, Pengcheng Yuan, Daxin Gu, Yonghong Tian

Primary object segmentation plays an important role in understanding videos generated by unmanned aerial vehicles.

Segmentation Semantic Segmentation

Peeking the Impact of Points of Interests on Didi

no code implementations6 Apr 2018 Yonghong Tian, Zeyu Li, Zhiwei Xu, Xuying Meng, Bing Zheng

Recently, the online car-hailing service, Didi, has emerged as a leader in the sharing economy.

Learning a Repression Network for Precise Vehicle Search

1 code implementation8 Aug 2017 Qiantong Xu, Ke Yan, Yonghong Tian

The growing explosion in the use of surveillance cameras in public security highlights the importance of vehicle search from large-scale image databases.

Multi-Task Learning Retrieval

YoTube: Searching Action Proposal via Recurrent and Static Regression Networks

no code implementations26 Jun 2017 Hongyuan Zhu, Romain Vial, Shijian Lu, Yonghong Tian, Xian-Bin Cao

In this paper, we present YoTube-a novel network fusion framework for searching action proposals in untrimmed videos, where each action proposal corresponds to a spatialtemporal video tube that potentially locates one human action.

Optical Flow Estimation regression

Joint Network based Attention for Action Recognition

no code implementations16 Nov 2016 Yemin Shi, Yonghong Tian, Yao-Wei Wang, Tiejun Huang

We also introduce an attention mechanism on the temporal domain to capture the long-term dependence meanwhile finding the salient portions.

Action Recognition Temporal Action Localization

Deep Transfer Learning for Person Re-identification

1 code implementation16 Nov 2016 Mengyue Geng, Yao-Wei Wang, Tao Xiang, Yonghong Tian

Second, a two-stepped fine-tuning strategy is developed to transfer knowledge from auxiliary datasets.

General Classification Image Classification +2

Learning long-term dependencies for action recognition with a biologically-inspired deep network

1 code implementation ICCV 2017 Yemin Shi, Yonghong Tian, Yao-Wei Wang, Tiejun Huang

Despite a lot of research efforts devoted in recent years, how to efficiently learn long-term dependencies from sequences still remains a pretty challenging task.

Action Recognition Temporal Action Localization

Sequential Deep Trajectory Descriptor for Action Recognition with Three-stream CNN

no code implementations10 Sep 2016 Yemin Shi, Yonghong Tian, Yao-Wei Wang, Tiejun Huang

Nevertheless, most of the existing features or descriptors cannot capture motion information effectively, especially for long-term motion.

Action Recognition Temporal Action Localization

Deep Relative Distance Learning: Tell the Difference Between Similar Vehicles

no code implementations CVPR 2016 Hongye Liu, Yonghong Tian, Yaowei Yang, Lu Pang, Tiejun Huang

To further facilitate the future research on this problem, we also present a carefully-organized large-scale image database "VehicleID", which includes multiple images of the same vehicle captured by different real-world cameras in a city.

Face Recognition Person Re-Identification +2

Unsupervised Cross-Dataset Transfer Learning for Person Re-Identification

no code implementations CVPR 2016 Peixi Peng, Tao Xiang, Yao-Wei Wang, Massimiliano Pontil, Shaogang Gong, Tiejun Huang, Yonghong Tian

Most existing person re-identification (Re-ID) approaches follow a supervised learning framework, in which a large number of labelled matching pairs are required for training.

Dictionary Learning Person Re-Identification +1

Quality Assessment for Comparing Image Enhancement Algorithms

no code implementations CVPR 2014 Zhengying Chen, Tingting Jiang, Yonghong Tian

As the image enhancement algorithms developed in recent years, how to compare the performances of different image enhancement algorithms becomes a novel task.

Image Enhancement Image Quality Assessment

Cannot find the paper you are looking for? You can Submit a new open access paper.