Search Results for author:
Found 86 papers, 28 papers with code
An Attention-driven Two-stage Clustering Method for Unsupervised Person Re-Identification
By iteratively learning with the two strategies, the attentive regions are gradually shifted from the background to the foreground and the features become more discriminative.
Unsupervised Optical Flow Estimation with Dynamic Timing Representation for Spike Camera
Moreover, we propose an unsupervised learning method for optical flow estimation in a spike-based manner to break the dependence on labeled data.
Generative Pretraining in Multimodality
We present Emu, a Transformer-based multimodal foundation model, which can seamlessly generate images and texts in multimodal context.
Ranked #5 on
Temporal/Casual QA
on NExT-QA
(using extra training data)
SVIT: Scaling up Visual Instruction Tuning
Thanks to the emerging of foundation models, the large language and vision models are integrated to acquire the multimodal ability of visual captioning, dialogue, question answering, etc.
Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views
To process the spikes, we build a novel model \textbf{SpkOccNet}, in which we integrate information of spikes from continuous viewpoints within multi-windows, and propose a novel cross-view mutual attention mechanism for effective fusion and refinement.
Mitigating Communication Costs in Neural Networks: The Role of Dendritic Nonlinearity
Our comprehension of biological neuronal networks has profoundly influenced the evolution of artificial neural networks (ANNs).
Pushing the Limits of 3D Shape Generation at Scale
Through the adaptation of the Auto-Regressive model and the utilization of large language models, we have developed a remarkable model with an astounding 3. 6 billion trainable parameters, establishing it as the largest 3D shape generation model to date, named Argus-3D.
Spike timing reshapes robustness against attacks in spiking neural networks
The success of deep learning in the past decade is partially shrouded in the shadow of adversarial attacks.
Large-scale Dataset Pruning with Dynamic Uncertainty
The state of the art of many learning tasks, e. g., image classification, is advanced by collecting larger datasets and then training larger models on them.
HUB: Guiding Learned Optimizers with Continuous Prompt Tuning
By incorporating hand-designed optimizers as the second component in our hybrid approach, we are able to retain the benefits of learned optimizers while stabilizing the training process and, more importantly, improving testing performance.
Spike Stream Denoising via Spike Camera Simulation
Based on our noise model, the first benchmark for spike stream denoising is proposed which includes clear (noisy) spike stream.
SegGPT: Segmenting Everything In Context
We unify various segmentation tasks into a generalist in-context learning framework that accommodates different kinds of segmentation data by transforming them into the same format of images.
Ranked #1 on
Few-Shot Semantic Segmentation
on PASCAL-5i (5-Shot)
Exploring Asymmetric Tunable Blind-Spots for Self-supervised Denoising in Real-World Scenarios
Self-supervised denoising has attracted widespread attention due to its ability to train without clean images.
SpikeCV: Open a Continuous Computer Vision Era
SpikeCV focuses on encapsulation for spike data, standardization for dataset interfaces, modularization for vision tasks, and real-time applications for challenging scenes.
EVA-02: A Visual Representation for Neon Genesis
We launch EVA-02, a next-generation Transformer-based visual representation pre-trained to reconstruct strong and robust language-aligned vision features via masked image modeling.
CLIP4MC: An RL-Friendly Vision-Language Model for Minecraft
One of the essential missions in the AI research community is to build an autonomous embodied agent that can attain high-level performance across a wide spectrum of tasks.
Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks
In this paper, we theoretically analyze ANN-SNN conversion error and derive the estimated activation function of SNNs.
Bridging the Gap between ANNs and SNNs by Calibrating Offset Spikes
The experimental results show that our proposed method achieves state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet datasets.
Hard-aware Instance Adaptive Self-training for Unsupervised Cross-domain Semantic Segmentation
The divergence between labeled training data and unlabeled testing data is a significant challenge for recent deep learning models.
Reducing ANN-SNN Conversion Error through Residual Membrane Potential
Spiking Neural Networks (SNNs) have received extensive academic attention due to the unique properties of low power consumption and high-speed computing on neuromorphic chips.
MUVA: A New Large-Scale Benchmark for Multi-View Amodal Instance Segmentation in the Shopping Scenario
To bridge this gap, we propose a new task called Multi-view Amodal Instance Segmentation (MAIS) and introduce the MUVA dataset, the first MUlti-View AIS dataset that takes the shopping scenario as instantiation.
1000 FPS HDR Video With a Spike-RGB Hybrid Camera
Capturing high frame rate and high dynamic range (HFR&HDR) color videos in high-speed scenes with conventional frame-based cameras is very challenging.
SegGPT: Towards Segmenting Everything in Context
We unify various segmentation tasks into a generalist in-context learning framework that accommodates different kinds of segmentation data by transforming them into the same format of images.
Images Speak in Images: A Generalist Painter for In-Context Visual Learning
In this work, we present Painter, a generalist model which addresses these obstacles with an "image"-centric solution, that is, to redefine the output of core vision tasks as images, and specify task prompts as also images.
Ranked #6 on
Personalized Segmentation
on PerSeg
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale
We launch EVA, a vision-centric foundation model to explore the limits of visual representation at scale using only publicly accessible data.
Ranked #1 on
Object Detection
on COCO-O
(using extra training data)
Entity Divider with Language Grounding in Multi-Agent Reinforcement Learning
We investigate the use of natural language to drive the generalization of policies in multi-agent settings.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Multi-Agent Sequential Decision-Making via Communication
Communication helps agents to obtain information about others so that better coordinated behavior can be learned.
Unsupervised Spike Depth Estimation via Cross-modality Cross-domain Knowledge Transfer
Neuromorphic spike data, an upcoming modality with high temporal resolution, has shown promising potential in real-world applications due to its inherent advantage to overcome high-velocity motion blur.
Uncertainty Guided Depth Fusion for Spike Camera
In this paper, we propose a novel Uncertainty-Guided Depth Fusion (UGDF) framework to fuse the predictions of monocular and stereo depth estimation networks for spike camera.
Annotation Efficient Person Re-Identification with Diverse Cluster-Based Pair Selection
To solve this problem, we propose the Annotation Efficient Person Re-Identification method to select image pairs from an alternative pair set according to the fallibility and diversity of pairs, and train the Re-ID model based on the annotation.
Optimized Potential Initialization for Low-latency Spiking Neural Networks
We evaluate our algorithm on the CIFAR-10, CIFAR-100 and ImageNet datasets and achieve state-of-the-art accuracy, using fewer time-steps.
Event-based Video Reconstruction via Potential-assisted Spiking Neural Network
We propose a novel Event-based Video reconstruction framework based on a fully Spiking Neural Network (EVSNN), which utilizes Leaky-Integrate-and-Fire (LIF) neuron and Membrane Potential (MP) neuron.
1000x Faster Camera and Machine Vision with Ordinary Devices
By treating vidar as spike trains in biological vision, we have further developed a spiking neural network-based machine vision system that combines the speed of the machine and the mechanism of biological vision, achieving high-speed object detection and tracking 1, 000x faster than human vision.
Deep Reinforcement Learning with Spiking Q-learning
With the help of special neuromorphic hardware, spiking neural networks (SNNs) are expected to realize artificial intelligence with less energy consumption.
A Robust Visual Sampling Model Inspired by Receptive Field
More importantly, due to mimicking receptive field mechanism to collect regional information, RVSM can filter high intensity noise effectively and improves the problem that Spike camera is sensitive to noise largely.
Sample Prior Guided Robust Model Learning to Suppress Noisy Labels
Imperfect labels are ubiquitous in real-world datasets and seriously harm the model performance.
Ranked #1 on
Learning with noisy labels
on CIFAR-100N
Noisy Adaptation Generates Lévy Flights in Attractor Neural Networks
To elucidate the underlying mechanism clearly, we first study continuous attractor neural networks (CANNs), and find that noisy neural adaptation, exemplified by spike frequency adaptation (SFA) in this work, can generate Lévy flights representing transitions of the network state in the attractor space.
Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling
Inspired by BERT, we devise a Masked Point Modeling (MPM) task to pre-train point cloud Transformers.
Ranked #10 on
Few-Shot 3D Point Cloud Classification
on ModelNet40 5-way (10-shot)
(using extra training data)
3D Point Cloud Linear Classification
Few-Shot 3D Point Cloud Classification
+2
Optical Flow Estimation for Spiking Camera
Further, for training SCFlow, we synthesize two sets of optical flow data for the spiking camera, SPIkingly Flying Things and Photo-realistic High-speed Motion, denoted as SPIFT and PHM respectively, corresponding to random high-speed and well-designed scenes.
Accelerating Training of Deep Spiking Neural Networks with Parameter Initialization
Despite that spiking neural networks (SNNs) show strong advantages in information encoding, power consuming, and computational capability, the underdevelopment of supervised learning algorithms is still a hindrance for training SNN.
Sequential Communication in Multi-Agent Reinforcement Learning
Agents determine the priority of decision-making by comparing the value of intention.
Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation
Recently, many deep learning methods have shown great success in providing promising solutions to many event-based problems, such as optical flow estimation.
High-Speed Image Reconstruction Through Short-Term Plasticity for Spiking Cameras
Mimicking the sampling mechanism of the fovea, a retina-inspired camera, named spiking camera, is developed to record the external information with a sampling rate of 40, 000 Hz, and outputs asynchronous binary spike streams.
Spk2ImgNet: Learning To Reconstruct Dynamic Scene From Continuous Spike Stream
Different from the conventional digital cameras that compact the photoelectric information within the exposure interval into a single snapshot, the spike camera produces a continuous spike stream to record the dynamic light intensity variation process.
Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networks
We show that the inference time can be reduced by optimizing the upper bound of the fit curve in the revised ANN to achieve fast inference.
Pruning of Deep Spiking Neural Networks through Gradient Rewiring
Our key innovation is to redefine the gradient to a new synaptic parameter, allowing better exploration of network structures by taking full advantage of the competition between pruning and regrowth of connections.
Deep Residual Learning in Spiking Neural Networks
Previous Spiking ResNet mimics the standard residual block in ANNs and simply replaces ReLU activation layers with spiking neurons, which suffers the degradation problem and can hardly implement residual learning.
Super Resolve Dynamic Scene From Continuous Spike Streams
In this paper, we properly exploit the relative motion and derive the relationship between light intensity and each spike, so as to recover the external scene with both high temporal and high spatial resolution.
NeuSpike-Net: High Speed Video Reconstruction via Bio-Inspired Neuromorphic Cameras
In this paper, we propose a NeuSpike-Net to learn both the high dynamic range and high motion sensitivity of DVS and the full texture sampling of spike camera to achieve high-speed and high dynamic image reconstruction.
UnModNet: Learning to Unwrap a Modulo Image for High Dynamic Range Imaging
A conventional camera often suffers from over- or under-exposure when recording a real-world scene with a very high dynamic range (HDR).
Learning Open Set Network with Discriminative Reciprocal Points
In this process, one of the key challenges is to reduce the risk of generalizing the inherent characteristics of numerous unknown samples learned from a small amount of known data.
Human Perception-based Evaluation Criterion for Ultra-high Resolution Cell Membrane Segmentation
Computer vision technology is widely used in biological and medical data analysis and understanding.
Vision at A Glance: Interplay between Fine and Coarse Information Processing Pathways
In this study, we build a computational model to elucidate the computational advantages associated with the interactions between two pathways.
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks
In this paper, we take inspiration from the observation that membrane-related parameters are different across brain regions, and propose a training algorithm that is capable of learning not only the synaptic weights but also the membrane time constants of SNNs.
Learning Individually Inferred Communication for Multi-Agent Cooperation
Empirically, we show that I2C can not only reduce communication overhead but also improve the performance in a variety of multi-agent cooperative scenarios, comparing to existing methods.
Kernel Quantization for Efficient Network Compression
Thus, KQ can represent the weight tensor in the convolution layer with low-bit indexes and a kernel codebook with limited size, which enables KQ to achieve significant compression ratio.
Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics
Meanwhile, we systematically review state-of-the-art techniques in video compression and feature compression from the unique perspective of MPEG standardization, which provides the academic and industrial evidence to realize the collaborative compression of video and feature streams in a broad range of AI applications.
Unsupervised Few-shot Learning via Self-supervised Training
The proposed model consists of two alternate processes, progressive clustering and episodic training.
Push-pull Feedback Implements Hierarchical Information Retrieval Efficiently
Experimental data has revealed that in addition to feedforward connections, there exist abundant feedback connections in a neural pathway.
Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning
By solving the problem with its closed-form solution on the fly with the setup of transduction, our approach efficiently tailors an episodic-wise metric for each task to adapt all features from a shared task-agnostic embedding space into a more discriminative task-specific metric space.
Unsupervised Few Shot Learning via Self-supervised Training
Using the benchmark dataset Omniglot, we show that our model outperforms other unsupervised few-shot learning methods to a large extend and approaches to the performances of supervised methods.
Knowledge Transfer via Student-Teacher Collaboration
One way to compress these heavy models is knowledge transfer (KT), in which a light student network is trained through absorbing the knowledge from a powerful teacher network.
Spatiotemporal Information Processing with a Reservoir Decision-making Network
Spatiotemporal information processing is fundamental to brain functions.
A Retina-inspired Sampling Method for Visual Texture Reconstruction
Conventional frame-based camera is not able to meet the demand of rapid reaction for real-time applications, while the emerging dynamic vision sensor (DVS) can realize high speed capturing for moving objects.
P-ODN: Prototype based Open Deep Network for Open Set Recognition
Predictors for new categories are added to the classification layer to "open" the deep neural networks to incorporate new categories dynamically.
Reconstruction of Natural Visual Scenes from Neural Spikes with Deep Neural Networks
The SID is an end-to-end decoder with one end as neural spikes and the other end as images, which can be trained directly such that visual scenes are reconstructed from spikes in a highly accurate fashion.
Bi-Directional Cascade Network for Perceptual Edge Detection
Exploiting multi-scale representations is critical to improve edge detection for objects at different scales.
Ranked #2 on
Edge Detection
on BRIND
Probabilistic Inference of Binary Markov Random Fields in Spiking Neural Networks through Mean-field Approximation
Recent studies have suggested that the cognitive process of the human brain is realized as probabilistic inference and can be further modeled by probabilistic graphical models like Markov random fields.
Multi-scale 3D Convolution Network for Video Based Person Re-Identification
A temporal stream in this network is constructed by inserting several Multi-scale 3D (M3D) convolution layers into a 2D CNN network.
Revealing Fine Structures of the Retinal Receptive Field by Deep Learning Networks
When a deep CNN with many layers is used for the visual system, it is not easy to compare the structure components of CNNs with possible neuroscience underpinnings due to highly complex circuits from the retina to higher visual cortex.
Graph Convolutional Reinforcement Learning
The key is to understand the mutual interplay between agents.
Neural System Identification with Spike-triggered Non-negative Matrix Factorization
Furthermore, we show that STNMF can separate spikes of a ganglion cell into a few subsets of spikes where each subset is contributed by one presynaptic bipolar cell.
Winner-Take-All as Basic Probabilistic Inference Unit of Neuronal Circuits
Taken together, our results suggest that the WTA circuit could be seen as the minimal inference unit of neuronal circuits.
A simple blind-denoising filter inspired by electrically coupled photoreceptors in the retina
Photoreceptors in the retina are coupled by electrical synapses called "gap junctions".
Depth-Aware Stereo Video Retargeting
As compared with traditional video retargeting, stereo video retargeting poses new challenges because stereo video contains the depth information of salient objects and its time dynamics.
Exploiting Multi-Grain Ranking Constraints for Precisely Searching Visually-Similar Vehicles
In this paper, we model the relationship of vehicle images as multiple grains.
E$^2$BoWs: An End-to-End Bag-of-Words Model via Deep Convolutional Neural Network
To conquer these issues, we propose an End-to-End BoWs (E$^2$BoWs) model based on Deep Convolutional Neural Network (DCNN).
Compact Descriptors for Video Analysis: the Emerging MPEG Standard
This paper provides an overview of the on-going compact descriptors for video analysis standard (CDVA) from the ISO/IEC moving pictures experts group (MPEG).
Improving Object Detection with Region Similarity Learning
Object detection aims to identify instances of semantic objects of a certain class in images or videos.
Incorporating Intra-Class Variance to Fine-Grained Visual Recognition
In this paper, we propose to leverage intra-class variance in metric learning of triplet network to improve the performance of fine-grained recognition.
Joint Network based Attention for Action Recognition
We also introduce an attention mechanism on the temporal domain to capture the long-term dependence meanwhile finding the salient portions.
Learning long-term dependencies for action recognition with a biologically-inspired deep network
Despite a lot of research efforts devoted in recent years, how to efficiently learn long-term dependencies from sequences still remains a pretty challenging task.
Sequential Deep Trajectory Descriptor for Action Recognition with Three-stream CNN
Nevertheless, most of the existing features or descriptors cannot capture motion information effectively, especially for long-term motion.
Unsupervised Cross-Dataset Transfer Learning for Person Re-Identification
Most existing person re-identification (Re-ID) approaches follow a supervised learning framework, in which a large number of labelled matching pairs are required for training.
Deep Relative Distance Learning: Tell the Difference Between Similar Vehicles
To further facilitate the future research on this problem, we also present a carefully-organized large-scale image database "VehicleID", which includes multiple images of the same vehicle captured by different real-world cameras in a city.
