Search Results for author:
Found 74 papers, 46 papers with code
Object Relation Detection Based on One-shot Learning
Detecting the relations among objects, such as "cat on sofa" and "person ride horse", is a crucial task in image understanding, and beneficial to bridging the semantic gap between images and natural language.
The Unconstrained Ear Recognition Challenge 2019 - ArXiv Version With Appendix
The goal of the challenge is to assess the performance of existing ear recognition techniques on a challenging large-scale ear dataset and to analyze performance of the technology from various viewpoints, such as generalization abilities to unseen data characteristics, sensitivity to rotations, occlusions and image resolution and performance bias on sub-groups of subjects, selected based on demographic criteria, i. e. gender and ethnicity.
Few-shot Adaptive Faster R-CNN
To address these challenges, we first introduce a pairing mechanism over source and target features to alleviate the issue of insufficient target domain samples.
Cycle-SUM: Cycle-consistent Adversarial LSTM Networks for Unsupervised Video Summarization
The evaluator defines a learnable information preserving metric between original video and summary video and "supervises" the selector to identify the most informative frames to form the summary video.
Ranked #7 on
Unsupervised Video Summarization
on TvSum
Distilling Object Detectors with Fine-grained Feature Imitation
To address the challenge of distilling knowledge in detection model, we propose a fine-grained feature imitation method exploiting the cross-location discrepancy of feature response.
Central Similarity Quantization for Efficient Image and Video Retrieval
In this work, we propose a new \emph{global} similarity metric, termed as \emph{central similarity}, with which the hash codes of similar data pairs are encouraged to approach a common center and those for dissimilar pairs to converge to different centers, to improve hash learning efficiency and retrieval accuracy.
Revisiting Knowledge Distillation via Label Smoothing Regularization
Without any extra computation cost, Tf-KD achieves up to 0. 65\% improvement on ImageNet over well-established baseline models, which is superior to label smoothing regularization.
YNU-HPCC at SemEval-2020 Task 8: Using a Parallel-Channel Model for Memotion Analysis
In recent years, the growing ubiquity of Internet memes on social media platforms, such as Facebook, Instagram, and Twitter, has become a topic of immense interest.
Exploring global diverse attention via pairwise temporal relation for video summarization
In this paper, we propose an efficient convolutional neural network architecture for video SUMmarization via Global Diverse Attention called SUM-GDA, which adapts attention mechanism in a global perspective to consider pairwise temporal relations of video frames.
Toward Accurate Person-level Action Recognition in Videos of Crowded Scenes
Prior works always fail to deal with this problem in two aspects: (1) lacking utilizing information of the scenes; (2) lacking training data in the crowd and complex scenes.
Towards Accurate Human Pose Estimation in Videos of Crowded Scenes
In this paper, we focus on improving human pose estimation in videos of crowded scenes from the perspectives of exploiting temporal context and collecting new data.
A Simple Baseline for Pose Tracking in Videos of Crowded Scenes
This paper presents our solution to ACM MM challenge: Large-scale Human-centric Video Analysis in Complex Events\cite{lin2020human}; specifically, here we focus on Track3: Crowd Pose Tracking in Complex Events.
Fooling the primate brain with minimal, targeted image manipulation
Here we propose an array of methods for creating minimal, targeted image perturbations that lead to changes in both neuronal activity and perception as reflected in behavior.
Graph Attention Network with Memory Fusion for Aspect-level Sentiment Analysis
Recent studies used attention-based methods that can effectively improve the performance of aspect-level sentiment analysis.
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
To overcome such limitations, we propose a new Tokens-To-Token Vision Transformer (T2T-ViT), which incorporates 1) a layer-wise Tokens-to-Token (T2T) transformation to progressively structurize the image to tokens by recursively aggregating neighboring Tokens into one Token (Tokens-to-Token), such that local structure represented by surrounding tokens can be modeled and tokens length can be reduced; 2) an efficient backbone with a deep-narrow structure for vision transformer motivated by CNN architecture design after empirical study.
Ranked #400 on
Image Classification
on ImageNet
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization
It is well-known that stochastic gradient noise (SGN) acts as implicit regularization for deep learning and is essentially important for both optimization and generalization of deep networks.
All Tokens Matter: Token Labeling for Training Better Vision Transformers
In this paper, we present token labeling -- a new training objective for training high-performance vision transformers (ViTs).
Ranked #3 on
Efficient ViTs
on ImageNet-1K (With LV-ViT-S)
Continual Learning via Bit-Level Information Preserving
Continual learning tackles the setting of learning different tasks sequentially.
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition
By realizing the importance of the positional information carried by 2D feature representations, unlike recent MLP-like models that encode the spatial information along the flattened spatial dimensions, Vision Permutator separately encodes the feature representations along the height and width dimensions with linear projections.
VOLO: Vision Outlooker for Visual Recognition
Though recently the prevailing vision transformers (ViTs) have shown great potential of self-attention based models in ImageNet classification, their performance is still inferior to that of the latest SOTA CNNs if no extra data are provided.
Ranked #1 on
Image Classification
on VizWiz-Classification
PnP-DETR: Towards Efficient Visual Analysis with Transformers
Recently, DETR pioneered the solution of vision tasks with transformers, it directly translates the image feature map into the object detection result.
Full Transformer Framework for Robust Point Cloud Registration with Deep Information Interaction
Recent Transformer-based methods have achieved advanced performance in point cloud registration by utilizing advantages of the Transformer in order-invariance and modeling dependency to aggregate information.
DynaMixer: A Vision MLP Architecture with Dynamic Mixing
In contrast with vision transformers and CNNs, the success of MLP-like models shows that simple information fusion operations among tokens and channels can yield a good representation power for deep recognition models.
Masked Autoencoders for Point Cloud Self-supervised Learning
Then, a standard Transformer based autoencoder, with an asymmetric design and a shifting mask tokens operation, learns high-level latent features from unmasked point patches, aiming to reconstruct the masked point patches.
Ranked #2 on
Point Cloud Segmentation
on PointCloud-C
3D Part Segmentation
Few-Shot 3D Point Cloud Classification
+2
Improving Vision Transformers by Revisiting High-frequency Components
Inspired by this finding, we first investigate the effects of existing techniques for improving ViT models from a new frequency perspective, and find that the success of some techniques (e. g., RandAugment) can be attributed to the better usage of the high-frequency components.
Ranked #2 on
Domain Generalization
on Stylized-ImageNet
Locality Guidance for Improving Vision Transformers on Tiny Datasets
Therefore, our locality guidance approach is very simple and efficient, and can serve as a basic performance enhancement method for VTs on tiny datasets.
Spikformer: When Spiking Neural Network Meets Transformer
Spikformer (66. 3M parameters) with comparable size to SEW-ResNet-152 (60. 2M, 69. 26%) can achieve 74. 81% top1 accuracy on ImageNet using 4 time steps, which is the state-of-the-art in directly trained SNNs models.
ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentation
Recently, self-supervised large-scale visual pre-training models have shown great promise in representing pixel-level semantic relationships, significantly promoting the development of unsupervised dense prediction tasks, e. g., unsupervised semantic segmentation (USS).
Out-of-Candidate Rectification for Weakly Supervised Semantic Segmentation
Weakly supervised semantic segmentation is typically inspired by class activation maps, which serve as pseudo masks with class-discriminative regions highlighted.
Optical Character Recognition (OCR)
Weakly supervised Semantic Segmentation
+1
Joint Multimodal Entity-Relation Extraction Based on Edge-enhanced Graph Alignment Network and Word-pair Relation Tagging
This paper is the first to propose jointly performing MNER and MRE as a joint multimodal entity-relation extraction task (JMERE).
Rethinking Point Cloud Registration as Masking and Reconstruction
In this paper, a critical observation is made that the invisible parts of each point cloud can be directly utilized as inherent masks, and the aligned point cloud pair can be regarded as the reconstruction target.
MADAv2: Advanced Multi-Anchor Based Active Domain Adaptation Segmentation
Unsupervised domain adaption has been widely adopted in tasks with scarce annotated data.
Parallel Vertex Diffusion for Unified Visual Grounding
An intuitive materialization of our paradigm is Parallel Vertex Diffusion (PVD) to directly set vertex coordinates as the generation target and use a diffusion model to train and infer.
DiffusionRet: Generative Text-Video Retrieval with Diffusion Model
Existing text-video retrieval solutions are, in essence, discriminant models focused on maximizing the conditional likelihood, i. e., p(candidates|query).
Ranked #15 on
Video Retrieval
on MSVD
Multi-granularity Interaction Simulation for Unsupervised Interactive Segmentation
Interactive segmentation enables users to segment as needed by providing cues of objects, which introduces human-computer interaction for many fields, such as image editing and medical image analysis.
Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning
Contrastive learning-based video-language representation learning approaches, e. g., CLIP, have achieved outstanding performance, which pursue semantic interaction upon pre-defined video-text pairs.
Ranked #8 on
Video Question Answering
on MSRVTT-QA
Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning
However, in this work, we find that the CE loss is not ideal for the base session training as it suffers poor class separation in terms of representations, which further degrades generalization to novel classes.
Img2Vec: A Teacher of High Token-Diversity Helps Masked AutoEncoders
To study which type of deep features is appropriate for MIM as a learning target, we propose a simple MIM framework with serials of well-trained self-supervised models to convert an Image to a feature Vector as the learning target of MIM, where the feature extractor is also known as a teacher model.
PointGPT: Auto-regressively Generative Pre-training from Point Clouds
Large language models (LLMs) based on the generative pre-training transformer (GPT) have demonstrated remarkable effectiveness across a diverse range of downstream tasks.
Text-Video Retrieval with Disentangled Conceptualization and Set-to-Set Alignment
In this paper, we propose the Disentangled Conceptualization and Set-to-set Alignment (DiCoSA) to simulate the conceptualizing and reasoning process of human beings.
Album Storytelling with Iterative Story-aware Captioning and Large Language Models
One natural approach is to use caption models to describe each photo in the album, and then use LLMs to summarize and rewrite the generated captions into an engaging story.
Temporal Contrastive Learning for Spiking Neural Networks
However, in this work, we find the method above is not ideal for the SNNs training as it omits the temporal dynamics of SNNs and degrades the performance quickly with the decrease of inference time steps.
ChatFace: Chat-Guided Real Face Editing via Diffusion Latent Space Manipulation
Despite the successful image reconstruction achieved by diffusion-based methods, there are still challenges in effectively manipulating fine-gained facial attributes with textual instructions. To address these issues and facilitate convenient manipulation of real facial images, we propose a novel approach that conduct text-driven image editing in the semantic latent space of diffusion model.
Auto-Spikformer: Spikformer Architecture Search
The integration of self-attention mechanisms into Spiking Neural Networks (SNNs) has garnered considerable interest in the realm of advanced deep learning, primarily due to their biological properties.
ChatLaw: Open-Source Legal Large Language Model with Integrated External Knowledge Bases
Furthermore, we propose a self-attention method to enhance the ability of large models to overcome errors present in reference data, further optimizing the issue of model hallucinations at the model level and improving the problem-solving capabilities of large models.
Spike-driven Transformer
In this paper, we incorporate the spike-driven paradigm into Transformer by the proposed Spike-driven Transformer with four unique properties: 1) Event-driven, no calculation is triggered when the input of Transformer is zero; 2) Binary spike communication, all matrix multiplications associated with the spike matrix can be transformed into sparse additions; 3) Self-attention with linear complexity at both token and channel dimensions; 4) The operations between spike-form Query, Key, and Value are mask and addition.
Learning Sparse Neural Networks with Identity Layers
Applying such theory, we propose a plug-and-play CKA-based Sparsity Regularization for sparse network training, dubbed CKA-SR, which utilizes CKA to reduce feature similarity between layers and increase network sparsity.
Fully Transformer-Equipped Architecture for End-to-End Referring Video Object Segmentation
Referring Video Object Segmentation (RVOS) requires segmenting the object in video referred by a natural language query.
Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation
Unsupervised Video Object Segmentation (VOS) aims at identifying the contours of primary foreground objects in videos without any prior knowledge.
Semantic Segmentation
Unsupervised Video Object Segmentation
+1
Triple-View Knowledge Distillation for Semi-Supervised Semantic Segmentation
To alleviate the expensive human labeling, semi-supervised semantic segmentation employs a few labeled images and an abundant of unlabeled images to predict the pixel-level label map with the same size.
Adversarial Attacks on Video Object Segmentation with Hard Region Discovery
Particularly, the gradients from the segmentation model are exploited to discover the easily confused region, in which it is difficult to identify the pixel-wise objects from the background in a frame.
LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples
This phenomenon forces us to revisit that hallucination may be another view of adversarial examples, and it shares similar features with conventional adversarial examples as the basic feature of LLMs.
LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment
We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M.
Ranked #1 on
Zero-shot Audio Classification
on VGG-Sound
(using extra training data)
HiFi-123: Towards High-fidelity One Image to 3D Content Generation
Our contributions are twofold: First, we propose a Reference-Guided Novel View Enhancement (RGNV) technique that significantly improves the fidelity of diffusion-based zero-shot novel view synthesis methods.
IDRNet: Intervention-Driven Relation Network for Semantic Segmentation
Next, a deletion diagnostics procedure is conducted to model relations of these semantic-level representations via perceiving the network outputs and the extracted relations are utilized to guide the semantic-level representations to interact with each other.
Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts
Recent text-to-3D generation methods achieve impressive 3D content creation capacity thanks to the advances in image diffusion models and optimizing strategies.
Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding
Large language models have demonstrated impressive universal capabilities across a wide range of open-ended tasks and have extended their utility to encompass multimodal conversations.
Image-based Generative Performance Benchmarking
Language Modelling
+9
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
In this work, we unify visual representation into the language feature space to advance the foundational LLM towards a unified LVLM.
Ranked #2 on
Zero-Shot Video Question Answer
on TGIF-QA
Advancing Vision Transformers with Group-Mix Attention
The attention map is computed based on the mixtures of tokens and group proxies and used to re-combine the tokens and groups in Value.
Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models
Video-based large language models (Video-LLMs) have been recently introduced, targeting both fundamental improvements in perception and comprehension, and a diverse range of user inquiries.
Regressor-Segmenter Mutual Prompt Learning for Crowd Counting
In this study, we propose mutual prompt learning (mPrompt), which leverages a regressor and a segmenter as guidance for each other, solving bias and inaccuracy caused by annotation variance while distinguishing foreground from background.
FreestyleRet: Retrieving Images from Style-Diversified Queries
In this paper, we propose the Style-Diversified Query-Based Image Retrieval task, which enables retrieval based on various query styles.
Machine Mindset: An MBTI Exploration of Large Language Models
We present a novel approach for integrating Myers-Briggs Type Indicator (MBTI) personality traits into large language models (LLMs), addressing the challenges of personality consistency in personalized AI.
Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting
The core idea is to combine the powerful image generation capability of the 2D diffusion model and the texture alignment ability of the repainting strategy for generating high-quality multi-view images with consistency.
Spikformer V2: Join the High Accuracy Club on ImageNet with an SNN Ticket
To the best of our knowledge, this is the first time that the SNN achieves 80+% accuracy on ImageNet.
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
In this work, we propose a simple yet effective training strategy MoE-Tuning for LVLMs.
Ranked #52 on
Visual Question Answering
on MM-Vet
Peer-review-in-LLMs: Automatic Evaluation Method for LLMs in Open-environment
Existing large language models (LLMs) evaluation methods typically focus on testing the performance on some closed-environment and domain-specific benchmarks with human annotations.
LLMBind: A Unified Modality-Task Integration Framework
In the multi-modal domain, the dependence of various models on specific input formats leads to user confusion and hinders progress.
Optimal ANN-SNN Conversion with Group Neurons
For instance, while converting artificial neural networks (ANNs) to SNNs circumvents the need for direct training of SNNs, it encounters issues related to conversion errors and high inference time delays.
A Logical Pattern Memory Pre-trained Model for Entailment Tree Generation
LMPM incorporates an external memory structure to learn and store the latent representations of logical patterns, which aids in generating logically consistent conclusions.
Envision3D: One Image to 3D with Anchor Views Interpolation
To address this issue, we propose a novel cascade diffusion framework, which decomposes the challenging dense views generation task into two tractable stages, namely anchor views generation and anchor views interpolation.
QKFormer: Hierarchical Spiking Transformer using Q-K Attention
ii) We incorporate the hierarchical structure, which significantly benefits the performance of both the brain and artificial neural networks, into spiking transformers to obtain multi-scale spiking representation.
MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators
Recent advances in Text-to-Video generation (T2V) have achieved remarkable success in synthesizing high-quality general videos from textual descriptions.
UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmark
To further evaluate the IAA capability of MLLMs, we construct the UNIAA-Bench, which consists of three aesthetic levels: Perception, Description, and Assessment.
