Search Results for author:
Found 109 papers, 54 papers with code
DiscoVLA: Discrepancy Reduction in Vision, Language, and Alignment for Parameter-Efficient Video-Text Retrieval
The parameter-efficient adaptation of the image-text pretraining model CLIP for video-text retrieval is a prominent area of research.
THU-Warwick Submission for EPIC-KITCHEN Challenge 2025: Semi-Supervised Video Object Segmentation
In this report, we describe our approach to egocentric video object segmentation.
AdaTP: Attention-Debiased Token Pruning for Video Large Language Models
However, these scores exhibit inherent biases: global bias reflects a tendency to focus on the two ends of the visual token sequence, while local bias leads to an over-concentration on the same spatial positions across different frames.
Rethinking Score Distilling Sampling for 3D Editing and Generation
Essentially, UDS refines the gradient terms used in vanilla SDS methods, unifying them to support both tasks.
From Gaze to Insight: Bridging Human Visual Attention and Vision Language Model Explanation for Weakly-Supervised Medical Image Segmentation
To implement this, the teacher model first learns from gaze points enhanced by VLM-generated descriptions of lesion morphology, establishing a foundation for guiding the student model.
ConsDreamer: Advancing Multi-View Consistency for Zero-Shot Text-to-3D Generation
Recent advances in zero-shot text-to-3D generation have revolutionized 3D content creation by enabling direct synthesis from textual descriptions.
LSNet: See Large, Focus Small
Vision network designs, including Convolutional Neural Networks and Vision Transformers, have significantly advanced the field of computer vision.
Unbiased Max-Min Embedding Classification for Transductive Few-Shot Learning: Clustering and Classification Are All You Need
Our UMMEC method significantly improves classification performance with minimal labeled data, advancing the state-of-the-art in TFSL.
Extremely Simple Out-of-distribution Detection for Audio-visual Generalized Zero-shot Learning
Zero-shot Learning(ZSL) attains knowledge transfer from seen classes to unseen classes by exploring auxiliary category information, which is a promising yet difficult research topic.
YOLOE: Real-Time Seeing Anything
Object detection and segmentation are widely employed in computer vision applications, yet conventional models like YOLO series, while efficient and accurate, are limited by predefined categories, hindering adaptability in open scenarios.
Finedeep: Mitigating Sparse Activation in Dense LLMs via Multi-Layer Fine-Grained Experts
Moreover, we find that Finedeep achieves optimal results when balancing depth and width, specifically by adjusting the number of expert sub-layers and the number of experts per sub-layer.
DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs
As large language models continue to scale, computational costs and resource consumption have emerged as significant challenges.
Rethinking the Upsampling Layer in Hyperspectral Image Super Resolution
Furthermore, we demonstrate, for the first time, that the low-rank property of the learnable upsampling layer is a key bottleneck in lightweight SHSR methods.
Hyperspectral Image Super-Resolution
Image Super-Resolution
+1
Unsupervised Patch-GAN with Targeted Patch Ranking for Fine-Grained Novelty Detection in Medical Imaging
Detecting novel anomalies in medical imaging is challenging due to the limited availability of labeled data for rare abnormalities, which often display high variability and subtlety.
Semi-supervised Semantic Segmentation for Remote Sensing Images via Multi-scale Uncertainty Consistency and Cross-Teacher-Student Attention
To address these problems, this paper proposes a novel semi-supervised Multi-Scale Uncertainty and Cross-Teacher-Student Attention (MUCA) model for RS image semantic segmentation tasks.
YOLO-UniOW: Efficient Universal Open-World Object Detection
In this work, we introduce Universal Open-World Object Detection (Uni-OWD), a new paradigm that unifies open-vocabulary and open-world object detection tasks.
Seamless Detection: Unifying Salient Object Detection and Camouflaged Object Detection
To this end, inspired by the agreeable nature of binary segmentation for SOD and COD, we propose a Contrastive Distillation Paradigm (CDP) to distil the foreground from the background, facilitating the identification of salient and camouflaged objects amidst their surroundings.
[CLS] Token Tells Everything Needed for Training-free Efficient MLLMs
In this paper, we demonstrate that in MLLMs, the [CLS] token in the visual encoder inherently knows which visual tokens are important for MLLMs.
PrefixKV: Adaptive Prefix KV Cache is What Vision Instruction-Following Models Need for Efficient Generation
With an adaptive layer-wise KV retention recipe based on binary search, the maximum contextual information can thus be preserved in each layer, facilitating the generation.
Promptable Anomaly Segmentation with SAM Through Self-Perception Tuning
Segment Anything Model (SAM) has made great progress in anomaly segmentation tasks due to its impressive generalization ability.
Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning
One is the popularly adopted single-teacher paradigm fails to impart comprehensive knowledge, The other is the existing methods inadequately leverage the multimodal information in the original training dataset, instead they rely on additional data for distillation, which increases computational and storage overhead.
A Fresh Look at Generalized Category Discovery through Non-negative Matrix Factorization
Generalized Category Discovery (GCD) aims to classify both base and novel images using labeled base data.
Part-Whole Relational Fusion Towards Multi-Modal Scene Understanding
To tackle the challenge, in this paper, we propose a Part-Whole Relational Fusion (PWRF) framework.
Context Enhancement with Reconstruction as Sequence for Unified Unsupervised Anomaly Detection
By incorporating the RASFormer block, our RAS method achieves superior contextual awareness capabilities, leading to remarkable performance.
Multi-class Anomaly Detection
Unsupervised Anomaly Detection
LLMI3D: Empowering LLM with 3D Perception from a Single 2D Image
Recent advancements in autonomous driving, augmented reality, robotics, and embodied intelligence have necessitated 3D perception algorithms.
Learn from the Learnt: Source-Free Active Domain Adaptation via Contrastive Sampling and Visual Persistence
In response, we present learn from the learnt (LFTL), a novel paradigm for SFADA to leverage the learnt knowledge from the source pretrained model and actively iterated models without extra overhead.
Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective
This work provides a theoretical foundation for understanding and improving PEFT-CL models, offering insights into the interplay between feature representation, task orthogonality, and generalization, contributing to the development of more efficient continual learning systems.
PVUW 2024 Challenge on Complex Video Understanding: Methods and Results
Moreover, we provide a new motion expression guided video segmentation dataset MeViS to study the natural language-guided video understanding in complex environments.
1st Place Solution for MeViS Track in CVPR 2024 PVUW Workshop: Motion Expression guided Video Segmentation
Motion Expression guided Video Segmentation (MeViS), as an emerging task, poses many new challenges to the field of referring video object segmentation (RVOS).
YOLOv10: Real-Time End-to-End Object Detection
In this work, we aim to further advance the performance-efficiency boundary of YOLOs from both the post-processing and model architecture.
Ranked #23 on
Real-Time Object Detection
on MS COCO
Modality Prompts for Arbitrary Modality Salient Object Detection
A novel modality-adaptive Transformer (MAT) will be proposed to investigate two fundamental challenges of AM SOD, ie more diverse modality discrepancies caused by varying modality types that need to be processed, and dynamic fusion design caused by an uncertain number of modalities present in the inputs of multimodal fusion strategy.
Salient Object Detection From Arbitrary Modalities
The most prominent characteristics of AM SOD are that the modality types and modality numbers will be arbitrary or dynamically changed.
Temporal Scaling Law for Large Language Models
In this paper, we propose the novel concept of Temporal Scaling Law, studying how the test loss of an LLM evolves as the training steps scale up.
Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal
Since BPE iteratively merges the most frequent token pair in the text corpus to generate a new token and keeps all generated tokens in the vocabulary, it unavoidably holds tokens that primarily act as components of a longer token and appear infrequently on their own.
Raformer: Redundancy-Aware Transformer for Video Wire Inpainting
Video Wire Inpainting (VWI) is a prominent application in video inpainting, aimed at flawlessly removing wires in films or TV series, offering significant time and labor savings compared to manual frame-by-frame removal.
On Exploring PDE Modeling for Point Cloud Video Representation Learning
We introduce Motion PointNet composed of a PointNet-like encoder and a PDE-solving module.
WaveFace: Authentic Face Restoration with Efficient Frequency Recovery
Although diffusion models are rising as a powerful solution for blind face restoration, they are criticized for two problems: 1) slow training and inference speed, and 2) failure in preserving identity and recovering fine-grained facial details.
PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation
Consequently, a simple combination of them cannot guarantee accomplishing both training efficiency and inference efficiency with minimal costs.
Pixel Sentence Representation Learning
To our knowledge, this is the first representation learning method devoid of traditional language models for understanding sentence and document semantics, marking a stride closer to human-like textual comprehension.
One-dimensional Adapter to Rule Them All: Concepts Diffusion Models and Erasing Applications
The prevalent use of commercial and open-source diffusion models (DMs) for text-to-image generation prompts risk mitigation to prevent undesired behaviors.
One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications
The prevalent use of commercial and open-source diffusion models (DMs) for text-to-image generation prompts risk mitigation to prevent undesired behaviors.
Towards Efficient Vision-Language Tuning: More Information Density, More Generalizability
To better understand the nature of prompt tuning, we propose the concept of ``Information Density'' (ID) to indicate whether a matrix strongly belongs to certain feature spaces rather than being evenly distributed across various feature spaces.
RepViT-SAM: Towards Real-Time Segmenting Anything
Here, to achieve real-time segmenting anything on mobile devices, following MobileSAM, we replace the heavyweight image encoder in SAM with RepViT model, ending up with the RepViT-SAM model.
Virtual Category Learning: A Semi-Supervised Learning Method for Dense Prediction with Extremely Limited Labels
Due to the costliness of labelled data in real-world applications, semi-supervised learning, underpinned by pseudo labelling, is an appealing solution.
CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs
We further employ a consistent dynamic channel pruning (CDCP) strategy to dynamically prune unimportant channels in ViTs.
Beyond One-to-One: Rethinking the Referring Image Segmentation
In this paper, we address this issue from two perspectives.
RepViT: Revisiting Mobile CNN From ViT Perspective
Recently, lightweight Vision Transformers (ViTs) demonstrate superior performance and lower latency, compared with lightweight Convolutional Neural Networks (CNNs), on resource-constrained mobile devices.
Dense Affinity Matching for Few-Shot Segmentation
Few-Shot Segmentation (FSS) aims to segment the novel class images with a few annotated samples.
Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler
In this paper, we propose a novel Knowledge-driven Differential Filter Sampler~(KDFS) with Masked Filter Modeling~(MFM) framework for filter pruning, which globally prunes the redundant filters based on the prior knowledge of a pre-trained model in a differential and non-alternative optimization.
SegGPT Meets Co-Saliency Scene
Co-salient object detection targets at detecting co-existed salient objects among a group of images.
Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels
In this paper, we advocate remedying the deficiency of label supervision for the MLR with incomplete labels by deriving a structured semantic prior about the label-to-label correspondence via a semantic prior prompter.
Efficient RGB-T Tracking via Cross-Modality Distillation
Most current RGB-T trackers adopt a two-stream structure to extract unimodal RGB and thermal features and complex fusion strategies to achieve multi-modal feature fusion, which require a huge number of parameters, thus hindering their real-life applications.
Ranked #12 on
Rgb-T Tracking
on GTOT
Confidence-guided Centroids for Unsupervised Person Re-Identification
Since samples with high confidence are exclusively involved in the formation of centroids, the identity information of low-confidence samples, i. e., boundary samples, are NOT likely to contribute to the corresponding centroid.
Physically-Based Face Rendering for NIR-VIS Face Recognition
Specifically, we reconstruct 3D face shape and reflectance from a large 2D facial dataset and introduce a novel method of transforming the VIS reflectance to NIR reflectance.
Ground Plane Matters: Picking Up Ground Plane Prior in Monocular 3D Object Detection
To pick up the ground plane prior for M3OD, we propose a Ground Plane Enhanced Network (GPENet) which resolves both issues at one go.
LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformers
Transformer with its underlying attention mechanism and the ability to capture long-range dependencies makes it become a natural choice for unordered point cloud data.
Ranked #1 on
3D Semantic Segmentation
on SensatUrban
MAPLE: Masked Pseudo-Labeling autoEncoder for Semi-supervised Point Cloud Action Recognition
In DestFormer, the spatial and temporal dimensions of the 4D point cloud videos are decoupled to achieve efficient self-attention for learning both long-term and short-term features.
Boosting Video-Text Retrieval with Explicit High-Level Semantics
Video-text retrieval (VTR) is an attractive yet challenging task for multi-modal understanding, which aims to search for relevant video (text) given a query (video).
Temporal Saliency Query Network for Efficient Video Recognition
For each category, the common pattern of it is employed as a query and the most salient frames are responded to it.
Ranked #5 on
Action Recognition
on ActivityNet
Semi-supervised Object Detection via Virtual Category Learning
Due to the costliness of labelled data in real-world applications, semi-supervised object detectors, underpinned by pseudo labelling, are appealing.
Re-parameterizing Your Optimizers rather than Architectures
For the extreme simplicity of model structure, we focus on a VGG-style plain model and showcase that such a simple model trained with a RepOptimizer, which is referred to as RepOpt-VGG, performs on par with or better than the recent well-designed models.
Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs
We revisit large kernel design in modern convolutional neural networks (CNNs).
Ranked #68 on
Image Classification
on ImageNet
On Exploring Pose Estimation as an Auxiliary Learning Task for Visible-Infrared Person Re-identification
Visible-infrared person re-identification (VI-ReID) has been challenging due to the existence of large discrepancies between visible and infrared modalities.
Cross-Modality Deep Feature Learning for Brain Tumor Segmentation
Recent advances in machine learning and prevalence of digital medical images have opened up an opportunity to address the challenging brain tumor segmentation (BTS) task by using deep convolutional neural networks.
FMCNet: Feature-Level Modality Compensation for Visible-Infrared Person Re-Identification
Then, a feature-level modality compensation module is present to generate those missing modality-specific features from existing modality-shared ones.
RepMLPNet: Hierarchical Vision MLP with Re-parameterized Locality
Our results reveal that 1) Locality Injection is a general methodology for MLP models; 2) RepMLPNet has favorable accuracy-efficiency trade-off compared to the other MLPs; 3) RepMLPNet is the first MLP that seamlessly transfer to Cityscapes semantic segmentation.
Ranked #70 on
Semantic Segmentation
on Cityscapes val
LODE: Deep Local Deblurring and A New Benchmark
While recent deep deblurring algorithms have achieved remarkable progress, most existing methods focus on the global deblurring problem, where the image blur mostly arises from severe camera shake.
Information Symmetry Matters: A Modal-Alternating Propagation Network for Few-Shot Learning
Semantic information provides intra-class consistency and inter-class discriminability beyond visual concepts, which has been employed in Few-Shot Learning (FSL) to achieve further gains.
Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN
The existence of redundancy in Convolutional Neural Networks (CNNs) enables us to remove some filters/channels with acceptable performance drops.
ABMDRNet: Adaptive-Weighted Bi-Directional Modality Difference Reduction Network for RGB-T Semantic Segmentation
Semantic segmentation models gain robustness against poor lighting conditions by virtue of complementary information from visible (RGB) and thermal images.
Ranked #34 on
Thermal Image Segmentation
on MFN Dataset
RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition
We propose RepMLP, a multi-layer-perceptron-style neural network building block for image recognition, which is composed of a series of fully-connected (FC) layers.
Ranked #818 on
Image Classification
on ImageNet
Middle-level Fusion for Lightweight RGB-D Salient Object Detection
The former one first uses two sub-networks to extract unimodal features from RGB and depth images, respectively, and then fuses them for SOD.
Exploring Modality-shared Appearance Features and Modality-invariant Relation Features for Cross-modality Person Re-Identification
Most existing cross-modality person re-identification works rely on discriminative modality-shared features for reducing cross-modality variations and intra-modality variations.
Cross-Modality Person Re-identification
Person Re-Identification
Few-Cost Salient Object Detection with Adversarial-Paced Learning
A fundamental challenge in training the existing deep saliency detection models is the requirement of large amounts of annotated data.
Onfocus Detection: Identifying Individual-Camera Eye Contact from Unconstrained Images
Onfocus detection aims at identifying whether the focus of the individual captured by a camera is on the camera or not.
Diverse Branch Block: Building a Convolution as an Inception-like Unit
We propose a universal building block of Convolutional Neural Network (ConvNet) to improve the performance without any inference-time costs.
Densely Nested Top-Down Flows for Salient Object Detection
To this end, this paper revisits the role of top-down modeling in salient object detection and designs a novel densely nested top-down flows (DNTDF)-based framework.
RepVGG: Making VGG-style ConvNets Great Again
We present a simple but powerful architecture of convolutional neural network, which has a VGG-like inference-time body composed of nothing but a stack of 3x3 convolution and ReLU, while the training-time model has a multi-branch topology.
Ranked #49 on
Semantic Segmentation
on Cityscapes val
Perception Consistency Ultrasound Image Super-resolution via Self-supervised CycleGAN
Due to the limitations of sensors, the transmission medium and the intrinsic properties of ultrasound, the quality of ultrasound imaging is always not ideal, especially its low spatial resolution.
ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting
Via training with regular SGD on the former but a novel update rule with penalty gradients on the latter, we realize structured sparsity.
Shallow Feature Based Dense Attention Network for Crowd Counting
In this paper, we propose a Shallow feature based Dense Attention Network (SDANet) for crowd counting from still images, which diminishes the impact of backgrounds via involving a shallow feature based attention model, and meanwhile, captures multi-scale information via densely connecting hierarchical image features.
IMRAM: Iterative Matching with Recurrent Attention Memory for Cross-Modal Image-Text Retrieval
Existing methods leverage the attention mechanism to explore such correspondence in a fine-grained manner.
Ranked #20 on
Cross-Modal Retrieval
on Flickr30k
NAS-Count: Counting-by-Density with Neural Architecture Search
Most of the recent advances in crowd counting have evolved from hand-designed density estimation networks, where multi-scale features are leveraged to address the scale variation problem, but at the expense of demanding design efforts.
2.75D: Boosting learning by representing 3D Medical imaging to 2D features for small data
Such performance gain is more pronounced with transfer learning or in the case of limited training data.
Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification
We refer to these models as 'Experts', and the proposed LFME framework aggregates the knowledge from multiple 'Experts' to learn a unified student model.
Ranked #27 on
Long-tail Learning
on Places-LT
Aggregation Signature for Small Object Tracking
In this paper, we propose a novel aggregation signature suitable for small object tracking, especially aiming for the challenge of sudden and large drift.
Global Sparse Momentum SGD for Pruning Very Deep Neural Networks
Deep Neural Network (DNN) is powerful but computationally expensive and memory intensive, thus impeding its practical usage on resource-constrained front-end devices.
Episode-based Prototype Generating Network for Zero-Shot Learning
We introduce a simple yet effective episode-based training framework for zero-shot learning (ZSL), where the learning system requires to recognize unseen classes given only the corresponding class semantics.
ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks
We propose Asymmetric Convolution Block (ACB), an architecture-neutral structure as a CNN building block, which uses 1D asymmetric convolutions to strengthen the square convolution kernels.
Incremental Few-Shot Learning for Pedestrian Attribute Recognition
Pedestrian attribute recognition has received increasing attention due to its important role in video surveillance applications.
Approximated Oracle Filter Pruning for Destructive CNN Width Optimization
It is not easy to design and run Convolutional Neural Networks (CNNs) due to: 1) finding the optimal number of filters (i. e., the width) at each layer is tricky, given an architecture; and 2) the computational intensity of CNNs impedes the deployment on computationally limited devices.
Saliency-Guided Attention Network for Image-Sentence Matching
Concretely, the saliency detector provides the visual saliency information as the guidance for the two attention modules.
Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure
The redundancy is widely recognized in Convolutional Neural Networks (CNNs), which enables to remove unimportant filters from convolutional layers so as to slim the network with acceptable performance drop.
Pixelated Semantic Colorization
We propose two ways to incorporate object semantics into the colorization model: through a pixelated semantic embedding and a pixelated semantic generator.
Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation
The advancement of deep convolutional neural networks (DCNNs) has driven significant improvement in the accuracy of recognition systems for many computer vision tasks.
Pixel-level Semantics Guided Image Colorization
While many image colorization algorithms have recently shown the capability of producing plausible color versions from gray-scale photographs, they still suffer from the problems of context confusion and edge color bleeding.
Modulated Convolutional Networks
In this paper, we propose new Modulated Convolutional Networks (MCNs) to improve the portability of CNNs via binarized filters.
Memory Attention Networks for Skeleton-based Action Recognition
Specifically, the TARM is deployed in a residual learning module that employs a novel attention learning network to recalibrate the temporal attention of frames in a skeleton sequence.
Ranked #104 on
Skeleton Based Action Recognition
on NTU RGB+D
One-Two-One Networks for Compression Artifacts Reduction in Remote Sensing
Compression artifacts reduction (CAR) is a challenging problem in the field of remote sensing.
The Structure Transfer Machine Theory and Applications
Representation learning is a fundamental but challenging problem, especially when the distribution of data is unknown.
Attribute-Guided Network for Cross-Modal Zero-Shot Hashing
To address the Cross-Modal Zero-Shot Hashing (CMZSH) retrieval task, we propose a novel Attribute-Guided Network (AgNet), which can perform not only IBIR, but also Text-Based Image Retrieval (TBIR).
Latent Constrained Correlation Filter
In this paper, we introduce an intermediate step -- solution sampling -- after the data sampling step to form a subspace, in which an optimal solution can be estimated.
Deep Fisher Discriminant Learning for Mobile Hand Gesture Recognition
Gesture recognition is a challenging problem in the field of biometrics.
Ranked #1 on
Hand Gesture Recognition
on MGB
Deep Spatio-temporal Manifold Network for Action Recognition
Visual data such as videos are often sampled from complex manifold.
From Zero-shot Learning to Conventional Supervised Classification: Unseen Visual Data Synthesis
Using the proposed Unseen Visual Data Synthesis (UVDS) algorithm, semantic attributes are effectively utilised as an intermediate clue to synthesise unseen visual features at the training stage.
Gabor Convolutional Networks
Steerable properties dominate the design of traditional filters, e. g., Gabor filters, and endow features the capability of dealing with spatial transformations.
Sparse Representation based Multi-sensor Image Fusion: A Review
As a result of several successful applications in computer vision and image processing, sparse representation (SR) has attracted significant attention in multi-sensor image fusion.
Latent Constrained Correlation Filters for Object Localization
There is a neglected fact in the traditional machine learning methods that the data sampling can actually lead to the solution sampling.
