Search Results for author:
Found 140 papers, 86 papers with code
RadioDUN: A Physics-Inspired Deep Unfolding Network for Radio Map Estimation
Inspired by the shadowing factor in the physical propagation model, we integrate obstacle-related factors to express the obstacle-induced signal stochastic decay.
Adversarial Semantic and Label Perturbation Attack for Pedestrian Attribute Recognition
To bridge this gap, this paper proposes the first adversarial attack and defense framework for pedestrian attribute recognition.
Your Language Model Can Secretly Write Like Humans: Contrastive Paraphrase Attacks on LLM-Generated Text Detectors
By subtracting the machine-like patterns from the human-like distribution during the decoding process, CoPA is able to produce sentences that are less discernible by text detectors.
High Quality Underwater Image Compression with Adaptive Correction and Codebook-based Augmentation
With the increasing exploration and exploitation of the underwater world, underwater images have become a critical medium for human interaction with marine environments, driving extensive research into their efficient transmission and storage.
Towards Facial Image Compression with Consistency Preserving Diffusion Prior
Simply adapting diffusion-based compression methods to facial compression tasks results in reconstructed images that perform poorly in downstream applications due to insufficient preservation of high-frequency information.
CountDiffusion: Text-to-Image Synthesis with Training-Free Counting-Guidance Diffusion
Stable Diffusion has advanced text-to-image synthesis, but training models to generate images with accurate object quantity is still difficult due to the high computational cost and the challenge of teaching models the abstract concept of quantity.
Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning
Visual In-Context Learning (VICL) enables adaptively solving vision tasks by leveraging pixel demonstrations, mimicking human-like task completion through analogy.
GraphATC: advancing multilevel and multi-label anatomical therapeutic chemical classification via atom-level graph learning
The accurate categorization of compounds within the anatomical therapeutic chemical (ATC) system is fundamental for drug development and fundamental research.
Ranked #1 on
Drug ATC Classification
on ATC-SMILES
VideoVista-CulturalLingo: 360$^\circ$ Horizons-Bridging Cultures, Languages, and Domains in Video Comprehension
Assessing the video comprehension capabilities of multimodal AI systems can effectively measure their understanding and reasoning abilities.
Harmony: A Unified Framework for Modality Incremental Learning
To address this task, we propose a novel framework named Harmony, designed to achieve modal alignment and knowledge retention, enabling the model to reduce the modal discrepancy and learn from a sequence of distinct modalities, ultimately completing tasks across multiple modalities within a unified framework.
Learning Compatible Multi-Prize Subnetworks for Asymmetric Retrieval
For example, when introducing a new platform into the retrieval systems, developers have to train an additional model at an appropriate capacity that is compatible with existing models via backward-compatible learning.
A Unified Agentic Framework for Evaluating Conditional Image Generation
This paper introduces CIGEval, a unified agentic framework for comprehensive evaluation of conditional image generation tasks.
Human Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset
In this work, we rethink human activity recognition by combining the RGB and event cameras.
AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing
To address this limitation, we propose a new framework, termed AutoSSVH, that employs adversarial frame sampling with hash-based contrastive learning.
Representation-based Reward Modeling for Efficient Safety Alignment of Large Language Model
Reinforcement Learning (RL) algorithms for safety alignment of Large Language Models (LLMs), such as Direct Preference Optimization (DPO), encounter the challenge of distribution shift.
Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understanding
To address these challenges, we propose learning the mapping between exocentric and egocentric domains, leveraging the extensive exocentric knowledge within existing MLLMs to enhance egocentric video understanding.
Sign Language Translation using Frame and Event Stream: Benchmark Dataset and Algorithms
Accurate sign language understanding serves as a crucial communication channel for individuals with disabilities.
DSPNet: Dual-vision Scene Perception for Robust 3D Question Answering
In this paper, we propose a Dual-vision Scene Perception Network (DSPNet), to comprehensively integrate multi-view and point cloud features to improve robustness in 3D QA.
OpenEarthSensing: Large-Scale Fine-Grained Benchmark for Open-World Remote Sensing
In open-world remote sensing, deployed models must continuously adapt to a steady influx of new data, which often exhibits various shifts compared to what the model encountered during the training phase.
Picking the Cream of the Crop: Visual-Centric Data Selection with Collaborative Agents
Specifically, our approach consists of 1) an image information quantification method via visual agents collaboration to select images with rich visual information, and 2) a visual-centric instruction quality assessment method to select high-quality instruction data related to high-quality images.
SelaVPR++: Towards Seamless Adaptation of Foundation Models for Efficient Place Recognition
In our previous work, we propose a novel method to realize seamless adaptation of foundation models to VPR (SelaVPR).
Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis
Then, we conduct extensive experiments with the baseline within each class, covering models with various sizes (7B-70B), bitwidths, training levels (LLaMA1/2/3/3. 1), architectures (Mixtral, DeepSeekMoE and Mamba) and modality (LLaVA1. 5 and VILA1. 5) on a wide range of evaluation metrics. Through comparative analysis on the results, we summarize the superior of each PTQ strategy and modelsize-bitwidth trade-off considering the performance.
PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models
To explore the real limit of PTQ, we propose an extremely low-bit PTQ method called PTQ1. 61, which enables weight quantization to 1. 61-bit for the first time.
EventSTR: A Benchmark Dataset and Baselines for Event Stream based Scene Text Recognition
Mainstream Scene Text Recognition (STR) algorithms are developed based on RGB cameras which are sensitive to challenging factors such as low illumination, motion blur, and cluttered backgrounds.
Pilot: Building the Federated Multimodal Instruction Tuning Framework
In this paper, we explore a novel federated multimodal instruction tuning task(FedMIT), which is significant for collaboratively fine-tuning MLLMs on different types of multimodal instruction data on distributed devices.
Activating Associative Disease-Aware Vision Token Memory for LLM-Based X-ray Report Generation
X-ray image based medical report generation achieves significant progress in recent years with the help of the large language model, however, these models have not fully exploited the effective information in visual image regions, resulting in reports that are linguistically sound but insufficient in describing key diseases.
Building Vision Models upon Heat Conduction
Visual representation models leveraging attention mechanisms are challenged by significant computational overhead, particularly when pursuing large receptive fields.
NN-Former: Rethinking Graph Structure in Neural Architecture Representation
Our approach consistently achieves promising performance in both accuracy and latency prediction, providing valuable insights for learning Directed Acyclic Graph (DAG) topology.
Sound Bridge: Associating Egocentric and Exocentric Videos via Audio Cues
However, existing visual-to-visual and visual-to-textual Ego-Exo video alignment methods struggle with the problem that there could be non-visual overlap for the same activity.
Video Language Model Pretraining with Spatio-temporal Masking
The development of self-supervised video-language models based on mask learning has significantly advanced downstream video tasks.
VELoRA: A Low-Rank Adaptation Approach for Efficient RGB-Event based Recognition
Inspired by the successful application of large models, the introduction of such large models can also be considered to further enhance the performance of multi-modal tasks.
Towards Visual Grounding: A Survey
Finally, we outline the challenges confronting visual grounding and propose valuable directions for future research, which may serve as inspiration for subsequent researchers.
Core Context Aware Attention for Long Context Language Modeling
As a result, the computational and memory complexity will be significantly reduced.
Efficient Dataset Distillation via Diffusion-Driven Patch Selection for Improved Generalization
Dataset distillation offers an efficient way to reduce memory and computational costs by optimizing a smaller dataset with performance comparable to the full-scale original.
Towards Long Video Understanding via Fine-detailed Video Story Generation
To avoid interference from redundant information in videos, we introduce a Semantic Redundancy Reduction mechanism that removes redundancy at both the visual and textual levels.
Do We Need to Design Specific Diffusion Models for Different Tasks? Try ONE-PIC
However, unlike Large Language Models (LLMs) that can learn multiple tasks in a single model based on instructed data, diffusion models always require additional branches, task-specific training strategies, and losses for effective adaptation to different downstream tasks.
CATCH: Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs
Large Vision-Language Model (LVLM) systems have demonstrated impressive vision-language reasoning capabilities but suffer from pervasive and severe hallucination issues, posing significant risks in critical domains such as healthcare and autonomous systems.
OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modeling
Simultaneously, the current mask visual language modeling (MVLM) fails to capture the nuanced referential relationship between image-text in referring tasks.
EMMA: Empowering Multi-modal Mamba with Structural and Hierarchical Alignment
In this work, we propose Empowering Multi-modal Mamba with Structural and Hierarchical Alignment (EMMA), which enables the MLLM to extract fine-grained visual information.
LSVOS Challenge Report: Large-scale Complex and Long Video Object Segmentation
Despite the promising performance of current video segmentation models on existing benchmarks, these models still struggle with complex scenes.
Discriminative Spatial-Semantic VOS Solution: 1st Place Solution for 6th LSVOS
Video object segmentation (VOS) is a crucial task in computer vision, but current VOS methods struggle with complex scenes and prolonged object motions.
Event Stream based Sign Language Translation: A High-Definition Benchmark Dataset and A New Algorithm
Based on this dataset and several other large-scale datasets, we propose a novel baseline method that fully leverages the Mamba model's ability to integrate temporal information of CNN features, resulting in improved sign language translation outcomes.
Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers
Semantic pattern of an object point cloud is determined by its topological configuration of local geometries.
Reshaping the Online Data Buffering and Organizing Mechanism for Continual Test-Time Adaptation
In this paper, we systematically analyze the challenges of this task: online environment, unsupervised nature, and the risks of error accumulation and catastrophic forgetting under continual domain shifts.
OV-DINO: Unified Open-Vocabulary Detection with Language-Aware Selective Fusion
To address these challenges, we propose a novel unified open-vocabulary detection method called OV-DINO, which is pre-trained on diverse large-scale datasets with language-aware selective fusion in a unified framework.
Ranked #5 on
Zero-Shot Object Detection
on MSCOCO
(AP metric, using extra
training data)
Learning Spatial-Semantic Features for Robust Video Object Segmentation
Tracking and segmenting multiple similar objects with complex or separate parts in long-term videos is inherently challenging due to the ambiguity of target parts and identity confusion caused by occlusion, background clutter, and long-term variations.
Retain, Blend, and Exchange: A Quality-aware Spatial-Stereo Fusion Approach for Event Stream Recognition
In this paper, we propose a novel dual-stream framework for event stream-based pattern recognition via differentiated fusion, termed EFV++.
PVUW 2024 Challenge on Complex Video Understanding: Methods and Results
Moreover, we provide a new motion expression guided video segmentation dataset MeViS to study the natural language-guided video understanding in complex environments.
1st Place Solution for MOSE Track in CVPR 2024 PVUW Workshop: Complex Video Object Segmentation
In this challenge, we propose a semantic embedding video object segmentation model and use the salient features of objects as query representations.
vHeat: Building Vision Models upon Heat Conduction
A fundamental problem in learning robust and expressive visual representations lies in efficiently estimating the spatial relationships of visual semantics throughout the entire image.
M$^3$GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generation
Third, M$^3$GPT learns to model the connections and synergies among various motion-relevant tasks.
MambaVC: Learned Visual Compression with Selective State Spaces
Learned visual compression is an important and active task in multimedia.
LG-VQ: Language-Guided Codebook Learning
Vector quantization (VQ) is a key technique in high-resolution and high-fidelity image synthesis, which aims to learn a codebook to encode an image with a sequence of discrete codes and then generate an image in an auto-regression manner.
Harnessing Vision-Language Pretrained Models with Temporal-Aware Adaptation for Referring Video Object Segmentation
As images and texts are mapped to uncoupled feature spaces, they face the arduous task of learning Vision-Language (VL) relation modeling from scratch.
Referring Expression Segmentation
Referring Video Object Segmentation
+3
Prompt Customization for Continual Learning
In contrast to conventional methods that employ hard prompt selection, PGM assigns different coefficients to prompts from a fixed-sized pool of prompts and generates tailored prompts.
Spatio-Temporal Side Tuning Pre-trained Foundation Models for Video-based Pedestrian Attribute Recognition
Specifically, we formulate the video-based PAR as a vision-language fusion problem and adopt a pre-trained foundation model CLIP to extract the visual features.
Motion-aware Latent Diffusion Models for Video Frame Interpolation
With the advancement of AIGC, video frame interpolation (VFI) has become a crucial component in existing video generation frameworks, attracting widespread research interest.
HiVG: Hierarchical Multimodal Fine-grained Modulation for Visual Grounding
The cross-modal bridge can address the inconsistency between visual features and those required for grounding, and establish a connection between multi-level visual and text features.
State Space Model for New-Generation Network Alternative to Transformers: A Survey
In this paper, we give the first comprehensive review of these works and also provide experimental comparisons and analysis to better demonstrate the features and advantages of SSM.
StoryImager: A Unified and Efficient Framework for Coherent Story Visualization and Completion
3) The story visualization and continuation models are trained and inferred independently, which is not user-friendly.
RTracker: Recoverable Tracking via PN Tree Structured Memory
Upon the PN tree memory, we develop corresponding walking rules for determining the state of the target and define a set of control flows to unite the tracker and the detector in different tracking scenarios.
Ranked #1 on
Visual Object Tracking
on VideoCube
Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance
The essence of our work lies in adapting LoRA, a technique that fine-tunes a small subset of model parameters without adding inference latency, to the domain of visual tracking.
Ranked #1 on
Visual Object Tracking
on UAV123
CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition
Over the past decade, most methods in visual place recognition (VPR) have used neural networks to produce feature representations.
Prompt-Driven Dynamic Object-Centric Learning for Single Domain Generalization
In this paper, we propose a dynamic object-centric perception network based on prompt learning, aiming to adapt to the variations in image complexity.
Towards Robust and Efficient Cloud-Edge Elastic Model Adaptation via Selective Entropy Distillation
Moreover, with the increasing data collected at the edge, this paradigm also fails to further adapt the cloud model for better performance.
Semi-supervised Counting via Pixel-by-pixel Density Distribution Modelling
This paper focuses on semi-supervised crowd counting, where only a small portion of the training data are labeled.
Towards Seamless Adaptation of Pre-trained Models for Visual Place Recognition
Experimental results show that our method outperforms the state-of-the-art methods with less training data and training time, and uses about only 3% retrieval runtime of the two-stage VPR methods with RANSAC-based spatial verification.
Ranked #4 on
Visual Place Recognition
on Pittsburgh-250k-test
Virtual Classification: Modulating Domain-Specific Knowledge for Multidomain Crowd Counting
Multidomain crowd counting aims to learn a general model for multiple diverse datasets.
MB-RACS: Measurement-Bounds-based Rate-Adaptive Image Compressed Sensing Network
In this paper, we propose a Measurement-Bounds-based Rate-Adaptive Image Compressed Sensing Network (MB-RACS) framework, which aims to adaptively determine the sampling rate for each image block in accordance with traditional measurement bounds theory.
VMamba: Visual State Space Model
At the core of VMamba is a stack of Visual State-Space (VSS) blocks with the 2D Selective Scan (SS2D) module.
Modality-Collaborative Test-Time Adaptation for Action Recognition
Existing image-based TTA methods cannot be directly applied to this task because video have domain shift in multimodal and temporal which brings difficulties to adaptation.
FFCA-Net: Stereo Image Compression via Fast Cascade Alignment of Side Information
To address this limitation, we propose a Feature-based Fast Cascade Alignment network (FFCA-Net) to fully leverage the side information on the decoder.
Regressor-Segmenter Mutual Prompt Learning for Crowd Counting
In this study, we propose mutual prompt learning (mPrompt), which leverages a regressor and a segmenter as guidance for each other, solving bias and inaccuracy caused by annotation variance while distinguishing foreground from background.
Recognizing Conditional Causal Relationships about Emotions and Their Corresponding Conditions
To highlight the context in such special causal relationships, we propose a new task to determine whether or not an input pair of emotion and cause has a valid causal relationship under different contexts and extract the specific context clauses that participate in the causal relationship.
Uncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog
In response to these challenges, we present an iterative search and reasoning framework, which consists of a textual encoder, a visual encoder, and a generator.
Learning Mask-aware CLIP Representations for Zero-Shot Segmentation
However, in the paper, we reveal that CLIP is insensitive to different mask proposals and tends to produce similar predictions for various mask proposals of the same image.
Ranked #10 on
Open Vocabulary Semantic Segmentation
on PascalVOC-20
Open Vocabulary Semantic Segmentation
Zero Shot Segmentation
CLIP-based Synergistic Knowledge Transfer for Text-based Person Retrieval
Text-based Person Retrieval (TPR) aims to retrieve the target person images given a textual query.
MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual Captioning
To deal with the label shortage problem, we present a simple yet effective zero-shot approach MultiCapCLIP that can generate visual captions for different scenarios and languages without any labeled vision-caption pairs of downstream datasets.
CiteTracker: Correlating Image and Text for Visual Tracking
Existing visual tracking methods typically take an image patch as the reference of the target to perform tracking.
MixBCT: Towards Self-Adapting Backward-Compatible Training
Instance-based methods like L2 regression take into account the distribution of old features but impose strong constraints on the performance of the new model itself.
Benign Shortcut for Debiasing: Fair Visual Recognition via Intervention with Shortcut Features
To this end, we propose \emph{Shortcut Debiasing}, to first transfer the target task's learning of bias attributes from bias features to shortcut features, and then employ causal intervention to eliminate shortcut features during inference.
Strip-MLP: Efficient Token Interaction for Vision MLP
Finally, based on the Strip MLP layer, we propose a novel \textbf{L}ocal \textbf{S}trip \textbf{M}ixing \textbf{M}odule (LSMM) to boost the token interaction power in the local region.
HQG-Net: Unpaired Medical Image Enhancement with High-Quality Guidance
Specifically, we extract features from an HQ image and explicitly insert the features, which are expected to encode HQ cues, into the enhancement network to guide the LQ enhancement with the variational normalization module.
Improving Deep Representation Learning via Auxiliary Learnable Target Coding
Deep representation learning is a subfield of machine learning that focuses on learning meaningful and useful representations of data through deep neural networks.
ShuffleMix: Improving Representations via Channel-Wise Shuffle of Interpolated Hidden States
Inspired by good robustness of alternative dropout strategies against over-fitting on limited patterns of training samples, this paper introduces a novel concept of ShuffleMix -- Shuffle of Mixed hidden features, which can be interpreted as a kind of dropout operation in feature space.
Manifold-Aware Self-Training for Unsupervised Domain Adaptation on Regressing 6D Object Pose
Domain gap between synthetic and real data in visual regression (e. g. 6D pose estimation) is bridged in this paper via global feature alignment and local refinement on the coarse classification of discretized anchor classes in target space, which imposes a piece-wise target manifold regularization into domain-invariant representation learning.
CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Grounding
In order to utilize vision and language pre-trained models to address the grounding problem, and reasonably take advantage of pseudo-labels, we propose CLIP-VG, a novel method that can conduct self-paced curriculum adapting of CLIP with pseudo-language labels.
Towards Efficient Task-Driven Model Reprogramming with Foundation Models
Moreover, the data used for pretraining foundation models are usually invisible and very different from the target data of downstream tasks.
KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigation
The proposed KERM can automatically select and gather crucial and relevant cues, obtaining more accurate action prediction.
Reliability-Hierarchical Memory Network for Scribble-Supervised Video Object Segmentation
To be specific, RHMNet first only uses the memory in the high-reliability level to locate the region with high reliability belonging to the target, which is highly similar to the initial target scribble.
ZeroNLG: Aligning and Autoencoding Domains for Zero-Shot Multimodal and Multilingual Natural Language Generation
We present the results of extensive experiments on twelve NLG tasks, showing that, without using any labeled downstream pairs for training, ZeroNLG generates high-quality and believable outputs and significantly outperforms existing zero-shot methods.
Unsupervised Domain Adaptation via Distilled Discriminative Clustering
Differently, motivated by the fundamental assumption for domain adaptability, we re-cast the domain adaptation problem as discriminative clustering of target data, given strong privileged information provided by the closely related, labeled source data.
Ranked #1 on
Unsupervised Domain Adaptation
on VisDA2017
(Average Accuracy metric)
Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey
We also give visualization and analysis of the model parameters and results on representative downstream tasks.
DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition
In this work, we explore effective Vision Transformers to pursue a preferable trade-off between the computational complexity and size of the attended receptive field.
CIGAR: Cross-Modality Graph Reasoning for Domain Adaptive Object Detection
To overcome these problems, we propose a cross-modality graph reasoning adaptation (CIGAR) method to take advantage of both visual and linguistic knowledge.
AsyFOD: An Asymmetric Adaptation Paradigm for Few-Shot Domain Adaptive Object Detection
Critically, in FSDAOD, the data-scarcity in the target domain leads to an extreme data imbalance between the source and target domains, which potentially causes over-adaptation in traditional feature alignment.
Unlearnable Clusters: Towards Label-agnostic Unlearnable Examples
Furthermore, we propose to leverage VisionandLanguage Pre-trained Models (VLPMs) like CLIP as the surrogate model to improve the transferability of the crafted UCs to diverse domains.
Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interference
To avoid obvious stage learning bottlenecks, we propose a brand-new stage-isolation based incremental learning framework, which leverages a series of stage-isolated classifiers to perform the learning task of each stage without the interference of others.
SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classification
Although significant progress has been made in few-shot learning, most of existing few-shot image classification methods require supervised pre-training on a large amount of samples of base classes, which limits their generalization ability in real world application.
Fast-iTPN: Integrally Pre-Trained Transformer Pyramid Network with Token Migration
iTPN is born with two elaborated designs: 1) The first pre-trained feature pyramid upon vision transformer (ViT).
Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric
In this paper, we propose a single-stage backbone network for Color-Event Unified Tracking (CEUTrack), which achieves the above functions simultaneously.
Ranked #3 on
Object Tracking
on COESOT
HARDVS: Revisiting Human Activity Recognition with Dynamic Vision Sensors
The main streams of human activity recognition (HAR) algorithms are developed based on RGB cameras which are suffered from illumination, fast motion, privacy-preserving, and large energy consumption.
Spikformer: When Spiking Neural Network Meets Transformer
Spikformer (66. 3M parameters) with comparable size to SEW-ResNet-152 (60. 2M, 69. 26%) can achieve 74. 81% top1 accuracy on ImageNet using 4 time steps, which is the state-of-the-art in directly trained SNNs models.
Semi-supervised Crowd Counting via Density Agency
In this paper, we propose a new agency-guided semi-supervised counting approach.
Learned Distributed Image Compression with Multi-Scale Patch Matching in Feature Domain
Specifically, MSFDPM consists of a side information feature extractor, a multi-scale feature domain patch matching module, and a multi-scale feature fusion network.
Identifying the kind behind SMILES—anatomical therapeutic chemical classification using structure-only representations
The model is with better explainability in the sense that it is consists of a straightforward tokenization that extracts and embeds statistically and physicochemically meaningful tokens, and a deep network backed by a set of pyramid kernels to capture multi-resolution chemical structural characteristics.
Ranked #2 on
Drug ATC Classification
on ATC-SMILES
DAS: Densely-Anchored Sampling for Deep Metric Learning
To this end, we propose a Densely-Anchored Sampling (DAS) scheme that considers the embedding with corresponding data point as "anchor" and exploits the anchor's nearby embedding space to densely produce embeddings without data points.
Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval
Our goal in this research is to study a more realistic environment in which we can conduct weakly-supervised multi-modal instance-level product retrieval for fine-grained product categories.
Prompt-based Learning for Unpaired Image Captioning
Inspired by the success of Vision-Language Pre-Trained Models (VL-PTMs) in this research, we attempt to infer the cross-domain cue information about a given image from the large VL-PTMs for the UIC task.
Global-Supervised Contrastive Loss and View-Aware-Based Post-Processing for Vehicle Re-Identification
The proposed VABPP method is the first time that the view-aware-based method is used as a post-processing method in the field of vehicle re-identification.
Fine-Grained Object Classification via Self-Supervised Pose Alignment
Semantic patterns of fine-grained objects are determined by subtle appearance difference of local parts, which thus inspires a number of part-based methods.
Boost Test-Time Performance with Closed-Loop Inference
Motivated by this, we propose to predict those hard-classified test samples in a looped manner to boost the model performance.
Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance
For example, MPQ search on ResNet18 with our indicators takes only 0. 06 s, which improves time efficiency exponentially compared to iterative search methods.
Unpaired Image Captioning by Image-level Weakly-Supervised Visual Concept Recognition
The image-level labels are utilized to train a weakly-supervised object recognition model to extract object information (e. g., instance) in an image, and the extracted instances are adopted to infer the relationships among different objects based on an enhanced graph neural network (GNN).
Boosting Crowd Counting via Multifaceted Attention
Secondly, we design the Local Attention Regularization to supervise the training of LRA by minimizing the deviation among the attention for different feature locations.
Conceptor Learning for Class Activation Mapping
Class Activation Mapping (CAM) has been widely adopted to generate saliency maps which provides visual explanations for deep neural networks (DNNs).
Towards End-to-End Image Compression and Analysis with Transformers
Meanwhile, we propose a feature aggregation module to fuse the compressed features with the selected intermediate features of the Transformer, and feed the aggregated features to a deconvolutional neural network for image reconstruction.
Frequency Spectrum Augmentation Consistency for Domain Adaptive Object Detection
In the second stage, augmented source and target data with pseudo labels are adopted to perform the self-training for prediction consistency.
Learning to Share in Multi-Agent Reinforcement Learning
Inspired by the fact that sharing plays a key role in human's learning of cooperation, we propose LToS, a hierarchically decentralized MARL framework that enables agents to learn to dynamically share reward with neighbors so as to encourage agents to cooperate on the global objective through collectives.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
An Informative Tracking Benchmark
Along with the rapid progress of visual tracking, existing benchmarks become less informative due to redundancy of samples and weak discrimination between current trackers, making evaluations on all datasets extremely time-consuming.
Optimized Separable Convolution: Yet Another Efficient Convolution Operator
In this research, we propose a novel and principled operator called optimized separable convolution by optimal design for the internal number of groups and kernel sizes for general separable convolutions can achieve the complexity of O(C^{\frac{3}{2}}K).
M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining
Despite the potential of multi-modal pre-training to learn highly discriminative feature representations from complementary data modalities, current progress is being slowed by the lack of large-scale modality-diverse datasets.
VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows
Different from visible cameras which record intensity images frame by frame, the biologically inspired event camera produces a stream of asynchronous and sparse events with much lower latency.
Ranked #1 on
Object Tracking
on VisEvent
MFGNet: Dynamic Modality-Aware Filter Generation for RGB-T Tracking
The visible and thermal filters will be used to conduct a dynamic convolutional operation on their corresponding input feature maps respectively.
Ranked #41 on
Rgb-T Tracking
on RGBT234
Direct Measure Matching for Crowd Counting
Second, we derive a semi-balanced form of Sinkhorn divergence, based on which a Sinkhorn counting loss is designed for measure matching.
Self-Supervised Tracking via Target-Aware Data Synthesis
While deep-learning based tracking methods have achieved substantial progress, they entail large-scale and high-quality annotated data for sufficient training.
Learning Scalable lY=-Constrained Near-Lossless Image Compression via Joint Lossy Image and Residual Compression
To achieve scalable compression with the error bound larger than zero, we derive the probability model of the quantized residual by quantizing the learned probability model of the original residual, instead of training multiple networks.
Tracking by Joint Local and Global Search: A Target-aware Attention based Approach
In this paper, we propose a novel and general target-aware attention mechanism (termed TANet) and integrate it with tracking-by-detection framework to conduct joint local and global search for robust tracking.
Conformer: Local Features Coupling Global Representations for Visual Recognition
Within Convolutional Neural Network (CNN), the convolution operations are good at extracting local features but experience difficulty to capture global representations.
Anomaly Detection with Prototype-Guided Discriminative Latent Embeddings
Recent efforts towards video anomaly detection (VAD) try to learn a deep autoencoder to describe normal event patterns with small reconstruction errors.
PanGu-$α$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation
To enhance the generalization ability of PanGu-$\alpha$, we collect 1. 1TB high-quality Chinese data from a wide range of domains to pretrain the model.
Ranked #1 on
Reading Comprehension (One-Shot)
on DuReader
Cloze (multi-choices) (Few-Shot)
Cloze (multi-choices) (One-Shot)
+19
AAformer: Auto-Aligned Transformer for Person Re-Identification
In this article, we introduce an alignment scheme in transformer architecture for the first time and propose the auto-aligned transformer (AAformer) to automatically locate both the human parts and nonhuman ones at patch level.
Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark
We believe this benchmark will greatly boost related researches on natural language guided tracking.
Ranked #6 on
Visual Object Tracking
on TNL2K
(precision metric)
Learning Scalable $\ell_\infty$-constrained Near-lossless Image Compression via Joint Lossy Image and Residual Compression
To achieve scalable compression with the error bound larger than zero, we derive the probability model of the quantized residual by quantizing the learned probability model of the original residual, instead of training multiple networks.
Dynamic Attention guided Multi-Trajectory Analysis for Single Object Tracking
In this paper, we propose to introduce more dynamics by devising a dynamic attention-guided multi-trajectory tracking strategy.
Contrastive Neural Architecture Search with Neural Architecture Comparators
One of the key steps in Neural Architecture Search (NAS) is to estimate the performance of candidate architectures.
Classification of Single-View Object Point Clouds
By adapting existing ModelNet40 and ScanNet datasets to the single-view, partial setting, experiment results can verify the necessity of object pose estimation and superiority of our PAPNet to existing classifiers.
Modular Graph Attention Network for Complex Visual Relational Reasoning
Moreover, to capture the complex logic in a query, we construct a relational graph to represent the visual objects and their relationships, and propose a multi-step reasoning method to progressively understand the complex logic.
Ranked #2 on
Referring Expression Comprehension
on CLEVR-Ref+
