Search Results for author:
Found 62 papers, 46 papers with code
Cantor: Inspiring Multimodal Chain-of-Thought of MLLM
This paper delves into the realm of multimodal CoT to solve intricate visual reasoning tasks with multimodal large language models(MLLMs) and their cognitive capability.
HRVDA: High-Resolution Visual Document Assistant
In addition, we construct a document-oriented visual instruction tuning dataset and apply a multi-stage training strategy to enhance the model's document modeling capabilities.
A General and Efficient Training for Transformer via Token Expansion
The remarkable performance of Vision Transformers (ViTs) typically requires an extremely large training cost.
RESTORE: Towards Feature Shift for Vision-Language Prompt Learning
Prompt learning is effective for fine-tuning foundation models to improve their generalization across a variety of downstream tasks.
Enhancing Visual Document Understanding with Contrastive Learning in Large Visual-Language Models
It can represent that the contrastive learning between the visual holistic representations and the multimodal fine-grained features of document objects can assist the vision encoder in acquiring more effective visual cues, thereby enhancing the comprehension of text-rich documents in LVLMs.
Sinkhorn Distance Minimization for Knowledge Distillation
We propose the Sinkhorn Knowledge Distillation (SinKD) that exploits the Sinkhorn distance to ensure a nuanced and precise assessment of the disparity between teacher and student distributions.
FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schema
In the quest to facilitate the deep intelligence of Large Language Models (LLMs) accessible in final-end user-bot interactions, the art of prompt crafting emerges as a critical yet complex task for the average user.
A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise
They endow Large Language Models (LLMs) with powerful capabilities in visual understanding, enabling them to tackle diverse multi-modal tasks.
MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL
Our framework comprises a core decomposer agent for Text-to-SQL generation with few-shot chain-of-thought reasoning, accompanied by two auxiliary agents that utilize external tools or models to acquire smaller sub-databases and refine erroneous SQL queries.
SPD-DDPM: Denoising Diffusion Probabilistic Models in the Symmetric Positive Definite Space
On the other hand, the model unconditionally learns the probability distribution of the data $p(X)$ and generates samples that conform to this distribution.
MMICT: Boosting Multi-Modal Fine-Tuning with In-Context Examples
Although In-Context Learning (ICL) brings remarkable performance gains to Large Language Models (LLMs), the improvements remain lower than fine-tuning on downstream tasks.
Aligning and Prompting Everything All at Once for Universal Visual Perception
However, predominant paradigms, driven by casting instance-level tasks as an object-word alignment, bring heavy cross-modality interaction, which is not effective in prompting object detection and visual grounding.
Towards Robust Text Retrieval with Progressive Learning
However, existing embedding models for text retrieval usually have three non-negligible limitations.
Woodpecker: Hallucination Correction for Multimodal Large Language Models
Hallucination is a big shadow hanging over the rapidly evolving Multimodal Large Language Models (MLLMs), referring to the phenomenon that the generated text is inconsistent with the image content.
Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region Concentration
We propose a novel end-to-end document understanding model called SeRum (SElective Region Understanding Model) for extracting meaningful information from document images, including document analysis, retrieval, and office automation.
Unified and Dynamic Graph for Temporal Character Grouping in Long Videos
In this paper, we present a unified and dynamic graph (UniDG) framework for temporal character grouping.
Turning a CLIP Model into a Scene Text Spotter
Utilizing only 10% of the supervised data, FastTCM-CR50 improves performance by an average of 26. 5% and 5. 5% for text detection and spotting tasks, respectively.
MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversation
We propose MemoChat, a pipeline for refining instructions that enables large language models (LLMs) to effectively employ self-composed memos for maintaining consistent long-range open-domain conversations.
D3G: Exploring Gaussian Prior for Temporal Sentence Grounding with Glance Annotation
Under this setup, we propose a Dynamic Gaussian prior based Grounding framework with Glance annotation (D3G), which consists of a Semantic Alignment Group Contrastive Learning module (SA-GCL) and a Dynamic Gaussian prior Adjustment module (DGA).
Ranked #10 on
Temporal Sentence Grounding
on Charades-STA
Coarse-to-Fine: Learning Compact Discriminative Representation for Single-Stage Image Retrieval
Image retrieval targets to find images from a database that are visually similar to the query image.
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
Multimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image.
A Survey on Multimodal Large Language Models
Recently, Multimodal Large Language Model (MLLM) represented by GPT-4V has been a new rising research hotspot, which uses powerful Large Language Models (LLMs) as a brain to perform multimodal tasks.
Looking and Listening: Audio Guided Text Recognition
Text recognition in the wild is a long-standing problem in computer vision.
ICDAR 2023 Competition on Structured Text Extraction from Visually-Rich Document Images
It is hoped that this competition will attract many researchers in the field of CV and NLP, and bring some new thoughts to the field of Document AI.
SoftCLIP: Softer Cross-modal Alignment Makes CLIP Stronger
During the preceding biennium, vision-language pre-training has achieved noteworthy success on several downstream tasks.
Grab What You Need: Rethinking Complex Table Structure Recognition with Flexible Components Deliberation
Recently, Table Structure Recognition (TSR) task, aiming at identifying table structure into machine readable formats, has received increasing interest in the community.
Co-Salient Object Detection with Co-Representation Purification
Such irrelevant information in the co-representation interferes with its locating of co-salient objects.
Efficient Decoder-free Object Detection with Transformers
A natural usage of ViTs in detection is to replace the CNN-based backbone with a transformer-based backbone, which is straightforward and effective, with the price of bringing considerable computation burden for inference.
Training-free Transformer Architecture Search
Recently, Vision Transformer (ViT) has achieved remarkable success in several computer vision tasks.
DIFNet: Boosting Visual Information Flow for Image Captioning
Current Image captioning (IC) methods predict textual words sequentially based on the input visual information from the visual feature extractor and the partially generated sentence information.
RMNet: Equivalently Removing Residual Connection from Networks
In this paper, we aim to remedy this problem and propose to remove the residual connection in a vanilla ResNet equivalently by a reserving and merging (RM) operation on ResBlock.
Mitigating Memorization of Noisy Labels via Regularization between Representations
Designing robust loss functions is popular in learning with noisy labels while existing designs did not explicitly consider the overfitting property of deep neural networks (DNNs).
Self-supervised Models are Good Teaching Assistants for Vision Transformers
These findings motivate us to introduce an self-supervised teaching assistant (SSTA) besides the commonly used supervised teacher to improve the performance of transformers.
PR-Net: Preference Reasoning for Personalized Video Highlight Detection
Personalized video highlight detection aims to shorten a long video to interesting moments according to a user's preference, which has recently raised the community's attention.
Learning Canonical View Representation for 3D Shape Recognition with Arbitrary Views
In this way, each 3D shape with arbitrary views is represented by a fixed number of canonical view features, which are further aggregated to generate a rich and robust 3D shape representation for shape recognition.
Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer
Vision transformers (ViTs) have recently received explosive popularity, but the huge computational cost is still a severe issue.
Ranked #11 on
Efficient ViTs
on ImageNet-1K (with DeiT-T)
Discriminator-Free Generative Adversarial Attack
The Deep Neural Networks are vulnerable toadversarial exam-ples(Figure 1), making the DNNs-based systems collapsed byadding the inconspicuous perturbations to the images.
AS-MLP: An Axial Shifted MLP Architecture for Vision
Our proposed AS-MLP obtains 51. 5 mAP on the COCO validation set and 49. 5 MS mIoU on the ADE20K dataset, which is competitive compared to the transformer-based architectures.
Ranked #13 on
Semantic Segmentation
on DensePASS
Learning 3D Shape Feature for Texture-Insensitive Person Re-Identification
In this paper, rather than relying on texture based information, we propose to improve the robustness of person ReID against clothing texture by exploiting the information of a person's 3D shape.
Ranked #4 on
Person Re-Identification
on PRCC
Temporal Modulation Network for Controllable Space-Time Video Super-Resolution
To well exploit the temporal information, we propose a Locally-temporal Feature Comparison (LFC) module, along with the Bi-directional Deformable ConvLSTM, to extract short-term and long-term motion cues in videos.
DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning
Specifically, we find the final embedding obtained by the mainstream SSL methods contains the most fruitful information, and propose to distill the final embedding to maximally transmit a teacher's knowledge to a lightweight model by constraining the last embedding of the student to be consistent with that of the teacher.
On Evolving Attention Towards Domain Adaptation
However, considering that the configuration of attention, i. e., the type and the position of attention module, affects the performance significantly, it is more generalized to optimize the attention configuration automatically to be specialized for arbitrary UDA scenario.
Ranked #1 on
Partial Domain Adaptation
on Office-Home
Ask&Confirm: Active Detail Enriching for Cross-Modal Retrieval with Partial Query
However, the performance of existing methods suffers in real life since the user is likely to provide an incomplete description of an image, which often leads to results filled with false positives that fit the incomplete description.
An Empirical Study and Analysis on Open-Set Semi-Supervised Learning
Pseudo-labeling (PL) and Data Augmentation-based Consistency Training (DACT) are two approaches widely used in Semi-Supervised Learning (SSL) methods.
Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search
Secondly, a BERT with locality-constrained attention is proposed to obtain representations of descriptions at different scales.
Ranked #15 on
Text based Person Retrieval
on CUHK-PEDES
Learning To Know Where To See: A Visibility-Aware Approach for Occluded Person Re-Identification
Several mainstream methods utilize extra cues (e. g., human pose information) to distinguish human parts from obstacles to alleviate the occlusion problem.
One for More: Selecting Generalizable Samples for Generalizable ReID Model
Current training objectives of existing person Re-IDentification (ReID) models only ensure that the loss of the model decreases on selected training batch, with no regards to the performance on samples outside the batch.
Learning with Instance-Dependent Label Noise: A Sample Sieve Approach
This high-quality sample sieve allows us to treat clean examples and the corrupted ones separately in training a DNN solution, and such a separation is shown to be advantageous in the instance-dependent noise setting.
Image Classification with Label Noise
Learning with noisy labels
Pruning Filter in Filter
Through extensive experiments, we demonstrate that SWP is more effective compared to the previous FP-based methods and achieves the state-of-art pruning ratio on CIFAR-10 and ImageNet datasets without obvious accuracy drop.
Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning
Then we force the model to pull the feature of the distracting video and the feature of the original video closer, so that the model is explicitly restricted to resist the background influence, focusing more on the motion changes.
Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motion
Specifically, we construct a positive clip and a negative clip for each video.
Devil's in the Details: Aligning Visual Clues for Conditional Embedding in Person Re-Identification
Secondly, the Conditional Feature Embedding requires the overall feature of a query image to be dynamically adjusted based on the gallery image it matches, while most of the existing methods ignore the reference images.
Ranked #1 on
Person Re-Identification
on CUHK03-C
Do Not Disturb Me: Person Re-identification Under the Interference of Other Pedestrians
In the conventional person Re-ID setting, it is widely assumed that cropped person images are for each individual.
NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination
Greedy-NMS inherently raises a dilemma, where a lower NMS threshold will potentially lead to a lower recall rate and a higher threshold introduces more false positives.
Ranked #12 on
Object Detection
on CrowdHuman (full body)
Filter Grafting for Deep Neural Networks: Reason, Method, and Cultivation
To gain a universal improvement on both valid and invalid filters, we compensate grafting with distillation (\textbf{Cultivation}) to overcome the drawback of grafting .
Filter Grafting for Deep Neural Networks
To better perform the grafting process, we develop an entropy-based criterion to measure the information of filters and an adaptive weighting strategy for balancing the grafted information among networks.
Asymmetric Co-Teaching for Unsupervised Cross Domain Person Re-Identification
This procedure encourages that the selected training samples can be both clean and miscellaneous, and that the two models can promote each other iteratively.
Ranked #9 on
Unsupervised Domain Adaptation
on Market to Duke
Viewpoint-Aware Loss with Angular Regularization for Person Re-Identification
Instead of one subspace for each viewpoint, our method projects the feature from different viewpoints into a unified hypersphere and effectively models the feature distribution on both the identity-level and the viewpoint-level.
Ranked #5 on
Person Re-Identification
on Market-1501
(using extra training data)
Rethinking Temporal Fusion for Video-based Person Re-identification on Semantic and Time Aspect
Recently, the research interest of person re-identification (ReID) has gradually turned to video-based methods, which acquire a person representation by aggregating frame features of an entire video.
High-dimensional Dense Residual Convolutional Neural Network for Light Field Reconstruction
We consider the problem of high-dimensional light field reconstruction and develop a learning-based framework for spatial and angular super-resolution.
Pyramidal Person Re-IDentification via Multi-Loss Dynamic Training
Most existing Re-IDentification (Re-ID) methods are highly dependent on precise bounding boxes that enable images to be aligned with each other.
Ranked #2 on
Person Re-Identification
on CUHK03-C
Consistency Analysis for the Doubly Stochastic Dirichlet Process
This technical report proves components consistency for the Doubly Stochastic Dirichlet Process with exponential convergence of posterior probability.
