Search Results for author:
Found 48 papers, 32 papers with code
Enabling Deep Residual Networks for Weakly Supervised Object Detection
Weakly supervised object detection (WSOD) has attracted extensive research attention due to its great flexibility of exploiting large-scale image-level annotation for detector training.
FlashSloth: Lightning Multimodal Large Language Models via Embedded Visual Compression
Despite a big leap forward in capability, multimodal large language models (MLLMs) tend to behave like a sloth in practical use, i. e., slow response and large latency.
Dynamic-LLaVA: Efficient Multimodal Large Language Models via Dynamic Vision-language Context Sparsification
To address this problem, we proposed a dynamic vision-language context sparsification framework Dynamic-LLaVA, which dynamically reduces the redundancy of vision context in the prefill stage and decreases the memory and computation overhead of the generated language context during decoding.
T2Vid: Translating Long Text into Multi-Image is the Catalyst for Video-LLMs
In this work, our study of these approaches harvests an effective data augmentation method.
Scale Contrastive Learning with Selective Attentions for Blind Image Quality Assessment
This paper addresses two primary challenges: the significant redundancy of information across different scales, and the confusion caused by combining features from these scales, which may vary widely in quality.
Freeze-Omni: A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM
Our main contribution is that the speech input and output modalities can be easily connected to a textual LLM while keeping the LLM's parameters frozen throughout the training process.
VITA: Towards Open-Source Interactive Omni Multimodal LLM
The remarkable multimodal capabilities and interactive experience of GPT-4o underscore their necessity in practical applications, yet open-source models rarely excel in both areas.
HUWSOD: Holistic Self-training for Unified Weakly Supervised Object Detection
Most WSOD methods rely on traditional object proposals to generate candidate regions and are confronted with unstable training, which easily gets stuck in a poor local optimum.
VEGA: Learning Interleaved Image-Text Comprehension in Vision-Language Large Models
In support of this task, we further craft a new VEGA dataset, tailored for the IITC task on scientific content, and devised a subtask, Image-Text Association (ITA), to refine image-text correlation skills.
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
With Video-MME, we extensively evaluate various state-of-the-art MLLMs, including GPT-4 series and Gemini 1. 5 Pro, as well as open-source image models like InternVL-Chat-V1. 5 and video models like LLaVA-NeXT-Video.
Cantor: Inspiring Multimodal Chain-of-Thought of MLLM
This paper delves into the realm of multimodal CoT to solve intricate visual reasoning tasks with multimodal large language models(MLLMs) and their cognitive capability.
Multi-Modal Prompt Learning on Blind Image Quality Assessment
Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly.
A General and Efficient Training for Transformer via Token Expansion
The remarkable performance of Vision Transformers (ViTs) typically requires an extremely large training cost.
Feature Denoising Diffusion Model for Blind Image Quality Assessment
Blind Image Quality Assessment (BIQA) aims to evaluate image quality in line with human perception, without reference benchmarks.
Rethinking Centered Kernel Alignment in Knowledge Distillation
This paper first provides a theoretical perspective to illustrate the effectiveness of CKA, which decouples CKA to the upper bound of Maximum Mean Discrepancy~(MMD) and a constant term.
Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization
Semi-supervised learning (SSL), thanks to the significant reduction of data annotation costs, has been an active research topic for large-scale 3D scene understanding.
Solving the Catastrophic Forgetting Problem in Generalized Category Discovery
The LER optimizes the distribution of potential known class samples in unlabeled data thus ensuring the preservation of knowledge related to known categories while learning novel classes.
Weakly Supervised Open-Vocabulary Object Detection
Despite weakly supervised object detection (WSOD) being a promising step toward evading strong instance-level annotations, its capability is confined to closed-set categories within a single training dataset.
A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise
They endow Large Language Models (LLMs) with powerful capabilities in visual understanding, enabling them to tackle diverse multi-modal tasks.
SPD-DDPM: Denoising Diffusion Probabilistic Models in the Symmetric Positive Definite Space
On the other hand, the model unconditionally learns the probability distribution of the data $p(X)$ and generates samples that conform to this distribution.
Adaptive Feature Selection for No-Reference Image Quality Assessment by Mitigating Semantic Noise Sensitivity
Specifically, QFM-IQM enhances the semantic noise distinguish capabilities by matching image pairs with similar quality scores but varying semantic features as adversarial semantic noise and adaptively adjusting the upstream task's features by reducing sensitivity to adversarial noise perturbation.
Aligning and Prompting Everything All at Once for Universal Visual Perception
However, predominant paradigms, driven by casting instance-level tasks as an object-word alignment, bring heavy cross-modality interaction, which is not effective in prompting object detection and visual grounding.
Less is More: Learning Reference Knowledge Using No-Reference Image Quality Assessment
Concretely, by innovatively introducing a novel feature distillation method in IQA, we propose a new framework to learn comparative knowledge from non-aligned reference images.
Woodpecker: Hallucination Correction for Multimodal Large Language Models
Hallucination is a big shadow hanging over the rapidly evolving Multimodal Large Language Models (MLLMs), referring to the phenomenon that the generated text is inconsistent with the image content.
Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler
In this paper, we propose a novel Knowledge-driven Differential Filter Sampler~(KDFS) with Masked Filter Modeling~(MFM) framework for filter pruning, which globally prunes the redundant filters based on the prior knowledge of a pre-trained model in a differential and non-alternative optimization.
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
Multimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image.
Active Teacher for Semi-Supervised Object Detection
In this paper, we study teacher-student learning from the perspective of data initialization and propose a novel algorithm called Active Teacher(Source code are available at: \url{https://github. com/HunterJ-Lin/ActiveTeacher}) for semi-supervised object detection (SSOD).
Category-aware Allocation Transformer for Weakly Supervised Object Localization
Weakly supervised object localization (WSOL) aims to localize objects based on only image-level labels as supervision.
FoPro: Few-Shot Guided Robust Webly-Supervised Prototypical Learning
Most existing methods focus on learning noise-robust models from web images while neglecting the performance drop caused by the differences between web domain and real-world domain.
ECO-TR: Efficient Correspondences Finding Via Coarse-to-Fine Refinement
In this paper, we propose an efficient structure named Efficient Correspondence Transformer (ECO-TR) by finding correspondences in a coarse-to-fine manner, which significantly improves the efficiency of functional correspondence model.
LAB-Net: LAB Color-Space Oriented Lightweight Network for Shadow Removal
Finally, the LSA modules are further developed to fully use the prior information in non-shadow regions to cleanse the shadow regions.
Open Vocabulary Object Detection with Proposal Mining and Prediction Equalization
Open-vocabulary object detection (OVD) aims to scale up vocabulary size to detect objects of novel categories beyond the training vocabulary.
Ranked #17 on
Open Vocabulary Object Detection
on LVIS v1.0
Efficient Decoder-free Object Detection with Transformers
A natural usage of ViTs in detection is to replace the CNN-based backbone with a transformer-based backbone, which is straightforward and effective, with the price of bringing considerable computation burden for inference.
MISSU: 3D Medical Image Segmentation via Self-distilling TransUNet
U-Nets have achieved tremendous success in medical image segmentation.
PixelFolder: An Efficient Progressive Pixel Synthesis Network for Image Generation
Pixel synthesis is a promising research paradigm for image generation, which can well exploit pixel-wise prior knowledge for generation.
End-to-End Zero-Shot HOI Detection via Vision and Language Knowledge Distillation
Extensive experiments on HICO-Det dataset demonstrate that our model discovers potential interactive pairs and enables the recognition of unseen HOIs.
Human-Object Interaction Detection
Knowledge Distillation
+4
SeqTR: A Simple yet Universal Network for Visual Grounding
In this paper, we propose a simple yet universal network termed SeqTR for visual grounding tasks, e. g., phrase localization, referring expression comprehension (REC) and segmentation (RES).
Ranked #8 on
Referring Expression Segmentation
on RefCOCO testB
CF-ViT: A General Coarse-to-Fine Method for Vision Transformer
Our proposed CF-ViT is motivated by two important observations in modern ViT models: (1) The coarse-grained patch splitting can locate informative regions of an input image.
Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks
However, these methods suffer from severe performance degradation when quantizing the SR models to ultra-low precision (e. g., 2-bit and 3-bit) with the low-cost layer-wise quantizer.
HybridCR: Weakly-Supervised 3D Point Cloud Semantic Segmentation via Hybrid Contrastive Regularization
To address the huge labeling cost in large-scale point cloud semantic segmentation, we propose a novel hybrid contrastive regularization (HybridCR) framework in weakly-supervised setting, which obtains competitive performance compared to its fully-supervised counterpart.
LCTR: On Awakening the Local Continuity of Transformer for Weakly Supervised Object Localization
In this paper, we propose a novel framework built upon the transformer, termed LCTR (Local Continuity TRansformer), which targets at enhancing the local perception capability of global features among long-range feature dependencies.
Fine-grained Data Distribution Alignment for Post-Training Quantization
While post-training quantization receives popularity mostly due to its evasion in accessing the original complete training dataset, its poor performance also stems from scarce images.
Toward Joint Thing-and-Stuff Mining for Weakly Supervised Panoptic Segmentation
To date, learning weakly supervised panoptic segmentation (WSPS) with only image-level labels remains unexplored.
Parallel Detection-and-Segmentation Learning for Weakly Supervised Instance Segmentation
Weakly supervised instance segmentation (WSIS) with only image-level labels has recently drawn much attention.
UWSOD: Toward Fully-Supervised-Level Capacity Weakly Supervised Object Detection
In this paper, we propose a unified WSOD framework, termed UWSOD, to develop a high-capacity general detection model with only image-level labels, which is self-contained and does not require external modules or additional supervision.
Noise-Aware Fully Webly Supervised Object Detection
We investigate the emerging task of learning object detectors with sole image-level labels on the web without requiring any other supervision like precise annotations or additional images from well-annotated benchmark datasets.
Cyclic Guidance for Weakly Supervised Joint Detection and Segmentation
In this paper, we join weakly supervised object detection and segmentation tasks with a multi-task learning scheme for the first time, which uses their respective failure patterns to complement each other's learning.
Ranked #6 on
Image-level Supervised Instance Segmentation
on COCO 2017 val
(using extra training data)
Image-level Supervised Instance Segmentation
Multi-Task Learning
+6
