Search Results for author:
Found 30 papers, 20 papers with code
Pixel-SAIL: Single Transformer For Pixel-Grounded Understanding
We present Pixel-SAIL, a single transformer for pixel-wise MLLM tasks.
MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency
Answering questions with Chain-of-Thought (CoT) has significantly enhanced the reasoning capabilities of Large Language Models (LLMs), yet its impact on Large Multimodal Models (LMMs) still lacks a systematic assessment and in-depth investigation.
Lyra: An Efficient and Speech-Centric Framework for Omni-Cognition
As Multi-modal Large Language Models (MLLMs) evolve, expanding beyond single-domain capabilities is essential to meet the demands for more versatile and efficient AI.
Ranked #1 on
Visual Question Answering (VQA)
on EgoSchema
Beyond Pixels: Text Enhances Generalization in Real-World Image Restoration
Generalization has long been a central challenge in real-world image restoration.
LLaVA-OneVision: Easy Visual Task Transfer
We present LLaVA-OneVision, a family of open large multimodal models (LMMs) developed by consolidating our insights into data, models, and visual representations in the LLaVA-NeXT blog series.
Ranked #3 on
Zero-Shot Video Question Answer
on VNBench
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
With Video-MME, we extensively evaluate various state-of-the-art MLLMs, including GPT-4 series and Gemini 1. 5 Pro, as well as open-source image models like InternVL-Chat-V1. 5 and video models like LLaVA-NeXT-Video.
Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models
We try to narrow the gap by mining the potential of VLMs for better performance and any-to-any workflow from three aspects, i. e., high-resolution visual tokens, high-quality data, and VLM-guided generation.
RL-GPT: Integrating Reinforcement Learning and Code-as-policy
To seamlessly integrate both modalities, we introduce a two-level hierarchical framework, RL-GPT, comprising a slow agent and a fast agent.
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models
Current VLMs, while proficient in tasks like image captioning and visual question answering, face computational burdens when processing long videos due to the excessive visual tokens.
Ranked #11 on
Video Question Answering
on ActivityNet-QA
Image Captioning
Video-based Generative Performance Benchmarking
+2
LISA: Reasoning Segmentation via Large Language Model
In this work, we propose a new segmentation task -- reasoning segmentation.
Ranked #6 on
Referring Video Object Segmentation
on ReVOS
Democratizing Pathological Image Segmentation with Lay Annotators via Molecular-empowered Learning
The contribution of this paper is threefold: (1) We proposed a molecular-empowered learning scheme for multi-class cell segmentation using partial labels from lay annotators; (2) The proposed method integrated Giga-pixel level molecular-morphology cross-modality registration, molecular-informed annotation, and molecular-oriented segmentation model, so as to achieve significantly superior performance via 3 lay annotators as compared with 2 experienced pathologists; (3) A deep corrective learning (learning with imperfect label) method is proposed to further improve the segmentation performance using partially annotated noisy data.
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction
This paper aims to efficiently enable Large Language Models (LLMs) to use multimodal tools.
End-to-end 3D Tracking with Decoupled Queries
In this work, we present an end-to-end framework for camera-based 3D multi-object tracking, called DQTrack.
Diversified Dynamic Routing for Vision Tasks
Deep learning models for vision tasks are trained on large datasets under the assumption that there exists a universal representation that can be used to make predictions for all samples.
Unifying Voxel-based Representation with Transformer for 3D Object Detection
To this end, the modality-specific space is first designed to represent different inputs in the voxel feature space.
Voxel Field Fusion for 3D Object Detection
In this work, we present a conceptually simple yet effective framework for cross-modality 3D object detection, named voxel field fusion.
Focal Sparse Convolutional Networks for 3D Object Detection
In this paper, we introduce two new modules to enhance the capability of Sparse CNNs, both are based on making feature sparsity learnable with position-wise importance prediction.
Multi-Scale Aligned Distillation for Low-Resolution Detection
However, this option traditionally hurts the detection performance much.
Fully Convolutional Networks for Panoptic Segmentation with Point-based Supervision
In particular, Panoptic FCN encodes each object instance or stuff category with the proposed kernel generator and produces the prediction by convolving the high-resolution feature directly.
Scale-aware Automatic Augmentation for Object Detection
We propose Scale-aware AutoAug to learn data augmentation policies for object detection.
Rethinking Learnable Tree Filter for Generic Feature Transform
The Learnable Tree Filter presents a remarkable approach to model structure-preserving relations for semantic segmentation.
Fine-Grained Dynamic Head for Object Detection
To this end, we propose a fine-grained dynamic head to conditionally select a pixel-level combination of FPN features from different scales for each instance, which further releases the ability of multi-scale feature representation.
Fully Convolutional Networks for Panoptic Segmentation
In this paper, we present a conceptually simple, strong, and efficient framework for panoptic segmentation, called Panoptic FCN.
Ranked #1 on
Panoptic Segmentation
on COCO minival
(SQ metric)
Dynamic Scale Training for Object Detection
We propose a Dynamic Scale Training paradigm (abbreviated as DST) to mitigate scale variation challenge in object detection.
Learning Dynamic Routing for Semantic Segmentation
To demonstrate the superiority of the dynamic property, we compare with several static architectures, which can be modeled as special cases in the routing space.
Learnable Tree Filter for Structure-preserving Feature Transform
To this end, tree filtering modules are embedded to formulate a unified framework for semantic segmentation.
FastPose: Towards Real-time Pose Estimation and Tracking via Scale-normalized Multi-task Networks
Given the results of MTN, we adopt an occlusion-aware Re-ID feature strategy in the pose tracking module, where pose information is utilized to infer the occlusion state to make better use of Re-ID feature.
State-aware Re-identification Feature for Multi-target Multi-camera Tracking
Multi-target Multi-camera Tracking (MTMCT) aims to extract the trajectories from videos captured by a set of cameras.
Identity-Enhanced Network for Facial Expression Recognition
Facial expression recognition is a challenging task, arguably because of large intra-class variations and high inter-class similarities.
Facial Expression Recognition
Facial Expression Recognition (FER)
+1
Attention-guided Unified Network for Panoptic Segmentation
This paper studies panoptic segmentation, a recently proposed task which segments foreground (FG) objects at the instance level as well as background (BG) contents at the semantic level.
Ranked #24 on
Panoptic Segmentation
on COCO test-dev
