Search Results for author:
Found 24 papers, 18 papers with code
TAPTRv3: Spatial and Temporal Context Foster Robust Tracking of Any Point in Long Video
In this paper, we present TAPTRv3, which is built upon TAPTRv2 to improve its point tracking robustness in long videos.
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding
From the data perspective, we build a fully automated data engine and construct the Rexverse-2M dataset which possesses multiple granularities to support the joint training of perception and understanding.
DINO-X: A Unified Vision Model for Open-World Object Detection and Understanding
DINO-X employs the same Transformer-based encoder-decoder architecture as Grounding DINO 1. 5 to pursue an object-level representation for open-world object understanding.
TAPTRv2: Attention-based Position Update Improves Tracking Any Point
In this paper, we present TAPTRv2, a Transformer-based approach built upon TAPTR for solving the Tracking Any Point (TAP) task.
Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detection
Empirical results demonstrate the effectiveness of Grounding DINO 1. 5, with the Grounding DINO 1. 5 Pro model attaining a 54. 3 AP on the COCO detection benchmark and a 55. 7 AP on the LVIS-minival zero-shot transfer benchmark, setting new records for open-set object detection.
Ranked #1 on
Zero-Shot Object Detection
on MSCOCO
(using extra training data)
T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergy
Recognizing the complementary strengths and weaknesses of both text and visual prompts, we introduce T-Rex2 that synergizes both prompts within a single model through contrastive learning.
TAPTR: Tracking Any Point with Transformers as Detection
Based on the observation that point tracking bears a great resemblance to object detection and tracking, we borrow designs from DETR-like algorithms to address the task of TAP.
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
We introduce Grounded SAM, which uses Grounding DINO as an open-set object detector to combine with the segment anything model (SAM).
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
To address this issue, we have created GVC data that allows for the combination of grounding and chat capabilities.
T-Rex: Counting by Visual Prompting
Guided by the visual feedback from T-Rex, users can also interactively refine the counting results by prompting on missing or falsely-detected objects.
Visual In-Context Prompting
In-context prompting in large language models (LLMs) has become a prevalent approach to improve zero-shot capabilities, but this idea is less explored in the vision domain.
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
LLaVA-Plus is a general-purpose multimodal assistant that expands the capabilities of large multimodal models.
Ranked #1 on
LMM real-life tasks
on Leaderboard
ChatCounselor: A Large Language Models for Mental Health Support
This paper presents ChatCounselor, a large language model (LLM) solution designed to provide mental health support.
DFA3D: 3D Deformable Attention For 2D-to-3D Feature Lifting
Existing feature lifting approaches, such as Lift-Splat-based and 2D attention-based, either use estimated depth to get pseudo LiDAR features and then splat them to a 3D space, which is a one-pass operation without feature refinement, or ignore depth and lift features by 2D attention mechanisms, which achieve finer semantics while suffering from a depth ambiguity problem.
Systematic Investigation of Sparse Perturbed Sharpness-Aware Minimization Optimizer
Sharpness-Aware Minimization (SAM) is a popular solution that smooths the loss landscape by minimizing the maximized change of training loss when adding a perturbation to the weight.
detrex: Benchmarking Detection Transformers
To address this issue, we develop a unified, highly modular, and lightweight codebase called detrex, which supports a majority of the mainstream DETR-based instance recognition algorithms, covering various fundamental tasks, including object detection, segmentation, and pose estimation.
Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models
To validate MMA, we apply it to a recent LLM called LLaMA and term this formed large vision-language instructed model as LaVIN.
A Strong and Reproducible Object Detector with Only Public Datasets
This work presents Focal-Stable-DINO, a strong and reproducible object detection model which achieves 64. 6 AP on COCO val2017 and 64. 8 AP on COCO test-dev using only 700M parameters without any test time augmentation.
Ranked #5 on
Object Detection
on COCO test-dev
Detection Transformer with Stable Matching
We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR.
You Only Segment Once: Towards Real-Time Panoptic Segmentation
To reduce the computational overhead, we design a feature pyramid aggregator for the feature map extraction, and a separable dynamic decoder for the panoptic kernel generation.
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion.
Ranked #2 on
Zero-Shot Object Detection
on ODinW
(using extra training data)
Exploring Vision Transformers as Diffusion Learners
We further provide a hypothesis on the implication of disentangling the generative backbone as an encoder-decoder structure and show proof-of-concept experiments verifying the effectiveness of a stronger encoder for generative tasks with ASymmetriC ENcoder Decoder (ASCEND).
Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach
One of the popular solutions is Sharpness-Aware Minimization (SAM), which smooths the loss landscape via minimizing the maximized change of training loss when adding a perturbation to the weight.
TRAR: Routing the Attention Spans in Transformer for Visual Question Answering
Due to the superior ability of global dependency modeling, Transformer and its variants have become the primary choice of many vision-and-language tasks.
