Search Results for author:
Found 42 papers, 33 papers with code
Instance-Aware Embedding for Point Cloud Instance Segmentation
However, these methods are unaware of the instance context and fail to realize the boundary and geometric information of an instance, which are critical to separate adjacent objects.
Beyond Literal Descriptions: Understanding and Locating Open-World Objects Aligned with Human Intentions
Previous datasets and methods for classic VG task mainly rely on the prior assumption that the given expression must literally refer to the target object, which greatly impedes the practical deployment of agents in real-world scenarios.
EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters
Scaling up contrastive language-image pretraining (CLIP) is critical for empowering both vision and multimodal models.
Ranked #1 on
Zero-Shot Transfer Image Classification
on SUN
Image Classification
Zero-Shot Transfer Image Classification
Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
The results demonstrate that Vim is capable of overcoming the computation & memory constraints on performing Transformer-style understanding for high-resolution images and it has great potential to be the next-generation backbone for vision foundation models.
Generative Multimodal Models are In-Context Learners
The human ability to easily solve multimodal tasks in context (i. e., with only a few demonstrations or simple instructions), is what current multimodal systems have largely struggled to imitate.
Ranked #19 on
Visual Question Answering
on MM-Vet
Tokenize Anything via Prompting
The semantic token is responsible for learning the semantic priors in a predefined concept space.
Unveiling Parts Beyond Objects:Towards Finer-Granularity Referring Expression Segmentation
To foster future research into fine-grained visual grounding, our benchmark RefCOCOm, the MRES-32M dataset and model UniRES will be publicly available at https://github. com/Rubics-Xuan/MRES
GeoDream: Disentangling 2D and Geometric Priors for High-Fidelity and Consistent 3D Generation
We justify that the refined 3D geometric priors aid in the 3D-aware capability of 2D diffusion priors, which in turn provides superior guidance for the refinement of 3D geometric priors.
CapsFusion: Rethinking Image-Text Data at Scale
To provide higher-quality and more scalable multimodal pretraining data, we propose CapsFusion, an advanced framework that leverages large language models to consolidate and refine information from both web-based image-text pairs and synthetic captions.
JudgeLM: Fine-tuned Large Language Models are Scalable Judges
To address this problem, we propose to fine-tune LLMs as scalable judges (JudgeLM) to evaluate LLMs efficiently and effectively in open-ended benchmarks.
3D-GPT: Procedural 3D Modeling with Large Language Models
Our work highlights the potential of LLMs in 3D modeling, offering a basic framework for future advancements in scene generation and animation.
Uni3D: Exploring Unified 3D Representation at Scale
Scaling up representations for images or text has been extensively investigated in the past few years and has led to revolutions in learning vision and language.
Ranked #1 on
Zero-shot 3D classification
on Objaverse LVIS
(using extra training data)
Generative Pretraining in Multimodality
We present Emu, a Transformer-based multimodal foundation model, which can seamlessly generate images and texts in multimodal context.
Ranked #1 on
Visual Question Answering
on VQA v2
Fine-Grained Visual Prompting
Previous works have suggested that incorporating visual prompts, such as colorful boxes or circles, can improve the ability of models to recognize objects of interest.
Towards Better Entity Linking with Multi-View Enhanced Distillation
Aiming at learning entity representations that can match divergent mentions, this paper proposes a Multi-View Enhanced Distillation (MVD) framework, which can effectively transfer knowledge of multiple fine-grained and mention-relevant parts within entities from cross-encoders to dual-encoders.
Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching
In this work, we present Matcher, a novel perception paradigm that utilizes off-the-shelf vision foundation models to address various perception tasks.
SegGPT: Segmenting Everything In Context
We unify various segmentation tasks into a generalist in-context learning framework that accommodates different kinds of segmentation data by transforming them into the same format of images.
Ranked #1 on
Few-Shot Semantic Segmentation
on PASCAL-5i (5-Shot)
(using extra training data)
Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Models
Our vid2vid-zero leverages off-the-shelf image diffusion models, and doesn't require training on any video.
EVA-CLIP: Improved Training Techniques for CLIP at Scale
Our approach incorporates new techniques for representation learning, optimization, and augmentation, enabling EVA-CLIP to achieve superior performance compared to previous CLIP models with the same number of parameters but significantly smaller training costs.
Ranked #4 on
Zero-Shot Transfer Image Classification
on Food-101
EVA-02: A Visual Representation for Neon Genesis
We launch EVA-02, a next-generation Transformer-based visual representation pre-trained to reconstruct strong and robust language-aligned vision features via masked image modeling.
Affective Image Filter: Reflecting Emotions from Text to Images
In this work, we propose Affective Image Filter (AIF), a novel model that is able to understand the visually-abstract emotions from the text and reflect them to visually-concrete images with appropriate colors and textures.
SegGPT: Towards Segmenting Everything in Context
We unify various segmentation tasks into a generalist in-context learning framework that accommodates different kinds of segmentation data by transforming them into the same format of images.
Images Speak in Images: A Generalist Painter for In-Context Visual Learning
In this work, we present Painter, a generalist model which addresses these obstacles with an "image"-centric solution, that is, to redefine the output of core vision tasks as images, and specify task prompts as also images.
Ranked #6 on
Personalized Segmentation
on PerSeg
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale
We launch EVA, a vision-centric foundation model to explore the limits of visual representation at scale using only publicly accessible data.
Point-Teaching: Weakly Semi-Supervised Object Detection with Point Annotations
Point annotations are considerably more time-efficient than bounding box annotations.
FreeSOLO: Learning to Segment Objects without Annotations
FreeSOLO further demonstrates superiority as a strong pre-training method, outperforming state-of-the-art self-supervised pre-training methods by +9. 8% AP when fine-tuning instance segmentation with only 5% COCO masks.
Poseur: Direct Human Pose Regression with Transformers
We propose a direct, regression-based approach to 2D human pose estimation from single images.
Ranked #2 on
Keypoint Detection
on MS COCO
SOLO: A Simple Framework for Instance Segmentation
Besides instance segmentation, our method yields state-of-the-art results in object detection (from our mask byproduct) and panoptic segmentation.
FCPose: Fully Convolutional Multi-Person Pose Estimation with Dynamic Instance-Aware Convolutions
We propose a fully convolutional multi-person pose estimation framework using dynamic instance-aware convolutions, termed FCPose.
TFPose: Direct Human Pose Estimation with Transformers
We propose a human pose estimation framework that solves the task in the regression-based fashion.
Ranked #26 on
Pose Estimation
on MPII Human Pose
(using extra training data)
Conditional Positional Encodings for Vision Transformers
Built on PEG, we present Conditional Position encoding Vision Transformer (CPVT).
Diverse Knowledge Distillation for End-to-End Person Search
Person search aims to localize and identify a specific person from a gallery of images.
BoxInst: High-Performance Instance Segmentation with Box Annotations
We present a high-performance method that can achieve mask-level instance segmentation with only bounding-box annotations for training.
End-to-End Video Instance Segmentation with Transformers
Here, we propose a new video instance segmentation framework built upon Transformers, termed VisTR, which views the VIS task as a direct end-to-end parallel sequence decoding/prediction problem.
Ranked #33 on
Video Instance Segmentation
on YouTube-VIS validation
Dense Contrastive Learning for Self-Supervised Visual Pre-Training
Compared to the baseline method MoCo-v2, our method introduces negligible computation overhead (only <1% slower), but demonstrates consistently superior performance when transferring to downstream dense prediction tasks including object detection, semantic segmentation and instance segmentation; and outperforms the state-of-the-art methods by a large margin.
SOLOv2: Dynamic and Fast Instance Segmentation
Importantly, we take one step further by dynamically learning the mask head of the object segmenter such that the mask head is conditioned on the location.
Ranked #10 on
Real-time Instance Segmentation
on MSCOCO
DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data
Compared with previous learning objectives, i. e., learning metric depth or relative depth, we propose to learn the affine-invariant depth using our diverse dataset to ensure both generalization and high-quality geometric shapes of scenes.
SOLO: Segmenting Objects by Locations
We present a new, embarrassingly simple approach to instance segmentation in images.
Ranked #67 on
Instance Segmentation
on COCO test-dev
Task-Aware Monocular Depth Estimation for 3D Object Detection
In this paper, we first analyse the data distributions and interaction of foreground and background, then propose the foreground-background separated monocular depth estimation (ForeSeE) method, to estimate the foreground depth and background depth using separate optimization objectives and depth decoders.
Associatively Segmenting Instances and Semantics in Point Clouds
A 3D point cloud describes the real scene precisely and intuitively. To date how to segment diversified elements in such an informative 3D scene is rarely discussed.
Ranked #15 on
3D Instance Segmentation
on S3DIS
(mRec metric)
Repulsion Loss: Detecting Pedestrians in a Crowd
In this paper, we first explore how a state-of-the-art pedestrian detector is harmed by crowd occlusion via experimentation, providing insights into the crowd occlusion problem.
Ranked #9 on
Pedestrian Detection
on Caltech
(using extra training data)
Adversarial Generation of Training Examples: Applications to Moving Vehicle License Plate Recognition
Our experimental results on a few data sets demonstrate the effectiveness of using GAN images: an improvement of 7. 5% over a strong baseline with moderate-sized real data being available.
