Search Results for author:
Found 43 papers, 29 papers with code
Seed1.5-VL Technical Report
We present Seed1. 5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning.
Ranked #1 on
Video Question Answering
on TVBench
Pixel-SAIL: Single Transformer For Pixel-Grounded Understanding
We present Pixel-SAIL, a single transformer for pixel-wise MLLM tasks.
The Scalability of Simplicity: Empirical Analysis of Vision-Language Learning with a Single Transformer
This paper introduces SAIL, a single transformer unified multimodal large language model (MLLM) that integrates raw pixel encoding and language decoding within a singular architecture.
GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for Autoregressive Image Generation
In autoregressive (AR) image generation, visual tokenizers compress images into compact discrete latent tokens, enabling efficient training of downstream autoregressive models for visual generation via next-token prediction.
GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generation
The recent breakthroughs in OpenAI's GPT4o model have demonstrated surprisingly good capabilities in image generation and editing, resulting in significant excitement in the community.
4th PVUW MeViS 3rd Place Report: Sa2VA
Referring video object segmentation (RVOS) is a challenging task that requires the model to segment the object in a video given the language description.
Scene4U: Hierarchical Layered 3D Scene Reconstruction from Single Panoramic Image for Your Immerse Exploration
Additionally, to demonstrate the robustness of Scene4U and allow users to experience immersive scenes from various landmarks, we build WorldVista3D dataset for 3D scene reconstruction, which contains panoramic images of globally renowned sites.
Video Depth Anything: Consistent Depth Estimation for Super-Long Videos
The model is trained on a joint dataset of video depth and unlabeled images, similar to Depth Anything V2.
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos
This work presents Sa2VA, the first unified model for dense grounded understanding of both images and videos.
Classification Done Right for Vision-Language Pre-Training
Due to the absence of the text encoding as contrastive target, SuperClass does not require a text encoder and does not need to maintain a large batch size as CLIP does.
LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models
With the rapid development of AI-generated content, the future internet may be inundated with synthetic data, making the discrimination of authentic and credible multimodal data increasingly challenging.
CrossViewDiff: A Cross-View Diffusion Model for Satellite-to-Street View Synthesis
In this work, we propose CrossViewDiff, a cross-view diffusion model for satellite-to-street view synthesis.
Lightweight Model Pre-training via Language Guided Knowledge Distillation
In this paper, for the first time, we introduce language guidance to the distillation process and propose a new method named Language-Guided Distillation (LGD) system, which uses category names of the target downstream task to help refine the knowledge transferred between the teacher and student.
DiG: Scalable and Efficient Diffusion Models with Gated Linear Attention
In this paper, we aim to incorporate the sub-quadratic modeling capability of Gated Linear Attention (GLA) into the 2D diffusion backbone.
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
To this end, we scale up the dataset by designing a data engine to collect and automatically annotate large-scale unlabeled data (~62M), which significantly enlarges the data coverage and thus is able to reduce the generalization error.
Ranked #4 on
Monocular Depth Estimation
on ETH3D
Harnessing Diffusion Models for Visual Perception with Meta Prompts
Our key insight is to introduce learnable embeddings (meta prompts) to the pre-trained diffusion models to extract proper features for perception.
Ranked #2 on
Semantic Segmentation
on Cityscapes test
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs
Our experiments show that BuboGPT achieves impressive multi-modality understanding and visual grounding abilities during the interaction with human.
Disentangled Pre-training for Image Matting
The pre-training task is designed in a similar manner as image matting, where random trimap and alpha matte are generated to achieve an image disentanglement objective.
SeaFormer++: Squeeze-enhanced Axial Transformer for Mobile Visual Recognition
Since the introduction of Vision Transformers, the landscape of many computer vision tasks (e. g., semantic segmentation), which has been overwhelmingly dominated by CNNs, recently has significantly revolutionized.
Executing your Commands via Motion Diffusion in Latent Space
We study a challenging task, conditional human motion generation, which produces plausible human motion sequences according to various conditional inputs, such as action classes or textual descriptors.
Ranked #2 on
Motion Synthesis
on HumanAct12
Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report
Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks.
Coordinates Are NOT Lonely -- Codebook Prior Helps Implicit Neural 3D Representations
Implicit neural 3D representation has achieved impressive results in surface or scene reconstruction and novel view synthesis, which typically uses the coordinate-based multi-layer perceptrons (MLPs) to learn a continuous scene representation.
TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation
Although vision transformers (ViTs) have achieved great success in computer vision, the heavy computational cost hampers their applications to dense prediction tasks such as semantic segmentation on mobile devices.
SeMask: Semantically Masked Transformers for Semantic Segmentation
To achieve this, we propose SeMask, a simple and effective framework that incorporates semantic information into the encoder with the help of a semantic attention operation.
Ranked #11 on
Semantic Segmentation
on Cityscapes val
Shuffle Transformer with Feature Alignment for Video Face Parsing
This is a short technical report introducing the solution of the Team TCParser for Short-video Face Parsing Track of The 3rd Person in Context (PIC) Workshop and Challenge at CVPR 2021.
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
In this work, we revisit the spatial shuffle as an efficient way to build connections among windows.
Ranked #49 on
Semantic Segmentation
on ADE20K val
Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report
While many solutions have been proposed for this task, they are usually very computationally expensive and thus are not applicable for on-device inference.
Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation
Despite their success for semantic segmentation, convolutional neural networks are ill-equipped for incremental learning, \ie, adapting the original segmentation model as new classes are available but the initial training data is not retained.
Human De-occlusion: Invisible Perception and Recovery for Humans
Besides, an Amodal Human Perception dataset (AHP) is collected to settle the task of human de-occlusion.
High-Resolution Deep Image Matting
Image matting is a key technique for image and video editing and composition.
Deep Learning-Based Automated Image Segmentation for Concrete Petrographic Analysis
The standard petrography test method for measuring air voids in concrete (ASTM C457) requires a meticulous and long examination of sample phase composition under a stereomicroscope.
The 1st Agriculture-Vision Challenge: Methods and Results
The first Agriculture-Vision Challenge aims to encourage research in developing novel and effective algorithms for agricultural pattern recognition from aerial images, especially for the semantic segmentation task associated with our challenge dataset.
AlignSeg: Feature-Aligned Segmentation Networks
Aggregating features in terms of different convolutional blocks or contextual embeddings has been proven to be an effective way to strengthen feature representations for semantic segmentation.
Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis
To encourage research in computer vision for agriculture, we present Agriculture-Vision: a large-scale aerial farmland image dataset for semantic segmentation of agricultural patterns.
SPGNet: Semantic Prediction Guidance for Scene Parsing
The multi-scale context module refers to the operations to aggregate feature responses from a large spatial extent, while the single-stage encoder-decoder structure encodes the high-level semantic information in the encoder path and recovers the boundary information in the decoder path.
Proposal, Tracking and Segmentation (PTS): A Cascaded Network for Video Object Segmentation
Video object segmentation (VOS) aims at pixel-level object tracking given only the annotations in the first frame.
Ranked #1 on
Visual Object Tracking
on YouTube-VOS 2018
(Jaccard (Seen) metric)
CCNet: Criss-Cross Attention for Semantic Segmentation
Compared with the non-local block, the proposed recurrent criss-cross attention module requires 11x less GPU memory usage.
Ranked #7 on
Semantic Segmentation
on FoodSeg103
(using extra training data)
Devil in the Details: Towards Accurate Single and Multiple Human Parsing
Human parsing has received considerable interest due to its wide application potentials.
Ranked #2 on
Person Re-Identification
on Market-1501-C
Weakly-Supervised Semantic Segmentation Network With Deep Seeded Region Growing
Inspired by the traditional image segmentation methods of seeded region growing, we propose to train a semantic segmentation network starting from the discriminative regions and progressively increase the pixel-level supervision using by seeded region growing.
Ranked #39 on
Weakly-Supervised Semantic Segmentation
on COCO 2014 val
(using extra training data)
Point Linking Network for Object Detection
The deep ConvNets based object detectors mainly focus on regressing the coordinates of bounding box, e. g., Faster-R-CNN, YOLO and SSD.
Deep Patch Learning for Weakly Supervised Object Classification and Discovery
Patch-level image representation is very important for object classification and detection, since it is robust to spatial transformation, scale variation, and cluttered background.
