Search Results for author:
Found 89 papers, 50 papers with code
PVUW 2025 Challenge Report: Advances in Pixel-level Understanding of Complex Videos in the Wild
This report provides a comprehensive overview of the 4th Pixel-level Video Understanding in the Wild (PVUW) Challenge, held in conjunction with CVPR 2025.
Learning to Animate Images from A Few Videos to Portray Delicate Human Actions
This approach minimizes overfitting to visual appearances in the limited training data and enhances the generalization of learned motion patterns.
Liquid: Language Models are Scalable Multi-modal Generators
We present Liquid, an auto-regressive generation paradigm that seamlessly integrates visual comprehension and generation by tokenizing images into discrete codes and learning these code embeddings alongside text tokens within a shared feature space for both vision and language.
PartGLEE: A Foundation Model for Recognizing and Parsing Any Objects
We present PartGLEE, a part-level foundation model for locating and identifying both objects and parts in images.
PVUW 2024 Challenge on Complex Video Understanding: Methods and Results
Moreover, we provide a new motion expression guided video segmentation dataset MeViS to study the natural language-guided video understanding in complex environments.
DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data
Unlike recent 3D generative models that rely on clean and well-aligned 3D data, limiting them to single or few-class generation, our model is directly trained on extensive noisy and unaligned `in-the-wild' 3D assets, mitigating the key challenge (i. e., data scarcity) in large-scale 3D generation.
Debiasing Text-to-Image Diffusion Models
Despite its simplicity, we show that IDA shows efficiency and fast convergence in resolving the social bias in TTI diffusion models.
Progress and Prospects in 3D Generative AI: A Technical Overview including 3D human
Since the year 2023 an abundant amount of research papers has emerged in the domain of 3D generation.
General Object Foundation Model for Images and Videos at Scale
We present GLEE in this work, an object-level foundation model for locating and identifying objects in images and videos.
Ranked #1 on
Referring Video Object Segmentation
on Refer-YouTube-VOS
(using extra training data)
Long-tail Video Object Segmentation
Multi-Object Tracking
+8
Learning to Holistically Detect Bridges from Large-Size VHR Remote Sensing Imagery
Therefore, to ensure the visibility and integrity of bridges, it is essential to perform holistic bridge detection in large-size very-high-resolution (VHR) RSIs.
Dataset Condensation via Generative Model
Dataset condensation aims to condense a large dataset with a lot of training samples into a small set.
Free-ATM: Exploring Unsupervised Learning on Diffusion-Generated Images with Free Attention Masks
Despite the rapid advancement of unsupervised learning in visual representation, it requires training on large-scale datasets that demand costly data collection, and pose additional challenges due to concerns regarding data privacy.
Lowis3D: Language-Driven Open-World Instance-Level 3D Scene Understanding
To address this challenge, we propose to harness pre-trained vision-language (VL) foundation models that encode extensive knowledge from image-text pairs to generate captions for multi-view images of 3D scenes.
Ranked #3 on
3D Open-Vocabulary Instance Segmentation
on S3DIS
DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing
In this work, we extend this editing framework to diffusion models and propose a novel approach DragDiffusion.
Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search
(2) We find regions in the latent space that lead to distorted images independent of the text prompt, suggesting that parts of the latent space are not well-structured.
InstMove: Instance Motion for Object-centric Video Segmentation
A common solution is to use optical flow to provide motion information, but essentially it only considers pixel-level motion, which still relies on appearance similarity and hence is often inaccurate under occlusion and fast movement.
MOSE: A New Dataset for Video Object Segmentation in Complex Scenes
However, since the target objects in these existing datasets are usually relatively salient, dominant, and isolated, VOS under complex scenes has rarely been studied.
PV3D: A 3D Generative Model for Portrait Video Generation
While some prior works have applied such image GANs to unconditional 2D portrait video generation and static 3D portrait synthesis, there are few works successfully extending GANs for generating 3D-aware portrait videos.
LUMix: Improving Mixup by Better Modelling Label Uncertainty
LUMix is simple as it can be implemented in just a few lines of code and can be universally applied to any deep networks \eg CNNs and Vision Transformers, with minimal computational cost.
PLA: Language-Driven Open-Vocabulary 3D Scene Understanding
Open-vocabulary scene understanding aims to localize and recognize unseen categories beyond the annotated label space.
Ranked #2 on
3D Open-Vocabulary Instance Segmentation
on S3DIS
3D Open-Vocabulary Instance Segmentation
Contrastive Learning
+4
The Runner-up Solution for YouTube-VIS Long Video Challenge 2022
This technical report describes our 2nd-place solution for the ECCV 2022 YouTube-VIS Long Video Challenge.
Holistically-Attracted Wireframe Parsing: From Supervised to Self-Supervised Learning
This article presents Holistically-Attracted Wireframe Parsing (HAWP), a method for geometric analysis of 2D images containing wireframes formed by line segments and junctions.
Is synthetic data from generative models ready for image recognition?
Recent text-to-image generation models have shown promising results in generating high-fidelity photo-realistic images.
Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning
To remedy this problem caused by the data heterogeneity, we propose {\sc FedDecorr}, a novel method that can effectively mitigate dimensional collapse in federated learning.
1st Place Solution to ECCV 2022 Challenge on Out of Vocabulary Scene Text Understanding: End-to-End Recognition of Out of Vocabulary Words
Our oCLIP-based model achieves 28. 59\% in h-mean which ranks 1st in end-to-end OOV word recognition track of OOV Challenge in ECCV2022 TiE Workshop.
Runner-Up Solution to ECCV 2022 Challenge on Out of Vocabulary Scene Text Understanding: Cropped Word Recognition
This report presents our 2nd place solution to ECCV 2022 challenge on Out-of-Vocabulary Scene Text Understanding (OOV-ST) : Cropped Word Recognition.
Explicit Occlusion Reasoning for Multi-person 3D Human Pose Estimation
Inspired by the remarkable ability of humans to infer occluded joints from visible cues, we develop a method to explicitly model this process that significantly improves bottom-up multi-person human pose estimation with or without occlusions.
Ranked #10 on
3D Multi-Person Pose Estimation (absolute)
on MuPoTS-3D
3D Human Pose Estimation
3D Multi-Person Pose Estimation (absolute)
+2
Contextual Text Block Detection towards Scene Text Understanding
We formulate the new setup by a dual detection task which first detects integral text units and then groups them into a CTB.
In Defense of Online Models for Video Instance Segmentation
In recent years, video instance segmentation (VIS) has been largely advanced by offline models, while online models gradually attracted less attention possibly due to their inferior performance.
Ranked #14 on
Video Instance Segmentation
on YouTube-VIS 2021
CenterNet++ for Object Detection
Our approach, named CenterNet, detects each object as a triplet keypoints (top-left and bottom-right corners and the center keypoint).
Ranked #37 on
Object Detection
on COCO test-dev
An Empirical Study of End-to-End Temporal Action Detection
Rather than end-to-end learning, most existing methods adopt a head-only learning paradigm, where the video encoder is pre-trained for action classification, and only the detection head upon the encoder is optimized for TAD.
Ranked #19 on
Temporal Action Localization
on THUMOS’14
Fourier Document Restoration for Robust Document Dewarping and Recognition
State-of-the-art document dewarping techniques learn to predict 3-dimensional information of documents which are prone to errors while dealing with documents with irregular distortions or large variations in depth.
Knowledge Distillation as Efficient Pre-training: Faster Convergence, Higher Data-efficiency, and Better Transferability
Large-scale pre-training has been proven to be crucial for various computer vision tasks.
Language Matters: A Weakly Supervised Vision-Language Pre-training Approach for Scene Text Detection and Spotting
Our network consists of an image encoder and a character-aware text encoder that extract visual and textual features, respectively, as well as a visual-textual decoder that models the interaction among textual and visual features for learning effective scene text representations.
Optical Character Recognition
Optical Character Recognition (OCR)
+2
YouMVOS: An Actor-Centric Multi-Shot Video Object Segmentation Dataset
Many video understanding tasks require analyzing multi-shot videos, but existing datasets for video object segmentation (VOS) only consider single-shot videos.
SeqFormer: Sequential Transformer for Video Instance Segmentation
Nevertheless, we observe that a stand-alone instance query suffices for capturing a time sequence of instances in a video, but attention mechanisms shall be done with each frame independently.
Ranked #2 on
Video Instance Segmentation
on HQ-YTVIS
Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning
Specifically, we experimentally show that directly encouraging CIL Learner at the initial phase to output similar representations as the model jointly trained on all classes can greatly boost the CIL performance.
DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion
A typical pipeline for multi-object tracking (MOT) is to use a detector for object localization, and following re-identification (re-ID) for object association.
TransMix: Attend to Mix for Vision Transformers
The confidence of the label will be larger if the corresponding input image is weighted higher by the attention map.
Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence
Artificial intelligence (AI) provides a promising substitution for streamlining COVID-19 diagnoses.
Occluded Video Instance Segmentation: Dataset and ICCV 2021 Challenge
To promote the development of occlusion understanding, we collect a large-scale dataset called OVIS for video instance segmentation in the occluded scenario.
Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation
Motivated by this, we propose a video instance segmentation method that alleviates the problem due to missing detections.
Ranked #31 on
Video Instance Segmentation
on YouTube-VIS validation
Contextual Text Detection
This paper presents Contextual Text Detection, a new setup that detects contextual text blocks for better understanding of texts in scenes.
LATFormer: Locality-Aware Point-View Fusion Transformer for 3D Shape Recognition
To investigate this, we propose a novel Locality-Aware Point-View Fusion Transformer (LATFormer) for 3D shape retrieval and classification.
PlaneTR: Structure-Guided Transformers for 3D Plane Recovery
This paper presents a neural network built upon Transformers, namely PlaneTR, to simultaneously detect and reconstruct planes from a single image.
Visual Parser: Representing Part-whole Hierarchies with Transformers
To model the representations of the two levels, we first encode the information from the whole into part vectors through an attention mechanism, then decode the global information within the part vectors back into the whole representation.
Ranked #333 on
Image Classification
on ImageNet
End-to-end Temporal Action Detection with Transformer
Temporal action detection (TAD) aims to determine the semantic label and the temporal interval of every action instance in an untrimmed video.
Ranked #10 on
Temporal Action Localization
on HACS
I2C2W: Image-to-Character-to-Word Transformers for Accurate Scene Text Recognition
The first task focuses on image-to-character (I2C) mapping which detects a set of character candidates from images based on different alignments of visual features in an non-sequential way.
Location-Sensitive Visual Recognition with Cross-IOU Loss
Object detection, instance segmentation, and pose estimation are popular visual recognition tasks which require localizing the object by internal or boundary landmarks.
Ranked #58 on
Object Detection
on COCO test-dev
Anchor-Free Person Search
Person search aims to simultaneously localize and identify a query person from realistic, uncropped images, which can be regarded as the unified task of pedestrian detection and person re-identification (re-id).
Ranked #10 on
Person Search
on CUHK-SYSU
SwiftNet: Real-time Video Object Segmentation
In this work we present SwiftNet for real-time semisupervised video object segmentation (one-shot VOS), which reports 77. 8% J &F and 70 FPS on DAVIS 2017 validation dataset, leading all present solutions in overall accuracy and speed performance.
Occluded Video Instance Segmentation: A Benchmark
On the OVIS dataset, the highest AP achieved by state-of-the-art algorithms is only 16. 3, which reveals that we are still at a nascent stage for understanding objects, instances, and videos in a real-world scenario.
Ranked #33 on
Video Instance Segmentation
on YouTube-VIS validation
Multi-shot Temporal Event Localization: a Benchmark
Current developments in temporal event or action localization usually target actions captured by a single camera.
Ranked #2 on
Temporal Action Localization
on MUSES
Importance-Aware Semantic Segmentation in Self-Driving with Discrete Wasserstein Training
However, the cross entropy loss can not take the different importance of each class in an self-driving system into account.
Dual Attention GANs for Semantic Image Synthesis
We also propose two novel modules, i. e., position-wise Spatial Attention Module (SAM) and scale-wise Channel Attention Module (CAM), to capture semantic structure attention in spatial and channel dimensions, respectively.
Reinforced Wasserstein Training for Severity-Aware Semantic Segmentation in Autonomous Driving
The ground metric of Wasserstein distance can be pre-defined following the experience on a specific task.
Bipartite Graph Reasoning GANs for Person Image Generation
We present a novel Bipartite Graph Reasoning GAN (BiGraphGAN) for the challenging person image generation task.
Ranked #1 on
Pose Transfer
on Market-1501
(PCKh metric)
Corner Proposal Network for Anchor-free, Two-stage Object Detection
On the MS-COCO dataset, CPN achieves an AP of 49. 2% which is competitive among state-of-the-art object detection methods.
Ranked #96 on
Object Detection
on COCO test-dev
FedOCR: Communication-Efficient Federated Learning for Scene Text Recognition
In this paper, we study how to make use of decentralized datasets for training a robust scene text recognizer while keeping them stay on local devices.
XingGAN for Person Image Generation
We propose a novel Generative Adversarial Network (XingGAN or CrossingGAN) for person image generation tasks, i. e., translating the pose of a given person to a desired one.
Ranked #1 on
Pose Transfer
on Market-1501
(IS metric)
Neural Architecture Search for Lightweight Non-Local Networks
However, it has been rarely explored to embed the NL blocks in mobile neural networks, mainly due to the following challenges: 1) NL blocks generally have heavy computation cost which makes it difficult to be applied in applications where computational resources are limited, and 2) it is an open problem to discover an optimal configuration to embed NL blocks into mobile neural networks.
Ranked #60 on
Neural Architecture Search
on ImageNet
Holistically-Attracted Wireframe Parsing
For computing line segment proposals, a novel exact dual representation is proposed which exploits a parsimonious geometric reparameterization for line segments and forms a holistic 4-dimensional attraction field map for an input image.
Ranked #4 on
Line Segment Detection
on wireframe dataset
AutoScale: Learning to Scale for Crowd Counting and Localization
A major issue is that the density map on dense regions usually accumulates density values from a number of nearby Gaussian blobs, yielding different large density values on a small set of pixels.
Learning Regional Attraction for Line Segment Detection
Given a line segment map, the proposed regional attraction first establishes the relationship between line segments and regions in the image lattice.
Anchor Diffusion for Unsupervised Video Object Segmentation
Unsupervised video object segmentation has often been tackled by methods based on recurrent neural networks and optical flow.
Ranked #21 on
Unsupervised Video Object Segmentation
on DAVIS 2016 val
Asymmetric Non-local Neural Networks for Semantic Segmentation
The non-local module works as a particularly useful technique for semantic segmentation while criticized for its prohibitive computation and GPU memory occupation.
Ranked #16 on
Semantic Segmentation
on COCO-Stuff test
Symmetry-constrained Rectification Network for Scene Text Recognition
Reading text in the wild is a very challenging task due to the diversity of text instances and the complexity of natural scenes.
View N-gram Network for 3D Object Retrieval
By doing so, spatial information across multiple views is captured, which helps to learn a discriminative global embedding for each 3D object.
Learn to Scale: Generating Multipolar Normalized Density Maps for Crowd Counting
Dense crowd counting aims to predict thousands of human instances from an image, by calculating integrals of a density map over image pixels.
Re-Ranking via Metric Fusion for Object Retrieval and Person Re-Identification
This work studies the unsupervised re-ranking procedure for object retrieval and person re-identification with a specific concentration on an ensemble of multiple metrics (or similarities).
CenterNet: Keypoint Triplets for Object Detection
In object detection, keypoint-based approaches often suffer a large number of incorrect object bounding boxes, arguably due to the lack of an additional look into the cropped regions.
Ranked #119 on
Object Detection
on COCO test-dev
Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation
Accurate multi-organ abdominal CT segmentation is essential to many clinical applications such as computer-aided intervention.
Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses
We observe the property of regional homogeneity in adversarial perturbations and suggest that the defenses are less robust to regionally homogeneous perturbations.
Adversarial Metric Attack and Defense for Person Re-identification
However, our work observes the extreme vulnerability of existing distance metrics to adversarial examples, generated by simply adding human-imperceptible perturbations to person images.
Hypergraph Convolution and Hypergraph Attention
To efficiently learn deep embeddings on the high-order graph-structured data, we introduce two end-to-end trainable operators to the family of graph neural networks, i. e., hypergraph convolution and hypergraph attention.
Learn to Interpret Atari Agents
Deep reinforcement learning (DeepRL) agents surpass human-level performance in many tasks.
Learning Transferable Adversarial Examples via Ghost Networks
The critical principle of ghost networks is to apply feature-level perturbations to an existing model to potentially create a huge set of diverse models.
Learning Attraction Field Representation for Robust Line Segment Detection
In experiments, our method is tested on the WireFrame dataset and the YorkUrban dataset with state-of-the-art performance obtained.
Hard-Aware Point-to-Set Deep Metric for Person Re-identification
Person re-identification (re-ID) is a highly challenging task due to large variations of pose, viewpoint, illumination, and occlusion.
PCL: Proposal Cluster Learning for Weakly Supervised Object Detection
The iterative instance classifier refinement is implemented online using multiple streams in convolutional neural networks, where the first is an MIL network and the others are for instance classifier refinement supervised by the preceding one.
Ranked #1 on
Weakly Supervised Object Detection
on ImageNet
Semi-Supervised Multi-Organ Segmentation via Deep Multi-Planar Co-Training
In multi-organ segmentation of abdominal CT scans, most existing fully supervised deep learning algorithms require lots of voxel-wise annotations, which are usually difficult, expensive, and slow to obtain.
Improving Transferability of Adversarial Examples with Input Diversity
We hope that our proposed attack strategy can serve as a strong benchmark baseline for evaluating the robustness of networks to adversaries and the effectiveness of different defense methods in the future.
Triplet-Center Loss for Multi-View 3D Object Retrieval
Most existing 3D object recognition algorithms focus on leveraging the strong discriminative power of deep learning models with softmax loss for the classification of 3D data, while learning discriminative features with deep metric learning for 3D object retrieval is more or less neglected.
Ensemble Diffusion for Retrieval
This stimulates a great research interest of considering similarity fusion in the framework of diffusion process (i. e., fusion with diffusion) for robust retrieval.
Divide and Fuse: A Re-ranking Approach for Person Re-identification
Finally, the new features from the same image are fused into one vector for re-ranking.
Scalable Person Re-identification on Supervised Smoothed Manifold
Most existing person re-identification algorithms either extract robust visual features or learn discriminative metrics for person images.
Ranked #110 on
Person Re-Identification
on Market-1501
Multidimensional Scaling on Multiple Input Distance Matrices
How to do multidimensional scaling on multiple input distance matrices is still unsolved to our best knowledge.
GIFT: A Real-time and Scalable 3D Shape Search Engine
We name the proposed 3D shape search engine, which combines GPU acceleration and Inverted File Twice, as GIFT.
Deep Learning Representation using Autoencoder for 3D Shape Retrieval
By combing the global deep learning representation and the local descriptor representation, our method can obtain the state-of-the-art performance on 3D shape retrieval benchmarks.
