Search Results for author:
Found 264 papers, 185 papers with code
Hybrid Task Cascade for Instance Segmentation
In exploring a more effective approach, we find that the key to a successful instance segmentation cascade is to fully leverage the reciprocal relationship between detection and segmentation.
Ranked #32 on
Object Detection
on COCO-O
CARAFE: Content-Aware ReAssembly of FEatures
CARAFE introduces little computational overhead and can be readily integrated into modern network architectures.
Ranked #3 on
Feature Upsampling
on ImageNet
MMDetection: Open MMLab Detection Toolbox and Benchmark
In this paper, we introduce the various features of this toolbox.
Seesaw Loss for Long-Tailed Instance Segmentation
Instances of head classes dominate a long-tailed dataset and they serve as negative samples of tail categories.
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation
In contrast to the occupancy pruning used in Neural Radiance Fields, we demonstrate that the progressive densification of 3D Gaussians converges significantly faster for 3D generative tasks.
VToonify: Controllable High-Resolution Portrait Video Style Transfer
Although a series of successful portrait image toonification models built upon the powerful StyleGAN have been proposed, these image-oriented methods have obvious limitations when applied to videos, such as the fixed frame size, the requirement of face alignment, missing non-facial details and temporal inconsistency.
Otter: A Multi-Modal Model with In-Context Instruction Tuning
Large language models (LLMs) have demonstrated significant universal capabilities as few/zero-shot learners in various tasks due to their pre-training on vast amounts of text data, as exemplified by GPT-3, which boosted to InstrctGPT and ChatGPT, effectively following natural language instructions to accomplish real-world tasks.
Ranked #8 on
Visual Question Answering
on BenchLMM
MIMIC-IT: Multi-Modal In-Context Instruction Tuning
We release the MIMIC-IT dataset, instruction-response collection pipeline, benchmarks, and the Otter model.
Ranked #81 on
Visual Question Answering
on MM-Vet
OtterHD: A High-Resolution Multi-modality Model
In this paper, we present OtterHD-8B, an innovative multimodal model evolved from Fuyu-8B, specifically engineered to interpret high-resolution visual inputs with granular precision.
Ranked #79 on
Visual Question Answering
on MM-Vet
Online Deep Clustering for Unsupervised Representation Learning
In this way, labels and the network evolve shoulder-to-shoulder rather than alternatingly.
Delving into Inter-Image Invariance for Unsupervised Visual Representations
In this work, we present a comprehensive empirical study to better understand the role of inter-image invariance learning from three main constituting components: pseudo-label maintenance, sampling strategy, and decision boundary design.
MMBench: Is Your Multi-modal Model an All-around Player?
In response to these challenges, we propose MMBench, a novel multi-modality benchmark.
Ranked #1 on
Visual Question Answering
on MMBench
MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation.
Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer
Recent studies on StyleGAN show high performance on artistic portrait generation by transfer learning with limited data.
Dynamic Graph CNN for Learning on Point Clouds
Point clouds provide a flexible geometric representation suitable for countless applications in computer graphics; they also comprise the raw output of most 3D data acquisition devices.
Ranked #6 on
Point Cloud Segmentation
on PointCloud-C
Learning to Prompt for Vision-Language Models
Large pre-trained vision-language models like CLIP have shown great potential in learning representations that are transferable across a wide range of downstream tasks.
Ranked #2 on
Few-shot Age Estimation
on MORPH Album2
Conditional Prompt Learning for Vision-Language Models
With the rise of powerful pre-trained vision-language models like CLIP, it becomes essential to investigate ways to adapt these models to downstream datasets.
Ranked #3 on
Prompt Engineering
on ImageNet V2
FreeU: Free Lunch in Diffusion U-Net
In this paper, we uncover the untapped potential of diffusion U-Net, which serves as a "free lunch" that substantially improves the generation quality on the fly.
Knowledge Distillation Meets Self-Supervision
Knowledge distillation, which involves extracting the "dark knowledge" from a teacher network to guide the learning of a student network, has emerged as an important technique for model compression and transfer learning.
Ranked #32 on
Knowledge Distillation
on ImageNet
MMFashion: An Open-Source Toolbox for Visual Fashion Analysis
This toolbox supports a wide spectrum of fashion analysis tasks, including Fashion Attribute Prediction, Fashion Recognition and Retrieval, Fashion Landmark Detection, Fashion Parsing and Segmentation and Fashion Compatibility and Recommendation.
LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation
2) 3D Backbone: We present an asymmetric U-Net as a high-throughput backbone operating on multi-view images, which can be produced from text or single-view image input by leveraging multi-view diffusion models.
StyleGAN-Human: A Data-Centric Odyssey of Human Generation
In addition, a model zoo and human editing applications are demonstrated to facilitate future research in the community.
Domain Generalization: A Survey
Generalization to out-of-distribution (OOD) data is a capability natural to humans yet challenging for machines to reproduce.
Semi-Supervised Domain Generalization with Stochastic StyleMatch
We find that the DG methods, which by design are unable to handle unlabeled data, perform poorly with limited labels in SSDG; the SSL methods, especially FixMatch, obtain much better results but are still far away from the basic vanilla model trained using full labels.
AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars
Our key insight is to take advantage of the powerful vision-language model CLIP for supervising neural human generation, in terms of 3D geometry, texture and animation.
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
While speech content information can be defined by learning the intrinsic synchronization between audio-visual modalities, we identify that a pose code will be complementarily learned in a modulated convolution-based reconstruction framework.
F$^{2}$-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories
Based on our analysis, we further propose a novel space-warping method called perspective warping, which allows us to handle arbitrary trajectories in the grid-based NeRF framework.
InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation
Specifically, we utilize a multi-scale approach to generate video-related descriptions.
Large-Scale Long-Tailed Recognition in an Open World
We define Open Long-Tailed Recognition (OLTR) as learning from such naturally distributed data and optimizing the classification accuracy over a balanced test set which include head, tail, and open classes.
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation
Talking face generation aims to synthesize a sequence of face images that correspond to a clip of speech.
Text2Human: Text-Driven Controllable Human Image Generation
In this work, we present a text-driven controllable framework, Text2Human, for a high-quality and diverse human generation.
Self-Supervised Scene De-occlusion
This is achieved via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner.
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected.
Ranked #17 on
Motion Synthesis
on KIT Motion-Language
Generalized Out-of-Distribution Detection: A Survey
In this survey, we first present a unified framework called generalized OOD detection, which encompasses the five aforementioned problems, i. e., AD, ND, OSR, OOD detection, and OD.
Full-Spectrum Out-of-Distribution Detection
In this paper, we take into account both shift types and introduce full-spectrum OOD (FS-OOD) detection, a more realistic problem setting that considers both detecting semantic shift and being tolerant to covariate shift; and designs three benchmarks.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
OpenOOD: Benchmarking Generalized Out-of-Distribution Detection
Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications and has thus been extensively studied, with a plethora of methods developed in the literature.
OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection
Out-of-Distribution (OOD) detection is critical for the reliable operation of open-world intelligent systems.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
SAD: Segment Any RGBD
The Segment Anything Model (SAM) has demonstrated its effectiveness in segmenting any part of 2D RGB images.
Open Vocabulary Semantic Segmentation
Panoptic Segmentation
+1
LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models
To this end, we propose LaVie, an integrated video generation framework that operates on cascaded video latent diffusion models, comprising a base T2V model, a temporal interpolation model, and a video super-resolution model.
Ranked #4 on
Text-to-Video Generation
on EvalCrafter Text-to-Video (ECTV) Dataset
(using extra training data)
Deep Class-Incremental Learning: A Survey
Deep models, e. g., CNNs and Vision Transformers, have achieved impressive achievements in many vision tasks in the closed world.
Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need
ADAM is a general framework that can be orthogonally combined with any parameter-efficient tuning method, which holds the advantages of PTM's generalizability and adapted model's adaptivity.
FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation
In this paper, we introduce FRESCO, intra-frame correspondence alongside inter-frame correspondence to establish a more robust spatial-temporal constraint.
Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs
Through our investigation, we found that such a pre-trained GAN indeed contains rich 3D knowledge and thus can be used to recover 3D shape from a single 2D image in an unsupervised manner.
Transformer-Based Visual Segmentation: A Survey
Recently, transformers, a type of neural network based on self-attention originally designed for natural language processing, have considerably surpassed previous convolutional or recurrent approaches in various vision processing tasks.
EVA3D: Compositional 3D Human Generation from 2D Image Collections
At the core of EVA3D is a compositional human NeRF representation, which divides the human body into local parts.
SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collections
Our approach begins with an efficient bird's-eye-view (BEV) representation generated from simplex noise, which includes a height field for surface elevation and a semantic field for detailed scene semantics.
Ranked #3 on
Scene Generation
on GoogleEarth
3DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors
Specifically, it is powered by a text-conditioned tri-plane latent diffusion model, which quickly generates coarse 3D samples for fast prototyping.
Text2Light: Zero-Shot Text-Driven HDR Panorama Generation
To achieve super-resolution inverse tone mapping, we derive a continuous representation of 360-degree imaging from the LDR panorama as a set of structured latent codes anchored to the sphere.
DeeperForensics Challenge 2020 on Real-World Face Forgery Detection: Methods and Results
This paper reports methods and results in the DeeperForensics Challenge 2020 on real-world face forgery detection.
CelebA-Spoof: Large-Scale Face Anti-Spoofing Dataset with Rich Annotations
The main reason is that current face anti-spoofing datasets are limited in both quantity and diversity.
CelebA-Spoof Challenge 2020 on Face Anti-Spoofing: Methods and Results
It is the largest face anti-spoofing dataset in terms of the numbers of the data and the subjects.
Segment Any Point Cloud Sequences by Distilling Vision Foundation Models
Recent advancements in vision foundation models (VFMs) have opened up new possibilities for versatile and efficient visual perception.
Rotate-and-Render: Unsupervised Photorealistic Face Rotation from Single-View Images
Though face rotation has achieved rapid progress in recent years, the lack of high-quality paired training data remains a great hurdle for existing methods.
CityDreamer: Compositional Generative Model of Unbounded 3D Cities
3D city generation is a desirable yet challenging task, since humans are more sensitive to structural distortions in urban environments.
Ranked #1 on
Scene Generation
on OSM
StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces
Recent advances in face manipulation using StyleGAN have produced impressive results.
Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition
Face recognition has witnessed great progress in recent years, mainly attributed to the high-capacity model designed and the abundant labeled data collected.
ReVersion: Diffusion-Based Relation Inversion from Images
Specifically, we propose a novel relation-steering contrastive learning scheme to impose two critical properties of the relation prompt: 1) The relation prompt should capture the interaction between objects, enforced by the preposition prior.
FreeInit: Bridging Initialization Gap in Video Diffusion Models
Though diffusion-based video generation has witnessed rapid progress, the inference results of existing models still exhibit unsatisfactory temporal consistency and unnatural dynamics.
OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation
Recent advances in modeling 3D objects mostly rely on synthetic datasets due to the lack of large-scale realscanned 3D databases.
Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction
Previous methods based on neural volume rendering mostly train a fully implicit model with MLPs, which typically require hours of training for a single scene.
DreamGaussian4D: Generative 4D Gaussian Splatting
Remarkable progress has been made in 4D content generation recently.
Deep Animation Video Interpolation in the Wild
In the animation industry, cartoon videos are usually produced at low frame rate since hand drawing of such frames is costly and time-consuming.
Panoptic Scene Graph Generation
Existing research addresses scene graph generation (SGG) -- a critical technology for scene understanding in images -- from a detection perspective, i. e., objects are detected using bounding boxes followed by prediction of their pairwise relationships.
Ranked #5 on
Panoptic Scene Graph Generation
on PSG Dataset
Collaborative Diffusion for Multi-Modal Face Generation and Editing
In this work, we present Collaborative Diffusion, where pre-trained uni-modal diffusion models collaborate to achieve multi-modal face generation and editing without re-training.
Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory
With the learned choreographic memory, dance generation is realized on the quantized units that meet high choreography standards, such that the generated dancing sequences are confined within the spatial constraints.
Ranked #1 on
Motion Synthesis
on AIST++
A Comprehensive Survey on 3D Content Generation
Recent years have witnessed remarkable advances in artificial intelligence generated content(AIGC), with diverse input modalities, e. g., text, image, video, audio and 3D.
Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets
We present a new loss function called Distribution-Balanced Loss for the multi-label recognition problems that exhibit long-tailed class distributions.
Ranked #7 on
Long-tail Learning
on VOC-MLT
CelebV-HQ: A Large-Scale Video Facial Attributes Dataset
Large-scale datasets have played indispensable roles in the recent success of face generation/editing and significantly facilitated the advances of emerging research fields.
Ranked #1 on
Unconditional Video Generation
on CelebV-HQ
Fashion Landmark Detection in the Wild
Fashion landmark is also compared to clothing bounding boxes and human joints in two applications, fashion attribute prediction and clothes retrieval, showing that fashion landmark is a more discriminative representation to understand fashion images.
Balanced MSE for Imbalanced Visual Regression
Data imbalance exists ubiquitously in real-world visual regressions, e. g., age estimation and pose estimation, hurting the model's generalizability and fairness.
FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling
With the availability of large-scale video datasets and the advances of diffusion models, text-driven video generation has achieved substantial progress.
Text2Performer: Text-Driven Human Video Generation
In this work, we present Text2Performer to generate vivid human videos with articulated motions from texts.
Talk-to-Edit: Fine-Grained Facial Editing via Dialog
In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system.
Ranked #1 on
Fine-Grained Facial Editing
on CelebA-Dialog
Video Object Segmentation with Re-identification
Specifically, our Video Object Segmentation with Re-identification (VS-ReID) model includes a mask propagation module and a ReID module.
ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model
However, the performance on more diverse motions remains unsatisfactory.
Ranked #1 on
Motion Synthesis
on KIT Motion-Language
SHERF: Generalizable Human NeRF from a Single Image
To this end, we propose a bank of 3D-aware hierarchical features, including global, point-level, and pixel-aligned features, to facilitate informative encoding.
RoboBEV: Towards Robust Bird's Eye View Perception under Corruptions
Our experiments further demonstrate that pre-training and depth-free BEV transformation has the potential to enhance out-of-distribution robustness.
Deep Geometrized Cartoon Line Inbetweening
To preserve the precision and detail of the line drawings, we propose a new approach, AnimeInbet, which geometrizes raster line drawings into graphs of endpoints and reframes the inbetweening task as a graph fusion problem with vertex repositioning.
Sparse Mixture-of-Experts are Domain Generalizable Learners
It is motivated by an empirical finding that transformer-based models trained with empirical risk minimization (ERM) outperform CNN-based models employing state-of-the-art (SOTA) DG algorithms on multiple DG datasets.
Ranked #11 on
Domain Generalization
on DomainNet
(using extra training data)
Robo3D: Towards Robust and Reliable 3D Perception against Corruptions
The robustness of 3D perception systems under natural corruptions from environments and sensors is pivotal for safety-critical applications.
Detecting and Grounding Multi-Modal Media Manipulation
In this paper, we highlight a new research problem for multi-modal fake media, namely Detecting and Grounding Multi-Modal Media Manipulation (DGM^4).
Detecting and Grounding Multi-Modal Media Manipulation and Beyond
HAMMER performs 1) manipulation-aware contrastive learning between two uni-modal encoders as shallow manipulation reasoning, and 2) modality-aware cross-attention by multi-modal aggregator as deep manipulation reasoning.
VBench: Comprehensive Benchmark Suite for Video Generative Models
We will open-source VBench, including all prompts, evaluation methods, generated videos, and human preference annotations, and also include more video generation models in VBench to drive forward the field of video generation.
Adaptive Affinity Fields for Semantic Segmentation
Semantic segmentation has made much progress with increasingly powerful pixel-wise classifiers and incorporating structural priors via Conditional Random Fields (CRF) or Generative Adversarial Networks (GAN).
Ranked #55 on
Semantic Segmentation
on Cityscapes test
DiffMimic: Efficient Motion Mimicking with Differentiable Physics
Motion mimicking is a foundational task in physics-based character animation.
LaserMix for Semi-Supervised LiDAR Semantic Segmentation
Densely annotating LiDAR point clouds is costly, which restrains the scalability of fully-supervised learning methods.
On-Device Domain Generalization
Another interesting observation is that the teacher-student gap on out-of-distribution data is bigger than that on in-distribution data, which highlights the capacity mismatch issue as well as the shortcoming of KD.
Relighting4D: Neural Relightable Human from Videos
Our key insight is that the space-time varying geometry and reflectance of the human body can be decomposed as a set of neural fields of normal, occlusion, diffuse, and specular maps.
Long-tailed Recognition by Routing Diverse Distribution-Aware Experts
We take a dynamic view of the training data and provide a principled model bias and variance analysis as the training data fluctuates: Existing long-tail classifiers invariably increase the model variance and the head-tail model bias gap remains large, due to more and larger confusion with hard negatives for the tail.
Ranked #22 on
Long-tail Learning
on iNaturalist 2018
GauHuman: Articulated Gaussian Splatting from Monocular Human Videos
We present, GauHuman, a 3D human model with Gaussian Splatting for both fast training (1 ~ 2 minutes) and real-time rendering (up to 189 FPS), compared with existing NeRF-based implicit representation modelling frameworks demanding hours of training and seconds of rendering per frame.
Ranked #1 on
Generalizable Novel View Synthesis
on ZJU-MoCap
LiDAR-based Panoptic Segmentation via Dynamic Shifting Network
2) Dynamic Shifting for complex point distributions.
Ranked #2 on
Panoptic Segmentation
on SemanticKITTI
LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network
In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner.
Octopus: Embodied Vision-Language Programmer from Environmental Feedback
Large vision-language models (VLMs) have achieved substantial progress in multimodal perception and reasoning.
TAda! Temporally-Adaptive Convolutions for Video Understanding
This work presents Temporally-Adaptive Convolutions (TAdaConv) for video understanding, which shows that adaptive weight calibration along the temporal dimension is an efficient way to facilitate modelling complex temporal dynamics in videos.
Ranked #67 on
Action Recognition
on Something-Something V2
(using extra training data)
Temporally-Adaptive Models for Efficient Video Understanding
Spatial convolutions are extensively used in numerous deep video models.
Ranked #3 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head Avatars
It is a large-scale digital library for head avatars with three key attributes: 1) High Fidelity: all subjects are captured by 60 synchronized, high-resolution 2K cameras in 360 degrees.
Video Frame Synthesis using Deep Voxel Flow
We combine the advantages of these two methods by training a deep network that learns to synthesize video frames by flowing pixel values from existing ones, which we call deep voxel flow.
Ranked #2 on
Video Prediction
on DAVIS 2017
Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation
In this work, we propose a novel diffusion-based framework, named Diffusion Co-Speech Gesture (DiffGesture), to effectively capture the cross-modal audio-to-gesture associations and preserve temporal coherence for high-fidelity audio-driven co-speech gesture generation.
Neural Prompt Search
The size of vision models has grown exponentially over the last few years, especially after the emergence of Vision Transformer.
Ranked #1 on
Image Classification
on OmniBenchmark
(using extra training data)
DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
Realistic human-centric rendering plays a key role in both computer vision and computer graphics.
Robust Reference-based Super-Resolution via C2-Matching
However, performing local transfer is difficult because of two gaps between input and reference images: the transformation gap (e. g. scale and rotation) and the resolution gap (e. g. HR and LR).
Reference-based Image and Video Super-Resolution via C2-Matching
To tackle these challenges, we propose C2-Matching in this work, which performs explicit robust matching crossing transformation and resolution.
One-shot Face Reenactment
However, in real-world scenario end-users often only have one target face at hand, rendering existing methods inapplicable.
Gemini vs GPT-4V: A Preliminary Comparison and Combination of Vision-Language Models Through Qualitative Cases
We conducted a series of structured experiments to evaluate their performance in various industrial application scenarios, offering a comprehensive perspective on their practical utility.
Unsupervised Image-to-Image Translation with Generative Prior
In this work, we present a novel framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm.
GP-UNIT: Generative Prior for Versatile Unsupervised Image-to-Image Translation
In this paper, we introduce a novel versatile framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), that improves the quality, applicability and controllability of the existing translation models.
GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
These metrics lack the flexibility to generalize to different evaluation criteria and might not align well with human preferences.
SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling
Synthetic data has emerged as a promising source for 3D human research as it offers low-cost access to large-scale human datasets.
WHAC: World-grounded Humans and Cameras
In this study, we aim to recover expressive parametric human models (i. e., SMPL-X) and corresponding camera poses jointly, by leveraging the synergy between three critical players: the world, the human, and the camera.
Benchmarking and Analyzing Point Cloud Classification under Corruptions
3D perception, especially point cloud classification, has achieved substantial progress.
Ranked #7 on
Point Cloud Classification
on PointCloud-C
Bamboo: Building Mega-Scale Vision Dataset Continually with Human-Machine Synergy
This work thus proposes a novel active learning framework for realistic dataset annotation.
Ranked #1 on
Image Classification
on Food-101
(using extra training data)
PERF: Panoramic Neural Radiance Field from a Single Panorama
In this paper, we present PERF, a 360-degree novel view synthesis framework that trains a panoramic neural radiance field from a single panorama.
What Makes Good Examples for Visual In-Context Learning?
To overcome the problem, we propose a prompt retrieval framework to automate the selection of in-context examples.
TCTrack: Temporal Contexts for Aerial Tracking
Temporal contexts among consecutive frames are far from being fully utilized in existing visual trackers.
Towards Real-World Visual Tracking with Temporal Contexts
To handle those problems, we propose a two-level framework (TCTrack) that can exploit temporal contexts efficiently.
Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis
In this paper, we present a spatial-temporal compression framework, \textbf{Fast-Vid2Vid}, which focuses on data aspects of generative models.
Variational Relational Point Completion Network
In particular, we propose a dual-path architecture to enable principled probabilistic modeling across partial and complete clouds.
Ranked #2 on
Point Cloud Completion
on Completion3D
Multi-View Partial (MVP) Point Cloud Challenge 2021 on Completion and Registration: Methods and Results
Based on the MVP dataset, this paper reports methods and results in the Multi-View Partial Point Cloud Challenge 2021 on Completion and Registration.
Self-Supervised Learning via Conditional Motion Propagation
Instead of explicitly modeling the motion probabilities, we design the pretext task as a conditional motion propagation problem.
Im2Avatar: Colorful 3D Reconstruction from a Single Image
In this work, we study a new problem, that is, simultaneously recovering 3D shape and surface color from a single image, namely "colorful 3D reconstruction".
Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other.
Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other.
Latte: Latent Diffusion Transformer for Video Generation
We propose a novel Latent Diffusion Transformer, namely Latte, for video generation.
StyleFaceV: Face Video Generation via Decomposing and Recomposing Pretrained StyleGAN3
Notably, StyleFaceV is capable of generating realistic $1024\times1024$ face videos even without high-resolution training videos.
Garment4D: Garment Reconstruction from Point Cloud Sequences
The main challenges are two-fold: 1) effective 3D feature learning for fine details, and 2) capture of garment dynamics caused by the interaction between garments and the human body, especially for loose garments like skirts.
DPatch: An Adversarial Patch Attack on Object Detectors
Successful realization of DPatch also illustrates the intrinsic vulnerability of the modern detector architectures to such patch-based adversarial attacks.
SinSR: Diffusion-Based Image Super-Resolution in a Single Step
Extensive experiments conducted on synthetic and real-world datasets demonstrate that the proposed method can achieve comparable or even superior performance compared to both previous SOTA methods and the teacher model, in just one sampling step, resulting in a remarkable up to x10 speedup for inference.
When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks
Recent advances in adversarial attacks uncover the intrinsic vulnerability of modern deep neural networks.
ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on
We build a series of scientific experiments to isolate effective design choices in video synthesis for virtual clothing try-on.
Detecting and Recovering Sequential DeepFake Manipulation
Moreover, we build a comprehensive benchmark and set up rigorous evaluation protocols and metrics for this new research problem.
Robust Sequential DeepFake Detection
However, existing methods only focus on detecting one-step facial manipulation.
Versatile Multi-Modal Pre-Training for Human-Centric Perception
To tackle the challenges, we design the novel Dense Intra-sample Contrastive Learning and Sparse Structure-aware Contrastive Learning targets by hierarchically learning a modal-invariant latent space featured with continuous and ordinal feature distribution and structure-aware semantic consistency.
Exploring Point-BEV Fusion for 3D Point Cloud Object Tracking with Transformer
In a point cloud sequence, 3D object tracking aims to predict the location and orientation of an object in consecutive frames given an object template.
StyleLight: HDR Panorama Generation for Lighting Estimation and Editing
To tackle this problem, we propose a coupled dual-StyleGAN panorama synthesis network (StyleLight) that integrates LDR and HDR panorama synthesis into a unified framework.
Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms
Experiments with 10 backbones, ranging from CNNs to transformers, show the knowledge learnt from a proximity task is readily transferable to human mesh recovery.
Not All Pixels Are Equal: Difficulty-aware Semantic Segmentation via Deep Layer Cascade
Third, in comparison to MC, LC is an end-to-end trainable framework, allowing joint learning of all sub-models.
Ranked #22 on
Semantic Segmentation
on PASCAL VOC 2012 test
Benchmarking Omni-Vision Representation through the Lens of Visual Realms
We benchmark ReCo and other advances in omni-vision representation studies that are different in architectures (from CNNs to transformers) and in learning paradigms (from supervised learning to self-supervised learning) on OmniBenchmark.
MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation
We present MosaicFusion, a simple yet effective diffusion-based data augmentation approach for large vocabulary instance segmentation.
PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation
We present PrimDiffusion, the first diffusion-based framework for 3D human generation.
Incorporating Convolution Designs into Visual Transformers
Motivated by the success of Transformers in natural language processing (NLP) tasks, there emerge some attempts (e. g., ViT and DeiT) to apply Transformers to the vision domain.
Ranked #2 on
Image Classification
on Oxford-IIIT Pets
Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination
Unsupervised feature learning has made great strides with contrastive learning based on instance discrimination and invariant mapping, as benchmarked on curated class-balanced datasets.
Contrastive Learning
Semi-Supervised Image Classification
+2
Adversarial Robustness under Long-Tailed Distribution
We then perform a systematic study on existing long-tailed recognition methods in conjunction with the adversarial training framework.
ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis
To counter this emerging threat, we construct the ForgeryNet dataset, an extremely large face forgery dataset with unified annotations in image- and video-level data across four tasks: 1) Image Forgery Classification, including two-way (real / fake), three-way (real / fake with identity-replaced forgery approaches / fake with identity-remained forgery approaches), and n-way (real and 15 respective forgery approaches) classification.
ReliTalk: Relightable Talking Portrait Generation from a Single Video
Our key insight is to decompose the portrait's reflectance from implicitly learned audio-driven facial normals and images.
Pair then Relation: Pair-Net for Panoptic Scene Graph Generation
Panoptic Scene Graph (PSG) is a challenging task in Scene Graph Generation (SGG) that aims to create a more comprehensive scene graph representation using panoptic segmentation instead of boxes.
FunQA: Towards Surprising Video Comprehension
Surprising videos, such as funny clips, creative performances, or visual illusions, attract significant attention.
Unconstrained Fashion Landmark Detection via Hierarchical Recurrent Transformer Networks
This work addresses unconstrained fashion landmark detection, where clothing bounding boxes are not provided in both training and test.
Delving Deep into the Generalization of Vision Transformers under Distribution Shifts
By comprehensively investigating these GE-ViTs and comparing with their corresponding CNN models, we observe: 1) For the enhanced model, larger ViTs still benefit more for the OOD generalization.
AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies
Existing correspondence datasets for two-dimensional (2D) cartoon suffer from simple frame composition and monotonic movements, making them insufficient to simulate real animations.
UnitedHuman: Harnessing Multi-Source Data for High-Resolution Human Generation
A holistic human dataset inevitably has insufficient and low-resolution information on local parts.
BiBERT: Accurate Fully Binarized BERT
The large pre-trained BERT has achieved remarkable performance on Natural Language Processing (NLP) tasks but is also computation and memory expensive.
FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing
Notably, FineMoGen further enables zero-shot motion editing capabilities with the aid of modern large language models (LLM), which faithfully manipulates motion sequences with fine-grained instructions.
Ranked #2 on
Motion Synthesis
on KIT Motion-Language
Delving Deep Into Hybrid Annotations for 3D Human Recovery in the Wild
Specifically, we focus on the challenging task of in-the-wild 3D human recovery from single images when paired 3D annotations are not fully available.
UNIF: United Neural Implicit Functions for Clothed Human Reconstruction and Animation
We propose united implicit functions (UNIF), a part-based method for clothed human reconstruction and animation with raw scans and skeletons as the input.
ConsistentNeRF: Enhancing Neural Radiance Fields with 3D Consistency for Sparse View Synthesis
In this paper, we propose ConsistentNeRF, a method that leverages depth information to regularize both multi-view and single-view 3D consistency among pixels.
Semantically Coherent Out-of-Distribution Detection
The proposed UDG can not only enrich the semantic knowledge of the model by exploiting unlabeled data in an unsupervised manner, but also distinguish ID/OOD samples to enhance ID classification and OOD detection tasks simultaneously.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
AID: Attention Interpolation of Text-to-Image Diffusion
To that end, we introduce a novel training-free technique named Attention Interpolation via Diffusion (AID).
Link-Context Learning for Multimodal LLMs
The ability to learn from context with novel concepts, and deliver appropriate responses are essential in human conversations.
Masked Frequency Modeling for Self-Supervised Visual Pre-Training
We present Masked Frequency Modeling (MFM), a unified frequency-domain-based approach for self-supervised pre-training of visual models.
Unsupervised Object-Level Representation Learning from Scene Images
Extensive experiments on COCO show that ORL significantly improves the performance of self-supervised learning on scene images, even surpassing supervised ImageNet pre-training on several downstream tasks.
Few-Shot Object Detection via Association and DIscrimination
1) In the association step, in contrast to implicitly leveraging multiple base classes, we construct a compact novel class feature space via explicitly imitating a specific base class feature space.
Deep Learning Face Attributes in the Wild
LNet is pre-trained by massive general object categories for face localization, while ANet is pre-trained by massive face identities for attribute prediction.
Ranked #6 on
Facial Attribute Classification
on LFWA
Panoptic Video Scene Graph Generation
PVSG relates to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects grounded with bounding boxes in videos.
DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
BiBench: Benchmarking and Analyzing Network Binarization
Network binarization emerges as one of the most promising compression approaches offering extraordinary computation and memory savings by minimizing the bit-width.
Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency
In addition, existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world.
Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement
Learning a good 3D human pose representation is important for human pose related tasks, e. g. human 3D pose estimation and action recognition.
DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection
Unlike existing deepfake detection methods merely focusing on low-level forgery patterns, the forgery detection process of our model can be regularized by generalizable high-level semantics from a pre-trained ViT and adapted by global and local low-level forgeries of deepfake data.
Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understanding
Safety-critical 3D scene understanding tasks necessitate not only accurate but also confident predictions from 3D perception models.
Robust Partial-to-Partial Point Cloud Registration in a Full Range
\textbf{3)} Based on a synergy of hierarchical graph networks and graphical modeling, we propose the {H}ierarchical {G}raphical {M}odeling (\textbf{HGM}) architecture to encode robust descriptors consisting of i) a unary term learned from {\textit{RI}} features; and ii) multiple smoothness terms encoded from neighboring point relations at different scales through our TPT modules.
Towards Robust and Expressive Whole-body Human Pose and Shape Estimation
Whole-body pose and shape estimation aims to jointly predict different behaviors (e. g., pose, hand gesture, facial expression) of the entire human body from a monocular image.
Hierarchy Flow For High-Fidelity Image-to-Image Translation
In this work, we propose Hierarchy Flow, a novel flow-based model to achieve better content preservation during translation.
Unsolvable Problem Detection: Evaluating Trustworthiness of Vision Language Models
This paper introduces a novel and significant challenge for Vision Language Models (VLMs), termed Unsolvable Problem Detection (UPD).
Visual Sound Localization in the Wild by Cross-Modal Interference Erasing
Specifically, we observe that the previous practice of learning only a single audio representation is insufficient due to the additive nature of audio signals.
Mind the Gap in Distilling StyleGANs
To further enhance the semantic consistency between the teacher and student model, we present a latent-direction-based distillation loss that preserves the semantic relations in latent space.
Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup
Our goal is to achieve a performance comparable to conventional knowledge distillation with a lower computation cost during training.
PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention
Generating 3D point clouds is challenging yet highly desired.
Exploiting Hierarchical Interactions for Protein Surface Learning
In this paper, we present a principled framework based on deep learning techniques, namely Hierarchical Chemical and Geometric Feature Interaction Network (HCGNet), for protein surface analysis by bridging chemical and geometric features with hierarchical interactions.
Iterative Human and Automated Identification of Wildlife Images
Camera trapping is increasingly used to monitor wildlife, but this technology typically requires extensive data annotation.
Free Lunch for Surgical Video Understanding by Distilling Self-Supervisions
Our key insight is to distill knowledge from publicly available models trained on large generic datasets4 to facilitate the self-supervised learning of surgical videos.
TripleE: Easy Domain Generalization via Episodic Replay
In this paper, we propose TripleE, and the main idea is to encourage the network to focus on training on subsets (learning with replay) and enlarge the data space in learning on subsets.
Vlogger: Make Your Dream A Vlog
More importantly, Vlogger can generate over 5-minute vlogs from open-world descriptions, without loss of video coherence on script and actor.
Unsupervised Landmark Learning from Unpaired Data
Recent attempts for unsupervised landmark learning leverage synthesized image pairs that are similar in appearance but different in poses.
InstructVideo: Instructing Video Diffusion Models with Human Feedback
To tackle this problem, we propose InstructVideo to instruct text-to-video diffusion models with human feedback by reward fine-tuning.
A Two-Stage Generative Model with CycleGAN and Joint Diffusion for MRI-based Brain Tumor Detection
However, current supervised learning methods require extensively annotated images and the state-of-the-art generative models used in unsupervised methods often have limitations in covering the whole data distribution.
Mix-and-Match Tuning for Self-Supervised Semantic Segmentation
The key of this new form of learning is to design a proxy task (e. g. image colorization), from which a discriminative loss can be formulated on unlabeled data.
Deep Learning Markov Random Field for Semantic Segmentation
Semantic segmentation tasks can be well modeled by Markov Random Field (MRF).
Semantic Facial Expression Editing using Autoencoded Flow
High-level manipulation of facial expressions in images --- such as changing a smile to a neutral expression --- is challenging because facial expression changes are highly non-linear, and vary depending on the appearance of the face.
Semantic Image Segmentation via Deep Parsing Network
This paper addresses semantic image segmentation by incorporating rich information into Markov Random Field (MRF), including high-order relations and mixture of label contexts.
Ranked #89 on
Semantic Segmentation
on Cityscapes test
Instance-level Facial Attributes Transfer with Geometry-Aware Flow
Geometry-aware flow is able to warp the source face attribute into the target face context and generate a warp-and-blend result.
DeepFashion: Powering Robust Clothes Recognition and Retrieval With Rich Annotations
To demonstrate the advantages of DeepFashion, we propose a new deep model, namely FashionNet, which learns clothing features by jointly predicting clothing attributes and landmarks.
Open Compound Domain Adaptation
A typical domain adaptation approach is to adapt models trained on the annotated data in a source domain (e. g., sunny weather) for achieving high performance on the test data in a target domain (e. g., rainy weather).
Vision-Infused Deep Audio Inpainting
Extensive experiments demonstrate that our framework is capable of inpainting realistic and varying audio segments with or without visual contexts.
Learning to Synthesize Fashion Textures
Another important observation is that fashion textures are multi-modal.
Learning Diverse Fashion Collocation by Neural Graph Filtering
To facilitate a comprehensive study on diverse fashion collocation, we reorganize Amazon Fashion dataset with carefully designed evaluation protocols.
Placepedia: Comprehensive Place Understanding with Multi-Faceted Annotations
Place is an important element in visual understanding.
Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation
Stereophonic audio is an indispensable ingredient to enhance human auditory experience.
Person-in-Context Synthesiswith Compositional Structural Space
To handle the stark difference in input structures, we proposed two separate neural branches to attentively composite the respective (context/person) inputs into shared ``compositional structural space'', which encodes shape, location and appearance information for both context and person structures in a disentangled manner.
CARAFE++: Unified Content-Aware ReAssembly of FEatures
Feature reassembly, i. e. feature downsampling and upsampling, is a key operation in a number of modern convolutional network architectures, e. g., residual networks and feature pyramids.
Chasing the Tail in Monocular 3D Human Reconstruction with Prototype Memory
The reason is that most of the current models perform regression based on a single human prototype, which is similar to common poses while far from the rare poses.
