Search Results for author:
Found 222 papers, 161 papers with code
SinSR: Diffusion-Based Image Super-Resolution in a Single Step
Extensive experiments conducted on synthetic and real-world datasets demonstrate that the proposed method can achieve comparable or even superior performance compared to both previous SOTA methods and the teacher model, in just one sampling step, resulting in a remarkable up to x10 speedup for inference.
OtterHD: A High-Resolution Multi-modality Model
In this paper, we present OtterHD-8B, an innovative multimodal model evolved from Fuyu-8B, specifically engineered to interpret high-resolution visual inputs with granular precision.
A Two-Stage Generative Model with CycleGAN and Joint Diffusion for MRI-based Brain Tumor Detection
However, current supervised learning methods require extensively annotated images and the state-of-the-art generative models used in unsupervised methods often have limitations in covering the whole data distribution.
Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images
In particular, we instruction tune vision-language models to generate detailed visual descriptions of camera trap images using similar terminology to experts.
SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction
The goal is to generate high-quality long videos with smooth and creative transitions between scenes and varying lengths of shot-level videos.
PERF: Panoramic Neural Radiance Field from a Single Panorama
In this paper, we present PERF, a 360-degree novel view synthesis framework that trains a panoramic neural radiance field from a single panorama.
Large Language Models are Visual Reasoning Coordinators
Visual reasoning requires multimodal perception and commonsense cognition of the world.
FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling
With the availability of large-scale video datasets and the advances of diffusion models, text-driven video generation has achieved substantial progress.
HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion
Our model enforces the joint learning of image appearance, spatial relationship, and geometry in a unified network, where each branch in the model complements to each other with both structural awareness and textural richness.
Octopus: Embodied Vision-Language Programmer from Environmental Feedback
Large vision-language models (VLMs) have achieved substantial progress in multimodal perception and reasoning.
Deep Geometrized Cartoon Line Inbetweening
To preserve the precision and detail of the line drawings, we propose a new approach, AnimeInbet, which geometrizes raster line drawings into graphs of endpoints and reframes the inbetweening task as a graph fusion problem with vertex repositioning.
Cloth2Body: Generating 3D Human Body Mesh from 2D Clothing
Finally, how to generate diverse and plausible results from a 2D clothing image.
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation
In contrast to the occupancy pruning used in Neural Radiance Fields, we demonstrate that the progressive densification of 3D Gaussians converges significantly faster for 3D generative tasks.
Robust Sequential DeepFake Detection
However, existing methods only focus on detecting one-step facial manipulation.
LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models
To this end, we propose LaVie, an integrated video generation framework that operates on cascaded video latent diffusion models, comprising a base T2V model, a temporal interpolation model, and a video super-resolution model.
Detecting and Grounding Multi-Modal Media Manipulation and Beyond
HAMMER performs 1) manipulation-aware contrastive learning between two uni-modal encoders as shallow manipulation reasoning, and 2) modality-aware cross-attention by multi-modal aggregator as deep manipulation reasoning.
UnitedHuman: Harnessing Multi-Source Data for High-Resolution Human Generation
A holistic human dataset inevitably has insufficient and low-resolution information on local parts.
MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation
We present MosaicFusion, a simple yet effective diffusion-based data augmentation approach for large vocabulary instance segmentation.
FreeU: Free Lunch in Diffusion U-Net
In this paper, we uncover the untapped potential of diffusion U-Net, which serves as a "free lunch" that substantially improves the generation quality on the fly.
Large-Vocabulary 3D Diffusion Model with Transformer
To this end, we propose a novel triplane-based 3D-aware Diffusion model with TransFormer, DiffTF, for handling challenges via three aspects.
DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
ReliTalk: Relightable Talking Portrait Generation from a Single Video
Our key insight is to decompose the portrait's reflectance from implicitly learned audio-driven facial normals and images.
CityDreamer: Compositional Generative Model of Unbounded 3D Cities
In recent years, extensive research has focused on 3D natural scene generation, but the domain of 3D city generation has not received as much exploration.
PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds
To tackle these challenges, we propose a principled framework, PointHPS, for accurate 3D HPS from point clouds captured in real-world settings, which iteratively refines point features through a cascaded architecture.
Towards Real-World Visual Tracking with Temporal Contexts
To handle those problems, we propose a two-level framework (TCTrack) that can exploit temporal contexts efficiently.
HumanLiff: Layer-wise 3D Human Generation with Diffusion Model
In this work, we propose HumanLiff, the first layer-wise 3D human generative model with a unified diffusion process.
Link-Context Learning for Multimodal LLMs
The ability to learn from context with novel concepts, and deliver appropriate responses are essential in human conversations.
Hierarchy Flow For High-Fidelity Image-to-Image Translation
In this work, we propose Hierarchy Flow, a novel flow-based model to achieve better content preservation during translation.
Temporally-Adaptive Models for Efficient Video Understanding
Spatial convolutions are extensively used in numerous deep video models.
Ranked #3 on
Action Recognition
on EPIC-KITCHENS-100
(using extra training data)
Benchmarking and Analyzing Generative Data for Visual Recognition
Advancements in large pre-trained generative models have expanded their potential as effective data generators in visual recognition.
Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis
This paradigm extends the 2D image diffusion model to a volumetric version with a slightly increasing number of parameters and computation, offering a principled solution for generic cross-modality 3D medical image synthesis.
DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
Realistic human-centric rendering plays a key role in both computer vision and computer graphics.
Pair then Relation: Pair-Net for Panoptic Scene Graph Generation
Panoptic Scene Graph (PSG) is a challenging task in Scene Graph Generation (SGG) that aims to create a more comprehensive scene graph representation using panoptic segmentation instead of boxes.
InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation
Specifically, we utilize a multi-scale approach to generate video-related descriptions.
MMBench: Is Your Multi-modal Model an All-around Player?
In response to these challenges, we propose MMBench, a novel multi-modality benchmark.
FunQA: Towards Surprising Video Comprehension
Surprising videos, e. g., funny clips, creative performances, or visual illusions, attract significant attention.
Segment Any Point Cloud Sequences by Distilling Vision Foundation Models
Recent advancements in vision foundation models (VFMs) have opened up new possibilities for versatile and efficient visual perception.
OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection
Out-of-Distribution (OOD) detection is critical for the reliable operation of open-world intelligent systems.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation
The framework includes two parts: key frame translation and full video translation.
MIMIC-IT: Multi-Modal In-Context Instruction Tuning
We release the MIMIC-IT dataset, instruction-response collection pipeline, benchmarks, and the Otter model.
Ranked #30 on
Visual Question Answering
on MM-Vet
GP-UNIT: Generative Prior for Versatile Unsupervised Image-to-Image Translation
In this paper, we introduce a novel versatile framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), that improves the quality, applicability and controllability of the existing translation models.
DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection
Unlike existing deepfake detection methods merely focusing on low-level forgery patterns, the forgery detection process of our model can be regularized by generalizable high-level semantics from a pre-trained ViT and adapted by global and local low-level forgeries of deepfake data.
Learning without Forgetting for Vision-Language Models
While traditional CIL methods focus on visual information to grasp core features, recent advances in Vision-Language Models (VLM) have shown promising capabilities in learning generalizable representations with the aid of textual information.
SAD: Segment Any RGBD
The Segment Anything Model (SAM) has demonstrated its effectiveness in segmenting any part of 2D RGB images.
Open Vocabulary Semantic Segmentation
Panoptic Segmentation
+1
ConsistentNeRF: Enhancing Neural Radiance Fields with 3D Consistency for Sparse View Synthesis
In this paper, we propose ConsistentNeRF, a method that leverages depth information to regularize both multi-view and single-view 3D consistency among pixels.
StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generator
Despite recent advances in syncing lip movements with any audio waves, current methods still struggle to balance generation quality and the model's generalization ability.
Otter: A Multi-Modal Model with In-Context Instruction Tuning
Large language models (LLMs) have demonstrated significant universal capabilities as few/zero-shot learners in various tasks due to their pre-training on vast amounts of text data, as exemplified by GPT-3, which boosted to InstrctGPT and ChatGPT, effectively following natural language instructions to accomplish real-world tasks.
Transmissive Reconfigurable Intelligent Surface Transmitter Empowered Cognitive RSMA Networks
In this paper, we investigated the downlink transmission problem of a cognitive radio network (CRN) equipped with a novel transmissive reconfigurable intelligent surface (TRIS) transmitter.
Collaborative Diffusion for Multi-Modal Face Generation and Editing
In this work, we present Collaborative Diffusion, where pre-trained uni-modal diffusion models collaborate to achieve multi-modal face generation and editing without re-training.
Transformer-Based Visual Segmentation: A Survey
Recently, transformers, a type of neural network based on self-attention originally designed for natural language processing, have considerably surpassed previous convolutional or recurrent approaches in various vision processing tasks.
Variational Relational Point Completion Network for Robust 3D Classification
Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details.
Text2Performer: Text-Driven Human Video Generation
In this work, we present Text2Performer to generate vivid human videos with articulated motions from texts.
RoboBEV: Towards Robust Bird's Eye View Perception under Corruptions
Our experiments further demonstrate that pre-training and depth-free BEV transformation has the potential to enhance out-of-distribution robustness.
DiffMimic: Efficient Motion Mimicking with Differentiable Physics
Motion mimicking is a foundational task in physics-based character animation.
Detecting and Grounding Multi-Modal Media Manipulation
In this paper, we highlight a new research problem for multi-modal fake media, namely Detecting and Grounding Multi-Modal Media Manipulation (DGM^4).
ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model
However, the performance on more diverse motions remains unsatisfactory.
Ranked #3 on
Motion Synthesis
on HumanML3D
Robo3D: Towards Robust and Reliable 3D Perception against Corruptions
The robustness of 3D perception systems under natural corruptions from environments and sensors is pivotal for safety-critical applications.
SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling
Synthetic data has emerged as a promising source for 3D human research as it offers low-cost access to large-scale human datasets.
SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis
Since coarse depth maps are not strictly scaled to the ground-truth depth maps, we propose a simple yet effective constraint, a local depth ranking method, on NeRFs such that the expected depth ranking of the NeRF is consistent with that of the coarse depth maps in local patches.
F$^{2}$-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories
Based on our analysis, we further propose a novel space-warping method called perspective warping, which allows us to handle arbitrary trajectories in the grid-based NeRF framework.
A Simple and Generic Framework for Feature Distillation via Channel-wise Transformation
Specifically, we propose a learnable nonlinear channel-wise transformation to align the features of the student and the teacher model.
ReVersion: Diffusion-Based Relation Inversion from Images
Specifically, we propose a novel relation-steering contrastive learning scheme to impose two critical properties of the relation prompt: 1) The relation prompt should capture the interaction between objects, enforced by the preposition prior.
SHERF: Generalizable Human NeRF from a Single Image
To this end, we propose a bank of 3D-aware hierarchical features, including global, point-level, and pixel-aligned features, to facilitate informative encoding.
Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation
In this work, we propose a novel diffusion-based framework, named Diffusion Co-Speech Gesture (DiffGesture), to effectively capture the cross-modal audio-to-gesture associations and preserve temporal coherence for high-fidelity audio-driven co-speech gesture generation.
Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need
ADAM is a general framework that can be orthogonally combined with any parameter-efficient tuning method, which holds the advantages of PTM's generalizability and adapted model's adaptivity.
StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces
Recent advances in face manipulation using StyleGAN have produced impressive results.
Rethinking Range View Representation for LiDAR Segmentation
We show that, for the first time, a range view method is able to surpass the point, voxel, and multi-view fusion counterparts in the competing LiDAR semantic and panoptic segmentation benchmarks, i. e., SemanticKITTI, nuScenes, and ScribbleKITTI.
Ranked #2 on
3D Semantic Segmentation
on SemanticKITTI
A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
This study provides a comprehensive review of recent research advancements, challenges, and opportunities for PFMs in text, image, graph, as well as other data modalities.
Make Your Brief Stroke Real and Stereoscopic: 3D-Aware Simplified Sketch to Portrait Generation
Creating the photo-realistic version of people sketched portraits is useful to various entertainment purposes.
Deep Class-Incremental Learning: A Survey
Deep models, e. g., CNNs and Vision Transformers, have achieved impressive achievements in many vision tasks in the closed world.
SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collections
Our approach begins with an efficient bird's-eye-view (BEV) representation generated from simplex noise, which includes a height field for surface elevation and a semantic field for detailed scene semantics.
What Makes Good Examples for Visual In-Context Learning?
To overcome the problem, we propose a prompt retrieval framework to automate the selection of in-context examples.
BiBench: Benchmarking and Analyzing Network Binarization
Network binarization emerges as one of the most promising compression approaches offering extraordinary computation and memory savings by minimizing the bit-width.
OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation
Recent advances in modeling 3D objects mostly rely on synthetic datasets due to the lack of large-scale realscanned 3D databases.
F2-NeRF: Fast Neural Radiance Field Training With Free Camera Trajectories
Existing fast grid-based NeRF training frameworks, like Instant-NGP, Plenoxels, DVGO, or TensoRF, are mainly designed for bounded scenes and rely on space warping to handle unbounded scenes.
Panoptic Video Scene Graph Generation
PVSG is related to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects localized with bounding boxes in videos.
DeformToon3D: Deformable Neural Radiance Fields for 3D Toonification
In this paper, we address the challenging problem of 3D toonification, which involves transferring the style of an artistic domain onto a target 3D face with stylized geometry and texture.
Reference-based Image and Video Super-Resolution via C2-Matching
To tackle these challenges, we propose C2-Matching in this work, which performs explicit robust matching crossing transformation and resolution.
Masked Lip-Sync Prediction by Audio-Visual Contextual Exploitation in Transformers
This requires masking a large percentage of the original image and seamlessly inpainting it with the aid of audio and reference frames.
Audio-Driven Co-Speech Gesture Video Generation
Our key insight is that the co-speech gestures can be decomposed into common motion patterns and subtle rhythmic dynamics.
AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies
Existing correspondence datasets for two-dimensional (2D) cartoon suffer from simple frame composition and monotonic movements, making them insufficient to simulate real animations.
Joint Communication and Computation Design in Transmissive RMS Transceiver Enabled Multi-Tier Computing Networks
We formulate a total energy consumption minimization problem by a joint optimization of subcarrier allocation, task input bits, time slot allocation, transmit power allocation and RMS transmissive coefficient while taking into account the constraints of communication resources and computing resources.
OpenOOD: Benchmarking Generalized Out-of-Distribution Detection
Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications and has thus been extensively studied, with a plethora of methods developed in the literature.
EVA3D: Compositional 3D Human Generation from 2D Image Collections
At the core of EVA3D is a compositional human NeRF representation, which divides the human body into local parts.
TripleE: Easy Domain Generalization via Episodic Replay
In this paper, we propose TripleE, and the main idea is to encourage the network to focus on training on subsets (learning with replay) and enlarge the data space in learning on subsets.
StyleSwap: Style-Based Generator Empowers Robust Face Swapping
Our core idea is to leverage a style-based generator to empower high-fidelity and robust face swapping, thus the generator's advantage can be adopted for optimizing identity similarity.
VToonify: Controllable High-Resolution Portrait Video Style Transfer
Although a series of successful portrait image toonification models built upon the powerful StyleGAN have been proposed, these image-oriented methods have obvious limitations when applied to videos, such as the fixed frame size, the requirement of face alignment, missing non-facial details and temporal inconsistency.
Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms
Experiments with 10 backbones, ranging from CNNs to transformers, show the knowledge learnt from a proximity task is readily transferable to human mesh recovery.
Text2Light: Zero-Shot Text-Driven HDR Panorama Generation
To achieve super-resolution inverse tone mapping, we derive a continuous representation of 360-degree imaging from the LDR panorama as a set of structured latent codes anchored to the sphere.
On-Device Domain Generalization
Another interesting observation is that the teacher-student gap on out-of-distribution data is bigger than that on in-distribution data, which highlights the capacity mismatch issue as well as the shortcoming of KD.
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected.
Ranked #10 on
Motion Synthesis
on KIT Motion-Language
Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction
Previous methods based on neural volume rendering mostly train a fully implicit model with MLPs, which typically require hours of training for a single scene.
Mind the Gap in Distilling StyleGANs
To further enhance the semantic consistency between the teacher and student model, we present a latent-direction-based distillation loss that preserves the semantic relations in latent space.
Open Long-Tailed Recognition in a Dynamic World
A practical recognition system must balance between majority (head) and minority (tail) classes, generalize across the distribution, and acknowledge novelty upon the instances of unseen classes (open classes).
StyleFaceV: Face Video Generation via Decomposing and Recomposing Pretrained StyleGAN3
Notably, StyleFaceV is capable of generating realistic $1024\times1024$ face videos even without high-resolution training videos.
Exploring Point-BEV Fusion for 3D Point Cloud Object Tracking with Transformer
In a point cloud sequence, 3D object tracking aims to predict the location and orientation of an object in consecutive frames given an object template.
StyleLight: HDR Panorama Generation for Lighting Estimation and Editing
To tackle this problem, we propose a coupled dual-StyleGAN panorama synthesis network (StyleLight) that integrates LDR and HDR panorama synthesis into a unified framework.
Multi-Forgery Detection Challenge 2022: Push the Frontier of Unconstrained and Diverse Forgery Detection
In this paper, we present the solutions from the Top 3 teams, in order to boost the research work in the field of image forgery detection.
CelebV-HQ: A Large-Scale Video Facial Attributes Dataset
Large-scale datasets have played indispensable roles in the recent success of face generation/editing and significantly facilitated the advances of emerging research fields.
Ranked #1 on
Unconditional Video Generation
on CelebV-HQ
Panoptic Scene Graph Generation
Existing research addresses scene graph generation (SGG) -- a critical technology for scene understanding in images -- from a detection perspective, i. e., objects are detected using bounding boxes followed by prediction of their pairwise relationships.
Ranked #3 on
Panoptic Scene Graph Generation
on PSG Dataset
UNIF: United Neural Implicit Functions for Clothed Human Reconstruction and Animation
We propose united implicit functions (UNIF), a part-based method for clothed human reconstruction and animation with raw scans and skeletons as the input.
Relighting4D: Neural Relightable Human from Videos
Our key insight is that the space-time varying geometry and reflectance of the human body can be decomposed as a set of neural fields of normal, occlusion, diffuse, and specular maps.
Benchmarking Omni-Vision Representation through the Lens of Visual Realms
We benchmark ReCo and other advances in omni-vision representation studies that are different in architectures (from CNNs to transformers) and in learning paradigms (from supervised learning to self-supervised learning) on OmniBenchmark.
Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis
In this paper, we present a spatial-temporal compression framework, \textbf{Fast-Vid2Vid}, which focuses on data aspects of generative models.
Detecting and Recovering Sequential DeepFake Manipulation
Moreover, we build a comprehensive benchmark and set up rigorous evaluation protocols and metrics for this new research problem.
LaserMix for Semi-Supervised LiDAR Semantic Segmentation
Densely annotating LiDAR point clouds is costly, which restrains the scalability of fully-supervised learning methods.
Masked Frequency Modeling for Self-Supervised Visual Pre-Training
We present Masked Frequency Modeling (MFM), a unified frequency-domain-based approach for self-supervised pre-training of visual models.
Neural Prompt Search
The size of vision models has grown exponentially over the last few years, especially after the emergence of Vision Transformer.
Ranked #1 on
Image Classification
on OmniBenchmark
(using extra training data)
Sparse Mixture-of-Experts are Domain Generalizable Learners
It is motivated by an empirical finding that transformer-based models trained with empirical risk minimization (ERM) outperform CNN-based models employing state-of-the-art (SOTA) DG algorithms on multiple DG datasets.
Ranked #10 on
Domain Generalization
on DomainNet
(using extra training data)
Text2Human: Text-Driven Controllable Human Image Generation
In this work, we present a text-driven controllable framework, Text2Human, for a high-quality and diverse human generation.
Free Lunch for Surgical Video Understanding by Distilling Self-Supervisions
Our key insight is to distill knowledge from publicly available models trained on large generic datasets4 to facilitate the self-supervised learning of surgical videos.
AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars
Our key insight is to take advantage of the powerful vision-language model CLIP for supervising neural human generation, in terms of 3D geometry, texture and animation.
HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling
4D human sensing and modeling are fundamental tasks in vision and graphics with numerous applications.
Robust Face Anti-Spoofing with Dual Probabilistic Modeling
In this work, we attempt to fill this gap by automatically addressing the noise problem from both label and data perspectives in a probabilistic manner.
StyleGAN-Human: A Data-Centric Odyssey of Human Generation
In addition, a model zoo and human editing applications are demonstrated to facilitate future research in the community.
Few-shot Forgery Detection via Guided Adversarial Interpolation
Notably, our method is also validated to be robust to choices of majority and minority forgery approaches.
Full-Spectrum Out-of-Distribution Detection
In this paper, we take into account both shift types and introduce full-spectrum OOD (FS-OOD) detection, a more realistic problem setting that considers both detecting semantic shift and being tolerant to covariate shift; and designs three benchmarks.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Unsupervised Image-to-Image Translation with Generative Prior
In this work, we present a novel framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm.
Balanced MSE for Imbalanced Visual Regression
Data imbalance exists ubiquitously in real-world visual regressions, e. g., age estimation and pose estimation, hurting the model's generalizability and fairness.
Versatile Multi-Modal Pre-Training for Human-Centric Perception
To tackle the challenges, we design the novel Dense Intra-sample Contrastive Learning and Sparse Structure-aware Contrastive Learning targets by hierarchically learning a modal-invariant latent space featured with continuous and ordinal feature distribution and structure-aware semantic consistency.
SeCo: Separating Unknown Musical Visual Sounds with Consistency Guidance
Recent years have witnessed the success of deep learning on the visual sound separation task.
Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer
Recent studies on StyleGAN show high performance on artistic portrait generation by transfer learning with limited data.
Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory
With the learned choreographic memory, dance generation is realized on the quantized units that meet high choreography standards, such that the generated dancing sequences are confined within the spatial constraints.
Ranked #4 on
Motion Synthesis
on AIST++
X-Learner: Learning Cross Sources and Tasks for Universal Visual Representation
2) Squeeze Stage: X-Learner condenses the model to a reasonable size and learns the universal and generalizable representation for various tasks transferring.
Bamboo: Building Mega-Scale Vision Dataset Continually with Human-Machine Synergy
This work thus proposes a novel active learning framework for realistic dataset annotation.
Ranked #1 on
Image Classification
on Food-101
(using extra training data)
LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network
In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner.
BiBERT: Accurate Fully Binarized BERT
The large pre-trained BERT has achieved remarkable performance on Natural Language Processing (NLP) tasks but is also computation and memory expensive.
Conditional Prompt Learning for Vision-Language Models
With the rise of powerful pre-trained vision-language models like CLIP, it becomes essential to investigate ways to adapt these models to downstream datasets.
Ranked #2 on
Prompt Engineering
on ImageNet V2
TCTrack: Temporal Contexts for Aerial Tracking
Temporal contexts among consecutive frames are far from being fully utilized in existing visual trackers.
Visual Sound Localization in the Wild by Cross-Modal Interference Erasing
Specifically, we observe that the previous practice of learning only a single audio representation is insufficient due to the additive nature of audio signals.
Benchmarking and Analyzing Point Cloud Classification under Corruptions
3D perception, especially point cloud classification, has achieved substantial progress.
Ranked #7 on
Point Cloud Classification
on PointCloud-C
Full-Range Virtual Try-On With Recurrent Tri-Level Transform
Virtual try-on aims to transfer a target clothing image onto a reference person.
Ranked #4 on
Virtual Try-on
on VITON
Multi-View Partial (MVP) Point Cloud Challenge 2021 on Completion and Registration: Methods and Results
Based on the MVP dataset, this paper reports methods and results in the Multi-View Partial Point Cloud Challenge 2021 on Completion and Registration.
ForgeryNet -- Face Forgery Analysis Challenge 2021: Methods and Results
The rapid progress of photorealistic synthesis techniques has reached a critical point where the boundary between real and manipulated images starts to blur.
Garment4D: Garment Reconstruction from Point Cloud Sequences
The main challenges are two-fold: 1) effective 3D feature learning for fine details, and 2) capture of garment dynamics caused by the interaction between garments and the human body, especially for loose garments like skirts.
Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other.
Robust Partial-to-Partial Point Cloud Registration in a Full Range
\textbf{3)} Based on a synergy of hierarchical graph networks and graphical modeling, we propose the {H}ierarchical {G}raphical {M}odeling (\textbf{HGM}) architecture to encode robust descriptors consisting of i) a unary term learned from {\textit{RI}} features; and ii) multiple smoothness terms encoded from neighboring point relations at different scales through our TPT modules.
Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other.
Few-Shot Object Detection via Association and DIscrimination
1) In the association step, in contrast to implicitly leveraging multiple base classes, we construct a compact novel class feature space via explicitly imitating a specific base class feature space.
Lifting 2D Human Pose to 3D with Domain Adapted 3D Body Concept
Inspired by this, we propose a new framework that leverages the labeled 3D human poses to learn a 3D concept of the human body to reduce the ambiguity.
Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinements
In this paper, we make the first attempt to reconstruct 3D interacting hands from monocular single RGB images.
Generalized Out-of-Distribution Detection: A Survey
In this survey, we first present a unified framework called generalized OOD detection, which encompasses the five aforementioned problems, i. e., AD, ND, OSR, OOD detection, and OD.
Playing for 3D Human Recovery
Specifically, we contribute GTA-Human, a large-scale 3D human dataset generated with the GTA-V game engine, featuring a highly diverse set of subjects, actions, and scenarios.
TAda! Temporally-Adaptive Convolutions for Video Understanding
This work presents Temporally-Adaptive Convolutions (TAdaConv) for video understanding, which shows that adaptive weight calibration along the temporal dimension is an efficient way to facilitate modelling complex temporal dynamics in videos.
Ranked #59 on
Action Recognition
on Something-Something V2
(using extra training data)
A Comprehensive Overhaul of Distilling Unconditional GANs
To further enhance the semantic consistency between the teacher and student model, we present another latent-direction-based distillation loss that preserves the semantic relations in latent space.
Bayesian Imbalanced Regression Debiasing
Compared to imbalanced and long-tailed classification, imbalanced regression has its unique challenges as the regression label space can be continuous, boundless, and high-dimensional.
Talk-to-Edit: Fine-Grained Facial Editing via Dialog
In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system.
Ranked #1 on
Fine-Grained Facial Editing
on CelebA-Dialog
Learning to Prompt for Vision-Language Models
Large pre-trained vision-language models like CLIP have shown great potential in learning representations that are transferable across a wide range of downstream tasks.
Semantically Coherent Out-of-Distribution Detection
The proposed UDG can not only enrich the semantic knowledge of the model by exploiting unlabeled data in an unsupervised manner, but also distinguish ID/OOD samples to enhance ID classification and OOD detection tasks simultaneously.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Energy-Based Open-World Uncertainty Modeling for Confidence Calibration
Confidence calibration is of great importance to the reliability of decisions made by machine learning systems.
Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency
In addition, existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world.
