Search Results for author:
Found 53 papers, 23 papers with code
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors
We present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors.
Diverse Image-to-Image Translation via Disentangled Representations
Our model takes the encoded content features extracted from a given input and the attribute vectors sampled from the attribute space to produce diverse outputs at test time.
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
In this work, we present an approach based on disentangled representation for generating diverse outputs without paired training images.
Dancing to Music
In the analysis phase, we decompose a dance into a series of basic dance units, through which the model learns how to move.
Ranked #3 on
Motion Synthesis
on BRACE
Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis
In this work, we propose a simple yet effective regularization term to address the mode collapse issue for cGANs.
Multimodal Unsupervised Image-To-Image Translation
Translation
StyleGAN of All Trades: Image Manipulation with Only Pretrained StyleGAN
Recently, StyleGAN has enabled various image manipulation and editing tasks thanks to the high-quality generation and the disentangled latent space.
SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation
To enable interactive generation, our method supports a variety of input modalities that can be easily provided by a human, including images, text, partially observed shapes and combinations of these, further allowing to adjust the strength of each input.
Text-Guided Synthesis of Eulerian Cinemagraphs
We introduce Text2Cinemagraph, a fully automated method for creating cinemagraphs from text descriptions - an especially challenging task when prompts feature imaginary elements and artistic styles, given the complexity of interpreting the semantics and motions of these images.
Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
Few-shot classification aims to recognize novel categories with only few labeled images in each class.
Ranked #6 on
Cross-Domain Few-Shot
on CUB
InfinityGAN: Towards Infinite-Pixel Image Synthesis
We present a novel framework, InfinityGAN, for arbitrary-sized image generation.
Ranked #2 on
Scene Generation
on OSM
Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning
In addition, our model can extract visual information as suggested by the text prompt, e. g., "an object in image one is moving northeast", and generate corresponding videos.
Unsupervised Representation Learning by Sorting Sequences
We present an unsupervised representation learning approach using videos without semantic labels.
Ranked #46 on
Self-Supervised Action Recognition
on HMDB51
Continuous and Diverse Image-to-Image Translation via Signed Attribute Vectors
Generating a smooth sequence of intermediate results bridges the gap of two different domains, facilitating the morphing effect across domains.
CrossDTR: Cross-view and Depth-guided Transformers for 3D Object Detection
To achieve accurate 3D object detection at a low cost for autonomous driving, many multi-camera methods have been proposed and solved the occlusion problem of monocular approaches.
ReDAL: Region-based and Diversity-aware Active Learning for Point Cloud Semantic Segmentation
Despite the success of deep learning on supervised point cloud semantic segmentation, obtaining large-scale point-by-point manual annotations is still a significant challenge.
Make-A-Story: Visual Memory Conditioned Consistent Story Generation
Our experiments for story generation on the MUGEN, the PororoSV and the FlintstonesSV dataset show that our method not only outperforms prior state-of-the-art in generating frames with high visual quality, which are consistent with the story, but also models appropriate correspondences between the characters and the background.
Exploiting Diffusion Prior for Generalizable Dense Prediction
Contents generated by recent advanced Text-to-Image (T2I) diffusion models are sometimes too imaginative for existing off-the-shelf dense predictors to estimate due to the immitigable domain gap.
Semantic View Synthesis
We tackle a new problem of semantic view synthesis -- generating free-viewpoint rendering of a synthesized scene using a semantic label map as input.
D2ADA: Dynamic Density-aware Active Domain Adaptation for Semantic Segmentation
In the field of domain adaptation, a trade-off exists between the model performance and the number of target domain annotations.
Learning Fine-Grained Visual Understanding for Video Question Answering via Decoupling Spatial-Temporal Modeling
While recent large-scale video-language pre-training made great progress in video question answering, the design of spatial modeling of video-language models is less fine-grained than that of image-language models; existing practices of temporal modeling also suffer from weak and noisy alignment between modalities.
Exploring Cross-Video and Cross-Modality Signals for Weakly-Supervised Audio-Visual Video Parsing
The audio-visual video parsing task aims to temporally parse a video into audio or visual event categories.
Coarse-to-Fine Point Cloud Registration with SE(3)-Equivariant Representations
As global feature alignment requires the features to preserve the poses of input point clouds and local feature matching expects the features to be invariant to these poses, we propose an SE(3)-equivariant feature extractor to simultaneously generate two types of features.
Unsupervised Discovery of Disentangled Manifolds in GANs
Interpretable generation process is beneficial to various image editing applications.
Learning Structured Semantic Embeddings for Visual Recognition
First, we exploit the discriminative constraints to capture the intra- and inter-class relationships of image embeddings.
Sub-GAN: An Unsupervised Generative Model via Subspaces
The recent years have witnessed significant growth in constructing robust generative models to capture informative distributions of natural data.
Soft-Segmentation Guided Object Motion Deblurring
To address these problems, we propose a novel model for object motion deblurring.
Self-supervised Audio Spatialization with Correspondence Classifier
Spatial audio is an essential medium to audiences for 3D visual and auditory experience.
Neural Design Network: Graphic Layout Generation with Constraints
The first module predicts a graph with complete relations from a graph with user-specified relations.
Large Margin Mechanism and Pseudo Query Set on Cross-Domain Few-Shot Learning
In recent years, few-shot learning problems have received a lot of attention.
RetrieveGAN: Image Synthesis via Differentiable Patch Retrieval
Image generation from scene description is a cornerstone technique for the controlled generation, which is beneficial to applications such as content creation and image editing.
Controllable Image Synthesis via SegVAE
We also apply an off-the-shelf image-to-image translation model to generate realistic RGB images to better understand the quality of the synthesized semantic maps.
In&Out : Diverse Image Outpainting via GAN Inversion
Existing image outpainting methods pose the problem as a conditional image-to-image translation task, often generating repetitive structures and textures by replicating the content available in the input image.
Unsupervised Sound Localization via Iterative Contrastive Learning
Sound localization aims to find the source of the audio signal in the visual scene.
Unveiling The Mask of Position-Information Pattern Through the Mist of Image Features
Recent studies show that paddings in convolutional neural networks encode absolute position information which can negatively affect the model performance for certain tasks.
InOut: Diverse Image Outpainting via GAN Inversion
Existing image outpainting methods pose the problem as a conditional image-to-image translation task, often generating repetitive structures and textures by replicating the content available in the input image.
Cross-Modal 3D Shape Generation and Manipulation
In this paper, we propose a generic multi-modal generative model that couples the 2D modalities and implicit 3D representations through shared latent spaces.
Vector Quantized Image-to-Image Translation
For example, it provides style variability for image generation and extension, and equips image-to-image translation with further extension capabilities.
Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion Model
Generating images from hand-drawings is a crucial and fundamental task in content creation.
ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenes
The two popular datasets ScanRefer [16] and ReferIt3D [3] connect natural language to real-world 3D data.
DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesis
Existing 3D-aware image synthesis approaches mainly focus on generating a single canonical object and show limited capacity in composing a complex scene containing a variety of objects.
3DAvatarGAN: Bridging Domains for Personalized Editable Avatars
Finally, we propose a novel inversion method for 3D-GANs linking the latent spaces of the source and the target domains.
InfiniCity: Infinite-Scale City Synthesis
Toward infinite-scale 3D city synthesis, we propose a novel framework, InfiniCity, which constructs and renders an unconstrainedly large and 3D-grounded environment from random noises.
Unsupervised Volumetric Animation
We propose a novel approach for unsupervised 3D animation of non-rigid deformable objects.
3D generation on ImageNet
Existing 3D-from-2D generators are typically designed for well-curated single-category datasets, where all the objects have (approximately) the same scale, 3D location, and orientation, and the camera always points to the center of the scene.
Text2Tex: Text-driven Texture Synthesis via Diffusion Models
We present Text2Tex, a novel method for generating high-quality textures for 3D meshes from the given text prompts.
SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors
We propose SceneTex, a novel method for effectively generating high-quality and style-consistent textures for indoor scenes using depth-to-image diffusion priors.
UpFusion: Novel View Diffusion from Unposed Sparse View Observations
We propose UpFusion, a system that can perform novel view synthesis and infer 3D representations for an object given a sparse set of reference images without corresponding pose information.
SceneWiz3D: Towards Text-guided 3D Scene Composition
We are witnessing significant breakthroughs in the technology for generating 3D objects from text.
Virtual Pets: Animatable Animal Generation in 3D Scenes
Toward unlocking the potential of generative models in immersive 4D experiences, we introduce Virtual Pet, a novel pipeline to model realistic and diverse motions for target animal species within a 3D environment.
Diffusion Priors for Dynamic View Synthesis from Monocular Videos
Dynamic novel view synthesis aims to capture the temporal evolution of visual content within videos.
AToM: Amortized Text-to-Mesh using 2D Diffusion
We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously.
Visual Concept-driven Image Generation with Text-to-Image Diffusion Model
We illustrate that such joint alternating refinement leads to the learning of better tokens for concepts and, as a bi-product, latent masks.
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers
Next, we finetune a retrieval model on a small subset where the best caption of each video is manually selected and then employ the model in the whole dataset to select the best caption as the annotation.
