Search Results for author:
Found 79 papers, 45 papers with code
RaFE: Generative Radiance Fields Restoration
Instead of reconstructing a blurred NeRF by averaging inconsistencies, we introduce a novel approach using Generative Adversarial Networks (GANs) for NeRF generation to better accommodate the geometric and appearance inconsistencies present in the multi-view images.
Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion
In practice, users often desire the ability to control object motion and camera movement independently for customized video creation.
Advances in 3D Generation: A Survey
In this survey, we aim to introduce the fundamental methodologies of 3D generation methods and establish a structured roadmap, encompassing 3D representation, generation methods, datasets, and corresponding applications.
StyleRetoucher: Generalized Portrait Image Retouching with GAN Priors
To address such issues, we present StyleRetoucher, a novel automatic portrait image retouching framework, leveraging StyleGAN's generation and generalization ability to improve an input portrait image's skin condition while preserving its facial details.
HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillation
The noisy image, landmarks, and text condition are then fed into the frozen ControlNet twice for noise prediction.
CAD: Photorealistic 3D Generation via Adversarial Distillation
The increased demand for 3D data in AR/VR, robotics and gaming applications, gave rise to powerful generative pipelines capable of synthesizing high-quality 3D objects.
Mesh-Guided Neural Implicit Field Editing
We first introduce a differentiable method using marching tetrahedra for polygonal mesh extraction from the neural implicit field and then design a differentiable color extractor to assign colors obtained from the volume renderings to this extracted mesh.
HumanRef: Single Image to 3D Human Generation via Reference-Guided Diffusion
Optimization-based methods that lift text-to-image diffusion models to 3D generation often fail to preserve the texture details of the reference image, resulting in inconsistent appearances in different views.
VQ-NeRF: Neural Reflectance Decomposition and Editing with Vector Quantization
By discretizing the materials, our model can reduce noise in the decomposition process and generate a segmentation map of discrete materials.
HairCLIPv2: Unifying Hair Editing via Proxy Feature Blending
Even though they can enable very fine-grained local control, such interaction modes are inefficient for the editing conditions that can be easily specified by language descriptions or reference images.
Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Model
Recently, text-to-image denoising diffusion probabilistic models (DDPMs) have demonstrated impressive image generation capabilities and have also been successfully applied to image inpainting.
Text-Guided Vector Graphics Customization
In this paper, we propose a novel pipeline that generates high-quality customized vector graphics based on textual prompts while preserving the properties and layer-wise information of a given exemplar SVG.
Preserving Tumor Volumes for Unsupervised Medical Image Registration
Our approach balances image similarity and volume preservation in different regions, i. e., normal and tumor regions, by using soft tumor masks to adjust the imposition of volume-preserving loss on each one.
Continuous Layout Editing of Single Images with Diffusion Models
However, none of these methods have been able to edit the layout of single existing images.
Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields
Extensive experiments demonstrate that our Text2NeRF outperforms existing methods in producing photo-realistic, multi-view consistent, and diverse 3D scenes from a variety of natural language prompts.
IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformers
More importantly, we demonstrate the flexibility of IconShop with multiple novel icon synthesis tasks, including icon editing, icon interpolation, icon semantic combination, and icon design auto-suggestion.
Learning Neural Duplex Radiance Fields for Real-Time View Synthesis
Neural radiance fields (NeRFs) enable novel view synthesis with unprecedented visual quality.
AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control
Neural implicit fields are powerful for representing 3D scenes and generating high-quality novel views, but it remains challenging to use such implicit representations for creating a 3D human avatar with a specific identity and artistic style that can be easily animated.
3D Video Loops from Asynchronous Input
Looping videos are short video clips that can be looped endlessly without visible seams or artifacts.
Weakly-Supervised 3D Medical Image Segmentation using Geometric Prior and Contrastive Similarity
Furthermore, we propose contrastive similarity to encourage organ pixels to gather around in the contrastive embedding space, which helps better distinguish low-contrast tissues.
NeRF-Art: Text-Driven Neural Radiance Fields Stylization
As a powerful representation of 3D scenes, the neural radiance field (NeRF) enables high-quality novel view synthesis from multi-view images.
Robust Point Cloud Segmentation with Noisy Annotations
To handle boundary-level label noise, we also propose a variant ``PNAL-boundary " with a progressive boundary label cleaning strategy.
Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles
Through our analysis, we find one important reason is that existing large-scale VL datasets do not contain much commonsense knowledge, which motivates us to improve the commonsense of VL-models from the data perspective.
Water Simulation and Rendering from a Still Photograph
We propose an approach to simulate and render realistic water animation from a single still input photograph.
UniColor: A Unified Framework for Multi-Modal Colorization with Transformer
In the first stage, multi-modal conditions are converted into a common representation of hint points.
DifferSketching: How Differently Do People Sketch 3D Objects?
We invited 70 novice users and 38 expert users to sketch 136 3D objects, which were presented as 362 images rendered from multiple views.
Adaptive Joint Optimization for 3D Reconstruction with Differentiable Rendering
Due to inevitable noises introduced during scanning and quantization, 3D reconstruction via RGB-D sensors suffers from errors both in geometry and texture, leading to artifacts such as camera drifting, mesh distortion, texture ghosting, and blurriness.
FDNeRF: Few-shot Dynamic Neural Radiance Fields for Face Reconstruction and Expression Editing
Unlike existing dynamic NeRFs that require dense images as input and can only be modeled for a single identity, our method enables face reconstruction across different persons with few-shot inputs.
Neural Parameterization for Dynamic Human Head Editing
Implicit radiance functions emerged as a powerful scene representation for reconstructing and rendering photo-realistic views of a 3D scene.
Bringing Old Films Back to Life
We present a learning-based framework, recurrent transformer network (RTN), to restore heavily degraded old films.
Ranked #6 on
Analog Video Restoration
on TAPE
3D Question Answering
To this end, we propose a novel transformer-based 3DQA framework "3DQA-TR", which consists of two encoders for exploiting the appearance and geometry information, respectively.
HairCLIP: Design Your Hair by Text and Reference Image
Hair editing is an interesting and challenging problem in computer vision and graphics.
CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields
Furthermore, we propose an inverse optimization method that accurately projects an input image to the latent codes for manipulation to enable editing on real images.
Deblur-NeRF: Neural Radiance Fields from Blurry Images
We demonstrate that our method can be used on both camera motion blur and defocus blur: the two most common types of blur in real scenes.
Poison Ink: Robust and Invisible Backdoor Attack
As the image structure can keep its semantic meaning during the data transformation, such trigger pattern is inherently robust to data transformations.
Exploring Structure Consistency for Deep Model Watermarking
However, little attention has been devoted to the protection of DNNs in image processing tasks.
Deep Portrait Lighting Enhancement with 3D Guidance
To address this, we present a novel deep learning framework for portrait lighting enhancement based on 3D facial guidance.
Learning with Noisy Labels for Robust Point Cloud Segmentation
Point cloud segmentation is a fundamental task in 3D.
PD-GAN: Probabilistic Diverse GAN for Image Inpainting
To this end, we propose spatially probabilistic diversity normalization (SPDNorm) inside the modulation to model the probability of generating a pixel conditioned on the context information.
Exemplar-Based 3D Portrait Stylization
In the second texture style transfer stage, we focus on performing style transfer on the canonical texture by adopting a differentiable renderer to optimize the texture in a multi-view framework.
Cross-Domain and Disentangled Face Manipulation with 3D Guidance
Face image manipulation via three-dimensional guidance has been widely applied in various interactive scenarios due to its semantically-meaningful understanding and user-friendly controllability.
Let's See Clearly: Contaminant Artifact Removal for Moving Cameras
This new dataset and our novel framework lead to our method that is able to address different contaminants and outperforms competitive restoration approaches both qualitatively and quantitatively.
E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion
This paper studies the problem of StyleGAN inversion, which plays an essential role in enabling the pretrained StyleGAN to be used for real image editing tasks.
High-Fidelity Pluralistic Image Completion with Transformers
Image completion has made tremendous progress with convolutional neural networks (CNNs), because of their powerful texture modeling capacity.
Ranked #6 on
Image Inpainting
on CelebA-HQ
DeFLOCNet: Deep Image Editing via Flexible Low-level Controls
While existing methods combine an input image and these low-level controls for CNN inputs, the corresponding feature representations are not sufficient to convey user intentions, leading to unfaithfully generated content.
Diverse Semantic Image Synthesis via Probability Distribution Modeling
In this paper, we propose a novel diverse semantic image synthesis framework from the perspective of semantic class distributions, which naturally supports diverse generation at semantic or even instance level.
Ranked #1 on
Image-to-Image Translation
on ADE20K Labels-to-Photos
(LPIPS metric)
Deep Model Intellectual Property Protection via Deep Watermarking
By jointly training the target model and watermark embedding, the extra barrier can even be absorbed into the target model.
Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud
Thus, Meta-PU even outperforms the existing methods trained for a specific scale factor only.
Graphics
Improved Image Matting via Real-time User Clicks and Uncertainty Estimation
Image matting is a fundamental and challenging problem in computer vision and graphics.
Efficient Semantic Image Synthesis via Class-Adaptive Normalization
Spatially-adaptive normalization (SPADE) is remarkably successful recently in conditional semantic image synthesis \cite{park2019semantic}, which modulates the normalized activation with spatially-varying transformations learned from semantic layouts, to prevent the semantic information from being washed away.
LG-GAN: Label Guided Adversarial Network for Flexible Targeted Attack of Point Cloud-based Deep Networks
To overcome these shortcomings, this paper proposes a novel label guided adversarial network (LG-GAN) for real-time flexible targeted point cloud attack.
MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing
In this paper, we present MichiGAN (Multi-Input-Conditioned Hair Image GAN), a novel conditional image generation method for interactive portrait hair manipulation.
Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search
The experiments on ImageNet verify such path distillation method can improve the convergence ratio and performance of the hypernetwork, as well as boosting the training of subnetworks.
Passport-aware Normalization for Deep Model Protection
Only when the model IP is suspected to be stolen by someone, the private passport-aware branch is added back for ownership verification.
Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network
We proposed a new I2I translation method that generates a new model in the target domain via a series of model transformations on a pre-trained StyleGAN2 model in the source domain.
Old Photo Restoration via Deep Latent Space Translation
Unlike conventional restoration tasks that can be solved through supervised learning, the degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize.
Deep Sketch-guided Cartoon Video Inbetweening
The key idea of the proposed approach is to estimate the dense cross-domain correspondence between the sketch and cartoon video frames, and employ a blending module with occlusion estimation to synthesize the middle frame guided by the sketch.
Multi-Density Sketch-to-Image Translation Network
Sketch-to-image (S2I) translation plays an important role in image synthesis and manipulation tasks, such as photo editing and colorization.
Bringing Old Photos Back to Life
Unlike conventional restoration tasks that can be solved through supervised learning, the degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize.
Rethinking Spatially-Adaptive Normalization
Despite its impressive performance, a more thorough understanding of the true advantages inside the box is still highly demanded, to help reduce the significant computation and parameter overheads introduced by these new structures.
Application of Structural Similarity Analysis of Visually Salient Areas and Hierarchical Clustering in the Screening of Similar Wireless Capsule Endoscopic Images
Small intestinal capsule endoscopy is the mainstream method for inspecting small intestinal lesions, but a single small intestinal capsule endoscopy will produce 60, 000 - 120, 000 images, the majority of which are similar and have no diagnostic value.
Model Watermarking for Image Processing Networks
In this way, when the attacker trains one surrogate model by using the input-output pairs of the target model, the hidden watermark will be learned and extracted afterward.
Style Mixer: Semantic-aware Multi-Style Transfer Network
Recent neural style transfer frameworks have obtained astonishing visual quality and flexibility in Single-style Transfer (SST), but little attention has been paid to Multi-style Transfer (MST) which refers to simultaneously transferring multiple styles to the same image.
Semantic Example Guided Image-to-Image Translation
Given a reference image and an input in another domain, a semantic matching is first performed between the two visual contents and generates the auxiliary image, which is explicitly encouraged to preserve semantic characteristics of the reference.
Document Rectification and Illumination Correction using a Patch-based CNN
We propose a novel learning method to rectify document images with various distortion types from a single input image.
Blind Geometric Distortion Correction on Images Through Deep Learning
We propose the first general framework to automatically correct different types of geometric distortion in a single input image.
Deep Exemplar-based Video Colorization
This paper presents the first end-to-end network for exemplar-based video colorization.
Transductive Zero-Shot Learning with Visual Structure Constraint
Based on the observation that visual features of test instances can be separated into different clusters, we propose a new visual structure constraint on class centers for transductive ZSL, to improve the generality of the projection function (i. e. alleviate the above domain shift problem).
Gated Context Aggregation Network for Image Dehazing and Deraining
Image dehazing aims to recover the uncorrupted content from a hazy image.
Ranked #1 on
Rain Removal
on DID-MDN
CariGANs: Unpaired Photo-to-Caricature Translation
Facial caricature is an art form of drawing faces in an exaggerated way to convey humor or sarcasm.
Deep Video-Based Performance Cloning
After training a deep generative network using a reference video capturing the appearance and dynamics of a target actor, we are able to generate videos where this actor reenacts other performances.
Deep Exemplar-based Colorization
More importantly, as opposed to other learning-based colorization methods, our network allows the user to achieve customizable results by simply feeding different references.
Arbitrary Style Transfer with Deep Feature Reshuffle
We theoretically prove that our new style loss based on reshuffle connects both global and local style losses respectively used by most parametric and non-parametric neural style transfer methods.
Neural Best-Buddies: Sparse Cross-Domain Correspondence
Correspondence between images is a fundamental problem in computer vision, with a variety of graphics applications.
Stereoscopic Neural Style Transfer
This paper presents the first attempt at stereoscopic neural style transfer, which responds to the emerging demand for 3D movies or AR/VR.
Progressive Color Transfer with Dense Semantic Correspondences
The proposed method can be successfully extended from one-to-one to one-to-many color transfer.
Visual Attribute Transfer through Deep Image Analogy
We propose a new technique for visual attribute transfer across images that may have very different appearance but have perceptually similar semantic structure.
Coherent Online Video Style Transfer
Training a feed-forward network for fast neural style transfer of images is proven to be successful.
StyleBank: An Explicit Representation for Neural Image Style Transfer
It also enables us to conduct incremental learning to add a new image style by learning a new filter bank while holding the auto-encoder fixed.
