Search Results for author:
Found 23 papers, 6 papers with code
LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting
Our approach makes two key contributions: a data curation strategy from the StyleGAN-based relighting model for our training, and a modified diffusion-based ControlNet that processes both latent intrinsic properties from the source image and latent extrinsic properties from the target image.
SpotLight: Shadow-Guided Object Relighting via Diffusion
In this paper, we show that precise lighting control can be achieved for object relighting simply by specifying the desired shadows of the object.
ScribbleLight: Single Image Indoor Relighting with Scribbles
Our key technical novelty is an Albedo-conditioned Stable Image Diffusion model that preserves the intrinsic color and texture of the original image after relighting and an encoder-decoder-based ControlNet architecture that enables geometry-preserving lighting effects with normal map and scribble annotations.
ZeroComp: Zero-shot Object Compositing from Image Intrinsics via Diffusion
We present ZeroComp, an effective zero-shot 3D object compositing approach that does not require paired composite-scene images during training.
Latent Intrinsics Emerge from Training to Relight
Image relighting is the task of showing what a scene from a source image would look like if illuminated differently.
Videoshop: Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversion
We introduce Videoshop, a training-free video editing algorithm for localized semantic edits.
StyLitGAN: Image-Based Relighting via Latent Control
We describe a novel method StyLitGAN for relighting and resurfacing images in the absence of labeled data.
Generative Models: What Do They Know? Do They Know Things? Let's Find Out!
Generative models excel at mimicking real scenes, suggesting they might inherently encode important intrinsic scene properties.
Shadows Don't Lie and Lines Can't Bend! Generative Models don't know Projective Geometry...for now
All three classifiers are denied access to image pixels, and look only at derived geometric features.
Improving Equivariance in State-of-the-Art Supervised Depth and Normal Predictors
Dense depth and surface normal predictors should possess the equivariant property to cropping-and-resizing -- cropping the input image should result in cropping the same output image.
Blocks2World: Controlling Realistic Scenes with Editable Primitives
We present Blocks2World, a novel method for 3D scene rendering and editing that leverages a two-step process: convex decomposition of images and conditioned synthesis.
MIMIC: Masked Image Modeling with Image Correspondences
We train multiple models with different masked image modeling objectives to showcase the following findings: Representations trained on our automatically generated MIMIC-3M outperform those learned from expensive crowdsourced datasets (ImageNet-1K) and those learned from synthetic environments (MULTIVIEW-HABITAT) on two dense geometric tasks: depth estimation on NYUv2 (1. 7%), and surface normals estimation on Taskonomy (2. 05%).
UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video
We present UrbanIR (Urban Scene Inverse Rendering), a new inverse graphics model that enables realistic, free-viewpoint renderings of scenes under various lighting conditions with a single video.
Make It So: Steering StyleGAN for Any Image Inversion and Editing
StyleGAN's disentangled style representation enables powerful image editing by manipulating the latent variables, but accurately mapping real-world images to their latent variables (GAN inversion) remains a challenge.
StyLitGAN: Prompting StyleGAN to Produce New Illumination Conditions
We propose a novel method, StyLitGAN, for relighting and resurfacing generated images in the absence of labeled data.
SIRfyN: Single Image Relighting from your Neighbors
Novel theory shows that one can use similar scenes to estimate the different lightings that apply to a given scene, with bounded expected error.
DIVeR: Real-time and Accurate Neural Radiance Fields with Deterministic Integration for Volume Rendering
DIVeR's representation is a voxel based field of features.
View Generalization for Single Image Textured 3D Models
We describe a cycle consistency loss that encourages model textures to be aligned, so as to encourage sharing.
Cut-and-Paste Object Insertion by Enabling Deep Image Prior for Reshading
We show how to insert an object from one image to another and get realistic results in the hard case, where the shading of the inserted object clashes with the shading of the scene.
Improving Style Transfer with Calibrated Metrics
Style transfer methods produce a transferred image which is a rendering of a content image in the manner of a style image.
Unrestricted Adversarial Examples via Semantic Manipulation
Machine learning models, especially deep neural networks (DNNs), have been shown to be vulnerable against adversarial examples which are carefully crafted samples with a small magnitude of the perturbation.
Quantitative Evaluation of Style Transfer
Style transfer methods produce a transferred image which is a rendering of a content image in the manner of a style image.
Detecting Anomalous Faces with 'No Peeking' Autoencoders
We describe a method for detecting an anomalous face image that meets these requirements.
