Search Results for author:
Found 36 papers, 7 papers with code
How Animals Dance (When You're Not Looking)
We present a keyframe-based framework for generating music-synchronized, choreography aware animal dance videos.
Bolt3D: Generating 3D Scenes in Seconds
We present a latent diffusion model for fast feed-forward 3D scene generation.
Continuous 3D Perception Model with Persistent State
We present a unified framework capable of solving a broad range of 3D tasks.
GPS as a Control Signal for Image Generation
We show that the GPS tags contained in photo metadata provide a useful control signal for image generation.
MegaSaM: Accurate, Fast and Robust Structure and Motion from Casual Dynamic Videos
We present a system that allows for accurate, fast, and robust estimation of camera parameters and depth maps from casual monocular videos of dynamic scenes.
Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos
Learning to understand dynamic 3D scenes from imagery is crucial for applications ranging from robotics to scene reconstruction.
SimVS: Simulating World Inconsistencies for Robust View Synthesis
Novel-view synthesis techniques achieve impressive results for static scenes but struggle when faced with the inconsistencies inherent to casual capture settings: varying illumination, scene motion, and other unintended effects that are difficult to model explicitly.
MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videos
We present a system that allows for accurate, fast, and robust estimation of camera parameters and depth maps from casual monocular videos of dynamic scenes.
CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
We present CAT4D, a method for creating 4D (dynamic 3D) scenes from monocular video.
VidPanos: Generative Panoramic Videos from Casual Panning Videos
Panoramic image stitching provides a unified, wide-angle view of a scene that extends beyond the camera's field of view.
Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation
We adapt a pretrained large-scale image-to-video diffusion model (originally trained to generate videos moving forward in time from a single input image) for key frame interpolation, i. e., to produce a video in between two input frames.
Diffusion Models as Data Mining Tools
Concretely, we show that after finetuning conditional diffusion models to synthesize images from a specific dataset, we can use these models to define a typicality measure on that dataset.
Rethinking Score Distillation as a Bridge Between Image Distributions
We compare our method to existing approaches for score distillation sampling and show that it can produce high-frequency details with realistic colors.
ExtraNeRF: Visibility-Aware View Extrapolation of Neural Radiance Fields with Diffusion Models
We propose ExtraNeRF, a novel method for extrapolating the range of views handled by a Neural Radiance Field (NeRF).
CAT3D: Create Anything in 3D with Multi-View Diffusion Models
Advances in 3D reconstruction have enabled high-quality 3D capture, but require a user to collect hundreds to thousands of images to create a 3D scene.
Infinite Texture: Text-guided High Resolution Diffusion Texture Synthesis
We present Infinite Texture, a method for generating arbitrarily large texture images from a text prompt.
Disentangled 3D Scene Generation with Layout Learning
We introduce a method to generate 3D scenes that are disentangled into their component objects.
ReconFusion: 3D Reconstruction with Diffusion Priors
3D reconstruction methods such as Neural Radiance Fields (NeRFs) excel at rendering photorealistic novel views of complex scenes.
Readout Guidance: Learning Control from Diffusion Features
We present Readout Guidance, a method for controlling text-to-image diffusion models with learned signals.
Generative Powers of Ten
We present a method that uses a text-to-image model to generate consistent content across multiple image scales, enabling extreme semantic zooms into a scene, e. g., ranging from a wide-angle landscape view of a forest to a macro shot of an insect sitting on one of the tree branches.
State of the Art on Diffusion Models for Visual Computing
The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes.
RealFill: Reference-Driven Generation for Authentic Image Completion
Once personalized, RealFill is able to complete a target image with visually compelling contents that are faithful to the original scene.
Tracking Everything Everywhere All at Once
We present a new test-time optimization method for estimating dense and long-range motion from a video sequence.
Diffusion Self-Guidance for Controllable Image Generation
However, many aspects of an image are difficult or impossible to convey through text.
Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence
We propose Diffusion Hyperfeatures, a framework for consolidating multi-scale and multi-timestep feature maps into per-pixel feature descriptors that can be used for downstream tasks.
Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFs
Casually captured Neural Radiance Fields (NeRFs) suffer from artifacts such as floaters or flawed geometry when rendered outside the camera trajectory.
DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion
We fine-tune on a collection of fashion videos from the UBC Fashion dataset.
Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions
We propose a method for editing NeRF scenes with text-instructions.
DreamPose: Fashion Video Synthesis with Stable Diffusion
We fine-tune on a collection of fashion videos from the UBC Fashion dataset.
InstructPix2Pix: Learning to Follow Image Editing Instructions
We propose a method for editing images from human instructions: given an input image and a written instruction that tells the model what to do, our model follows these instructions to edit the image.
Ranked #7 on
Image Editing
on ImgEdit-Data
SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stage
Our method only requires the user to capture a selfie video outdoors, rotating in place, and uses the varying angles between the sun and the face as guidance in joint reconstruction of facial geometry, reflectance, camera pose, and lighting parameters.
Animating Pictures with Eulerian Motion Fields
In this paper, we demonstrate a fully automatic method for converting a still image into a realistic animated looping video.
Reducing Drift in Structure From Motion Using Extended Features
Low-frequency long-range errors (drift) are an endemic problem in 3D structure from motion, and can often hamper reasonable reconstructions of the scene.
Seeing the World in a Bag of Chips
We address the dual problems of novel view synthesis and environment reconstruction from hand-held RGBD sensors.
Structure from Motion for Panorama-Style Videos
We present a novel Structure from Motion pipeline that is capable of reconstructing accurate camera poses for panorama-style video capture without prior camera intrinsic calibration.
