Search Results for author:
Found 31 papers, 12 papers with code
Probing the 3D Awareness of Visual Foundation Models
Given that such models can classify, delineate, and localize objects in 2D, we ask whether they also represent their 3D structure?
Lumiere: A Space-Time Diffusion Model for Video Generation
We introduce Lumiere -- a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion -- a pivotal challenge in video synthesis.
Ranked #6 on
Text-to-Video Generation
on UCF-101
WonderJourney: Going from Anywhere to Everywhere
We introduce WonderJourney, a modularized framework for perpetual 3D scene generation.
Idempotent Generative Network
We define the target manifold as the set of all instances that $f$ maps to themselves.
RealFill: Reference-Driven Generation for Authentic Image Completion
Once personalized, RealFill is able to complete a target image with visually compelling contents that are faithful to the original scene.
3D Motion Magnification: Visualizing Subtle Motions with Time Varying Radiance Fields
Motion magnification helps us visualize subtle, imperceptible motion.
HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models
By composing these weights into the diffusion model, coupled with fast finetuning, HyperDreamBooth can generate a person's face in various contexts and styles, with high subject details while also preserving the model's crucial knowledge of diverse styles and semantic modifications.
Background Prompting for Improved Object Depth
To infer object depth on a real image, we place the segmented object into the learned background prompt and run off-the-shelf depth networks.
StyleDrop: Text-to-Image Generation in Any Style
Pre-trained large text-to-image models synthesize impressive images with an appropriate use of text prompts.
Score-Based Diffusion Models as Principled Priors for Inverse Imaging
In this work, we empirically validate the theoretically-proven probability function of a score-based diffusion model.
DreamBooth3D: Subject-Driven Text-to-3D Generation
We present DreamBooth3D, an approach to personalize text-to-3D generative models from as few as 3-6 casually captured images of a subject.
Muse: Text-To-Image Generation via Masked Generative Transformers
Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.
Ranked #1 on
Text-to-Image Generation
on MS-COCO
(FID metric)
3D Motion Magnification: Visualizing Subtle Motions from Time-Varying Radiance Fields
Motion magnification helps us visualize subtle, imperceptible motion.
Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemble
Automatically estimating 3D skeleton, shape, camera viewpoints, and part articulation from sparse in-the-wild image ensembles is a severely under-constrained and challenging problem.
SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flow
Scene flow estimation is a long-standing problem in computer vision, where the goal is to find the 3D motion of a scene from its consecutive observations.
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
Once the subject is embedded in the output domain of the model, the unique identifier can be used to synthesize novel photorealistic images of the subject contextualized in different scenes.
TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliency
In this work, we focus on summarizing instructional videos, an under-explored area of video summarization.
LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery
In this work, we propose a practical problem setting to estimate 3D pose and shape of animals given only a few (10-30) in-the-wild images of a particular animal species (say, horse).
Disentangling Architecture and Training for Optical Flow
Our newly trained RAFT achieves an Fl-all score of 4. 31% on KITTI 2015, more accurate than all published optical flow methods at the time of writing.
Deep Saliency Prior for Reducing Visual Distraction
Using only a model that was trained to predict where people look at images, and no additional training data, we can produce a range of powerful editing effects for reducing distraction in images.
Omnimatte: Associating Objects and Their Effects in Video
We show results on real-world videos containing interactions between different types of subjects (cars, animals, people) and complex effects, ranging from semi-transparent elements such as smoke and reflections, to fully opaque effects such as objects attached to the subject.
Layered Neural Rendering for Retiming People in Video
We present a method for retiming people in an ordinary, natural video -- manipulating and editing the time in which different motions of individuals in the video occur.
SpeedNet: Learning the Speediness in Videos
We demonstrate how those learned features can boost the performance of self-supervised action recognition, and can be used for video retrieval.
Speech2Face: Learning the Face Behind a Voice
How much can we infer about a person's looks from the way they speak?
Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation
Solving this task using only audio as input is extremely challenging and does not provide an association of the separated speech signals with speakers in the video.
On the Effectiveness of Visible Watermarks
Since such an attack relies on the consistency of watermarks across image collection, we explore and evaluate how it is affected by various types of inconsistencies in the watermark embedding that could potentially be used to make watermarking more secured.
Visual Vibrometry: Estimating MaterialProperties from Small Motions in Video
The estimation of material properties is important for scene understanding, with many applications in vision, robotics, andstructural engineering.
Visual Vibrometry: Estimating Material Properties from Small Motions in Video
The estimation of material properties is important for scene understanding, with many applications in vision, robotics, and structural engineering.
Visual Vibrometry: Estimating Material Properties From Small Motion in Video
The estimation of material properties is important for scene understanding, with many applications in vision, robotics, and structural engineering.
Best-Buddies Similarity for Robust Template Matching
We propose a novel method for template matching in unconstrained environments.
Unsupervised Joint Object Discovery and Segmentation in Internet Images
In contrast to previous co-segmentation methods, our algorithm performs well even in the presence of significant amounts of noise images (images not containing a common object), as typical for datasets collected from Internet search.
