Search Results for author:
Found 24 papers, 10 papers with code
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
Once the subject is embedded in the output domain of the model, the unique identifier can be used to synthesize novel photorealistic images of the subject contextualized in different scenes.
Muse: Text-To-Image Generation via Masked Generative Transformers
Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.
Ranked #1 on
Text-to-Image Generation
on MS-COCO
(FID metric)
StyleDrop: Text-to-Image Generation in Any Style
Pre-trained large text-to-image models synthesize impressive images with an appropriate use of text prompts.
ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAs
Experiments on a wide range of subject and style combinations show that ZipLoRA can generate compelling results with meaningful improvements over baselines in subject and style fidelity while preserving the ability to recontextualize.
Probing the 3D Awareness of Visual Foundation Models
Given that such models can classify, delineate, and localize objects in 2D, we ask whether they also represent their 3D structure?
HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models
By composing these weights into the diffusion model, coupled with fast finetuning, HyperDreamBooth can generate a person's face in various contexts and styles, with high subject details while also preserving the model's crucial knowledge of diverse styles and semantic modifications.
SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections
Our method works on in-the-wild online image collections of an object and produces relightable 3D assets for several use-cases such as AR/VR.
Simplified Transfer Learning for Chest Radiography Models Using Less Data
Supervised contrastive learning enabled performance comparable to state-of-the-art deep learning models in multiple clinical tasks by using as few as 45 images and is a promising method for predictive modeling with use of small data sets and for predicting outcomes in shifting patient populations.
Debiasing Vision-Language Models via Biased Prompts
Machine learning models have been shown to inherit biases from their training datasets.
Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemble
Automatically estimating 3D skeleton, shape, camera viewpoints, and part articulation from sparse in-the-wild image ensembles is a severely under-constrained and challenging problem.
A mullti- or many- objective evolutionary algorithm with global loop update
The decomposition-based MOEAs emphasize convergence and diversity in a simple model and have made a great success in dealing with theoretical and practical multi- or many-objective optimization problems.
A Many-Objective Evolutionary Algorithm Based on Decomposition and Local Dominance
Recent studies show that a well designed combination of the decomposition method and the domination method can improve the performance , i. e., convergence and diversity, of a MOEA.
Deep Image-based Illumination Harmonization
Specifically, we firstly apply a physically-based rendering method to construct a large-scale, high-quality dataset (named IH) for our task, which contains various types of foreground objects and background scenes with different lighting conditions.
LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery
In this work, we propose a practical problem setting to estimate 3D pose and shape of animals given only a few (10-30) in-the-wild images of a particular animal species (say, horse).
MetaCLUE: Towards Comprehensive Visual Metaphors Research
Towards this goal, we introduce MetaCLUE, a set of vision tasks on visual metaphor.
StraIT: Non-autoregressive Generation with Stratified Image Transformer
We propose Stratified Image Transformer(StraIT), a pure non-autoregressive(NAR) generative model that demonstrates superiority in high-quality image synthesis over existing autoregressive(AR) and diffusion models(DMs).
DreamBooth3D: Subject-Driven Text-to-3D Generation
We present DreamBooth3D, an approach to personalize text-to-3D generative models from as few as 3-6 casually captured images of a subject.
ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistency
We present ShapeClipper, a novel method that reconstructs 3D object shapes from real-world single-view RGB images.
Background Prompting for Improved Object Depth
To infer object depth on a real image, we place the segmented object into the learned background prompt and run off-the-shelf depth networks.
RealFill: Reference-Driven Generation for Authentic Image Completion
Once personalized, RealFill is able to complete a target image with visually compelling contents that are faithful to the original scene.
Alchemist: Parametric Control of Material Properties with Diffusion Models
We propose a method to control material attributes of objects like roughness, metallic, albedo, and transparency in real images.
SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild
We present SHINOBI, an end-to-end framework for the reconstruction of shape, material, and illumination from object images captured with varying lighting, pose, and background.
Lumiere: A Space-Time Diffusion Model for Video Generation
We introduce Lumiere -- a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion -- a pivotal challenge in video synthesis.
Ranked #6 on
Text-to-Video Generation
on UCF-101
3D Congealing: 3D-Aware Image Alignment in the Wild
The framework optimizes for the canonical representation together with the pose for each input image, and a per-image coordinate map that warps 2D pixel coordinates to the 3D canonical frame to account for the shape matching.
