Search Results for author:
Found 23 papers, 17 papers with code
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion
In this work, we propose SV3D that adapts image-to-video diffusion model for novel multi-view synthesis and 3D generation, thereby leveraging the generalization and multi-view consistency of the video models, while further adding explicit camera control for NVS.
Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation
Distillation methods, like the recently introduced adversarial diffusion distillation (ADD) aim to shift the model from many-shot to single-step inference, albeit at the cost of expensive and difficult optimization due to its reliance on a fixed pretrained DINOv2 discriminator.
Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Rectified flow is a recent generative model formulation that connects data and noise in a straight line.
aMUSEd: An Open MUSE Reproduction
We present aMUSEd, an open-source, lightweight masked image model (MIM) for text-to-image generation based on MUSE.
DiffusionSat: A Generative Foundation Model for Satellite Imagery
Our method outperforms previous state-of-the-art methods for satellite image generation and is the first large-scale $\textit{generative}$ foundation model for satellite imagery.
Adversarial Diffusion Distillation
We introduce Adversarial Diffusion Distillation (ADD), a novel training approach that efficiently samples large-scale foundational image diffusion models in just 1-4 steps while maintaining high image quality.
Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
We then explore the impact of finetuning our base model on high-quality data and train a text-to-video model that is competitive with closed-source video generation.
SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
We present SDXL, a latent diffusion model for text-to-image synthesis.
NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models
We first train a scene auto-encoder to express a set of image and pose pairs as a neural field, represented as density and feature voxel grids that can be projected to produce novel views of the scene.
Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
We first pre-train an LDM on images only; then, we turn the image generator into a video generator by introducing a temporal dimension to the latent space diffusion model and fine-tuning on encoded image sequences, i. e., videos.
Ranked #5 on
Text-to-Video Generation
on MSR-VTT
(CLIP-FID metric)
On Distillation of Guided Diffusion Models
For standard diffusion models trained on the pixel-space, our approach is able to generate images visually comparable to that of the original model using as few as 4 sampling steps on ImageNet 64x64 and CIFAR-10, achieving FID/IS scores comparable to that of the original model while being up to 256 times faster to sample from.
Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models
In RDMs, a set of nearest neighbors is retrieved from an external database during training for each training instance, and the diffusion model is conditioned on these informative samples.
Semi-Parametric Neural Image Synthesis
Much of this success is due to the scalability of these architectures and hence caused by a dramatic increase in model complexity and in the computational resources invested in training these models.
High-Resolution Image Synthesis with Latent Diffusion Models
By decomposing the image formation process into a sequential application of denoising autoencoders, diffusion models (DMs) achieve state-of-the-art synthesis results on image data and beyond.
Ranked #2 on
Layout-to-Image Generation
on COCO-Stuff 256x256
ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis
Thus, in contrast to pure autoregressive models, it can solve free-form image inpainting and, in the case of conditional models, local, text-guided image modification without requiring mask-specific training.
Ranked #4 on
Text-to-Image Generation
on Conceptual Captions
High-Resolution Complex Scene Synthesis with Transformers
The use of coarse-grained layouts for controllable synthesis of complex scene images via deep generative models has recently gained popularity.
Stochastic Image-to-Video Synthesis using cINNs
Video understanding calls for a model to learn the characteristic interplay between static scene content and its dynamics: Given an image, the model must be able to predict a future progression of the portrayed scene and, conversely, a video should be explained in terms of its static image content and all the remaining characteristics not present in the initial frame.
Geometry-Free View Synthesis: Transformers and no 3D Priors
Is a geometric model required to synthesize novel views from a single image?
Ranked #1 on
Novel View Synthesis
on RealEstate10K
Taming Transformers for High-Resolution Image Synthesis
We demonstrate how combining the effectiveness of the inductive bias of CNNs with the expressivity of transformers enables them to model and thereby synthesize high-resolution images.
Ranked #3 on
Text-to-Image Generation
on LHQC
A Note on Data Biases in Generative Models
It is tempting to think that machines are less prone to unfairness and prejudice.
Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs
To open such a black box, it is, therefore, crucial to uncover the different semantic concepts a model has learned as well as those that it has learned to be invariant to.
Network-to-Network Translation with Conditional Invertible Neural Networks
Given the ever-increasing computational costs of modern machine learning models, we need to find new ways to reuse such expert models and thus tap into the resources that have been invested in their creation.
A Disentangling Invertible Interpretation Network for Explaining Latent Representations
We formulate interpretation as a translation of hidden representations onto semantic concepts that are comprehensible to the user.
