Search Results for author:
Found 53 papers, 40 papers with code
Understanding and Stabilizing GANs' Training Dynamics Using Control Theory
There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods.
CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model
In this work, we present the Convolutional Reconstruction Model (CRM), a high-fidelity feed-forward single image-to-3D generative model.
Graph Diffusion Policy Optimization
Recent research has made significant progress in optimizing diffusion models for specific downstream objectives, which is an important pursuit in fields such as graph generation for drug design.
The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing
We present a unified probabilistic formulation for diffusion-based image editing, where a latent variable is edited in a task-specific manner and generally deviates from the corresponding marginal distribution induced by the original stochastic or ordinary differential equation (SDE or ODE).
BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inference
In particular, we derive a novel uncertainty iteration principle to characterize the uncertainty dynamics in diffusion, and leverage the last-layer Laplace approximation for efficient Bayesian inference.
On Memorization in Diffusion Models
Looking into this, we first observe that memorization behaviors tend to occur on smaller-sized datasets, which motivates our definition of effective model memorization (EMM), a metric measuring the maximum size of training data at which a learned diffusion model approximates its theoretical optimum.
Inversion-by-Inversion: Exemplar-based Sketch-to-Photo Synthesis via Stochastic Differential Equations without Training
In the full-control inversion process, we propose an appearance-energy function to control the color and texture of the final generated photo. Importantly, our Inversion-by-Inversion pipeline is training-free and can accept different types of exemplars for color and texture control.
MissDiff: Training Diffusion Models on Tabular Data with Missing Values
The diffusion model has shown remarkable performance in modeling data distributions and synthesizing data.
ControlVideo: Conditional Control for One-shot Text-driven Video Editing and Beyond
This paper presents \emph{ControlVideo} for text-driven video editing -- generating a video that aligns with a given text while preserving the structure of the source video.
On Evaluating Adversarial Robustness of Large Vision-Language Models
Large vision-language models (VLMs) such as GPT-4 have achieved unprecedented performance in response generation, especially with visual inputs, enabling more creative and adaptable interaction than large language models such as ChatGPT.
ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation
In comparison, VSD works well with various CFG weights as ancestral sampling from diffusion models and simultaneously improves the diversity and sample quality with a common CFG weight (i. e., $7. 5$).
Towards Understanding Generalization of Macro-AUC in Multi-label Learning
We theoretically identify a critical factor of the dataset affecting the generalization bounds: \emph{the label-wise class imbalance}.
Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning
The main challenge for this setting is that the intermediate guidance during the diffusion sampling procedure, which is jointly defined by the sampling distribution and the energy function, is unknown and is hard to estimate.
A Closer Look at Parameter-Efficient Tuning in Diffusion Models
Large-scale diffusion models like Stable Diffusion are powerful and find various real-world applications while customizing such models by fine-tuning is both memory and time inefficient.
One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale
Inspired by the unified view, UniDiffuser learns all distributions simultaneously with a minimal modification to the original diffusion model -- perturbs data in all modalities instead of a single modality, inputs individual timesteps in different modalities, and predicts the noise of all modalities instead of a single modality.
Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels
In an effort to further advance semi-supervised generative and classification tasks, we propose a simple yet effective training strategy called dual pseudo training (DPT), built upon strong semi-supervised learners and diffusion models.
Revisiting Discriminative vs. Generative Classifiers: Theory and Implications
Theoretically, the paper considers the surrogate loss instead of the zero-one loss in analyses and generalizes the classical results from binary cases to multiclass ones.
Why Are Conditional Generative Models Better Than Unconditional Ones?
Extensive empirical evidence demonstrates that conditional generative models are easier to train and perform better than unconditional ones by exploiting the labels of data.
Diffusion Denoising Process for Perceptron Bias in Out-of-distribution Detection
In this paper, we introduce a new perceptron bias assumption that suggests discriminator models are more sensitive to certain features of the input, leading to the overconfidence problem.
DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
The commonly-used fast sampler for guided sampling is DDIM, a first-order diffusion ODE solver that generally needs 100 to 250 steps for high-quality samples.
Equivariant Energy-Guided SDE for Inverse Molecular Design
Inverse molecular design is critical in material science and drug discovery, where the generated molecules should satisfy certain desirable properties.
All are Worth Words: A ViT Backbone for Diffusion Models
We evaluate U-ViT in unconditional and class-conditional image generation, as well as text-to-image generation tasks, where U-ViT is comparable if not superior to a CNN-based U-Net of a similar size.
Ranked #4 on
Text-to-Image Generation
on MS COCO
Deep Generative Modeling on Limited Data with Regularization by Nontransferable Pre-trained Models
Deep generative models (DGMs) are data-eager because learning a complex model on limited data suffers from a large variance and easily overfits.
EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations
Further, we provide an alternative explanation of the EGSDE as a product of experts, where each of the three experts (corresponding to the SDE and two feature extractors) solely contributes to faithfulness or realism.
Ranked #1 on
Image-to-Image Translation
on AFHQ (Wild to Dog)
Fast Lossless Neural Compression with Integer-Only Discrete Flows
In this work, we propose Integer-only Discrete Flows (IODF), an efficient neural compressor with integer-only arithmetic.
Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matching
To fill up this gap, we show that the negative likelihood of the ODE can be bounded by controlling the first, second, and third-order score matching errors; and we further present a novel high-order denoising score matching method to enable maximum likelihood training of score-based diffusion ODEs.
Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models
Thus, the generation performance on a subset of timesteps is crucial, which is greatly influenced by the covariance design in DPMs.
DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
In this work, we propose an exact formulation of the solution of diffusion ODEs.
Memory Replay with Data Compression for Continual Learning
In this work, we propose memory replay with data compression (MRDC) to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer.
Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models
In this work, we present a surprising result that both the optimal reverse variance and the corresponding optimal KL divergence of a DPM have analytic forms w. r. t.
Stability and Generalization of Bilevel Programming in Hyperparameter Optimization
Our results can explain some mysterious behaviours of the bilevel programming in practice, for instance, overfitting to the validation set.
Rethinking and Reweighting the Univariate Losses for Multi-Label Ranking: Consistency and Generalization
Our results show that learning algorithms with the consistent univariate loss have an error bound of $O(c)$ ($c$ is the number of labels), while algorithms with the inconsistent pairwise loss depend on $O(\sqrt{c})$ as shown in prior work.
MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering
We present Mixture of Contrastive Experts (MiCE), a unified probabilistic clustering framework that simultaneously exploits the discriminative representations learned by contrastive learning and the semantic structures captured by a latent mixture model.
Ranked #9 on
Image Clustering
on Imagenet-dog-15
Implicit Normalizing Flows
Through theoretical analysis, we show that the function space of ImpFlow is strictly richer than that of ResFlows.
Relaxed Conditional Image Transfer for Semi-supervised Domain Adaptation
Semi-supervised domain adaptation (SSDA), which aims to learn models in a partially labeled target domain with the assistance of the fully labeled source domain, attracts increasing attention in recent years.
ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning
Continual learning usually assumes the incoming data are fully labeled, which might not be applicable in real applications.
Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models
The learning and evaluation of energy-based latent variable models (EBLVMs) without any structural assumptions are highly challenging, because the true posteriors and the partition functions in such models are generally intractable.
Bi-level Score Matching for Learning Energy-based Latent Variable Models
This paper presents a bi-level score matching (BiSM) method to learn EBLVMs with general structures by reformulating SM as a bi-level optimization problem.
Efficient Learning of Generative Models via Finite-Difference Score Matching
Several machine learning applications involve the optimization of higher-order derivatives (e. g., gradients of gradients) during training, which can be expensive in respect to memory and computation even with automatic differentiation.
Triple Generative Adversarial Networks
It is formulated as a three-player minimax game consisting of a generator, a classifier and a discriminator, and therefore is referred to as Triple Generative Adversarial Network (Triple-GAN).
Understanding and Stabilizing GANs' Training Dynamics with Control Theory
There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods.
Ranked #37 on
Image Generation
on CIFAR-10
(Inception score metric)
Multi-objects Generation with Amortized Structural Regularization
Deep generative models (DGMs) have shown promise in image generation.
Countering Noisy Labels By Learning From Auxiliary Clean Labels
We consider the learning from noisy labels (NL) problem which emerges in many real-world applications.
To Relieve Your Headache of Training an MRF, Take AdVIL
We propose a black-box algorithm called {\it Adversarial Variational Inference and Learning} (AdVIL) to perform inference and learning on a general Markov random field (MRF).
Learning Implicit Generative Models by Teaching Explicit Ones
Implicit generative models are difficult to train as no explicit density functions are defined.
Graphical Generative Adversarial Networks
We propose Graphical Generative Adversarial Networks (Graphical-GAN) to model structured data.
Population Matching Discrepancy and Applications in Deep Learning
In this paper, we propose population matching discrepancy (PMD) for estimating the distribution distance based on samples, as well as an algorithm to learn the parameters of the distributions using PMD as an objective.
Triple Generative Adversarial Nets
Generative Adversarial Nets (GANs) have shown promise in image generation and semi-supervised learning (SSL).
Collaborative Filtering with User-Item Co-Autoregressive Models
Deep neural networks have shown promise in collaborative filtering (CF).
Max-Margin Deep Generative Models for (Semi-)Supervised Learning
Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability.
Towards Better Analysis of Deep Convolutional Neural Networks
Deep convolutional neural networks (CNNs) have achieved breakthrough performance in many pattern recognition tasks such as image classification.
Learning to Generate with Memory
Memory units have been widely used to enrich the capabilities of deep networks on capturing long-term dependencies in reasoning and prediction tasks, but little investigation exists on deep generative models (DGMs) which are good at inferring high-level invariant representations from unlabeled data.
Max-margin Deep Generative Models
Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability.
