Search Results for author:
Found 42 papers, 14 papers with code
SimVS: Simulating World Inconsistencies for Robust View Synthesis
Novel-view synthesis techniques achieve impressive results for static scenes but struggle when faced with the inconsistencies inherent to casual capture settings: varying illumination, scene motion, and other unintended effects that are difficult to model explicitly.
CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
We present CAT4D, a method for creating 4D (dynamic 3D) scenes from monocular video.
Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion
Compared to pixel-space models that are trained end-to-end, latent models are perceived to be more efficient and to produce higher image quality at high resolution.
Ranked #1 on
Image Generation
on ImageNet 128x128
Think Twice Before You Act: Improving Inverse Problem Solving With MCMC
Despite the merits of being versatile in solving various inverse problems without re-training, the performance of DPS is hindered by the fact that this posterior approximation can be inaccurate especially for high noise levels.
Generative Hierarchical Materials Search
We confirm that GenMS is able to generate common crystal structures such as double perovskites, or spinels, solely from natural language input, and hence can form the foundation for more complex structure generation in near future.
Large Language Models are Limited in Out-of-Context Knowledge Reasoning
Using this dataset, we evaluated several LLMs and discovered that their proficiency in this aspect is limited, regardless of whether the knowledge is trained in a separate or adjacent training settings.
Latent Energy-Based Odyssey: Black-Box Optimization via Expanded Exploration in the Energy-Based Latent Space
To this end, we formulate an learnable energy-based latent space, and propose Noise-intensified Telescoping density-Ratio Estimation (NTRE) scheme for variational learning of an accurate latent space model without costly Markov Chain Monte Carlo.
EM Distillation for One-step Diffusion Models
We propose EM Distillation (EMD), a maximum likelihood-based approach that distills a diffusion model to a one-step generator model with minimal loss of perceptual quality.
An Investigation of Conformal Isometry Hypothesis for Grid Cells
As the agent moves, this vector rotates within a 2D manifold in the neural space, driven by a recurrent neural network.
CoLay: Controllable Layout Generation through Multi-conditional Latent Diffusion
Layout design generation has recently gained significant attention due to its potential applications in various fields, including UI, graphic, and floor plan design.
CAT3D: Create Anything in 3D with Multi-View Diffusion Models
Advances in 3D reconstruction have enabled high-quality 3D capture, but require a user to collect hundreds to thousands of images to create a 3D scene.
MagicMirror: Fast and High-Quality Avatar Generation with a Constrained Search Space
We introduce a novel framework for 3D human avatar generation and personalization, leveraging text prompts to enhance user engagement and customization.
ReconFusion: 3D Reconstruction with Diffusion Priors
3D reconstruction methods such as Neural Radiance Fields (NeRFs) excel at rendering photorealistic novel views of complex scenes.
Emergence of Grid-like Representations by Training Recurrent Networks with Conformal Normalization
As the agent moves, the vector is transformed by an RNN that takes the velocity of the agent as input.
Learning Energy-Based Prior Model with Diffusion-Amortized MCMC
To remedy this sampling issue, in this paper we introduce a simple but effective diffusion-based amortization method for long-run MCMC sampling and develop a novel learning algorithm for the latent space EBM based on it.
Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood
Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming, and there exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models.
Conformal Isometry of Lie Group Representation in Recurrent Network of Grid Cells
Recurrent neural networks have been proposed to explain the properties of the grid cells by updating the neural activity vector based on the velocity input of the animal.
On Distillation of Guided Diffusion Models
For standard diffusion models trained on the pixel-space, our approach is able to generate images visually comparable to that of the original model using as few as 4 sampling steps on ImageNet 64x64 and CIFAR-10, achieving FID/IS scores comparable to that of the original model while being up to 256 times faster to sample from.
Imagen Video: High Definition Video Generation with Diffusion Models
We present Imagen Video, a text-conditional video generation system based on a cascade of video diffusion models.
Ranked #1 on
Video Generation
on LAION-400M
Latent Diffusion Energy-Based Model for Interpretable Text Modeling
Latent space Energy-Based Models (EBMs), also known as energy-based priors, have drawn growing interests in generative modeling.
MCMC Should Mix: Learning Energy-Based Model with Flow-Based Backbone
However, MCMC sampling of EBMs in high-dimensional data space is generally not mixing, because the energy function, which is usually parametrized by deep network, is highly multi-modal in the data space.
Learning Neural Representation of Camera Pose with Matrix Representation of Pose Shift via View Synthesis
Specifically, the camera pose and 3D scene are represented as vectors and the local camera movement is represented as a matrix operating on the vector of the camera pose.
A Theory of Label Propagation for Subpopulation Shift
In this work, we propose a provably effective framework for domain adaptation based on label propagation.
Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis
3D data that contains rich geometry information of objects and scenes is valuable for understanding 3D physical world.
Learning Energy-Based Models by Diffusion Recovery Likelihood
Inspired by recent progress on diffusion probabilistic models, we present a diffusion recovery likelihood method to tractably learn and sample from a sequence of EBMs trained on increasingly noisy versions of a dataset.
Ranked #23 on
Image Generation
on CelebA 64x64
A Representational Model of Grid Cells' Path Integration Based on Matrix Lie Algebras
The grid cells in the mammalian medial entorhinal cortex exhibit striking hexagon firing patterns when the agent navigates in the open field.
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot
In this paper, we conduct sanity checks for the above beliefs on several recent unstructured pruning methods and surprisingly find that: (1) A set of methods which aims to find good subnetworks of the randomly-initialized network (which we call "initial tickets"), hardly exploits any information from the training data; (2) For the pruned networks obtained by these methods, randomly changing the preserved weights in each layer, while keeping the total number of preserved weights unchanged per layer, does not affect the final performance.
On Path Integration of Grid Cells: Group Representation and Isotropic Scaling
In this paper, we conduct theoretical analysis of a general representation model of path integration by grid cells, where the 2D self-position is encoded as a higher dimensional vector, and the 2D self-motion is represented by a general transformation of the vector.
MCMC Should Mix: Learning Energy-Based Model with Neural Transport Latent Space MCMC
Learning energy-based model (EBM) requires MCMC sampling of the learned model as an inner loop of the learning algorithm.
Flow Contrastive Estimation of Energy-Based Models
(2) The update of the flow model approximately minimizes the Jensen-Shannon divergence between the flow model and the data distribution.
Motion-Based Generator Model: Unsupervised Disentanglement of Appearance, Trackable and Intrackable Motions in Dynamic Patterns
To model the motions explicitly, it is natural for the model to be based on the motions or the displacement fields of the pixels.
Representation Learning: A Statistical Perspective
Learning representations of data is an important problem in statistics and machine learning.
Convergence of Adversarial Training in Overparametrized Neural Networks
Neural networks are vulnerable to adversarial examples, i. e. inputs that are imperceptibly perturbed from natural data and yet incorrectly classified by the network.
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems
First-order methods such as stochastic gradient descent (SGD) are currently the standard algorithm for training deep neural networks.
Learning V1 Simple Cells with Vector Representation of Local Content and Matrix Representation of Local Motion
This paper proposes a representational model for image pairs such as consecutive video frames that are related by local pixel displacements, in the hope that the model may shed light on motion perception in primary visual cortex (V1).
Learning Dynamic Generator Model by Alternating Back-Propagation Through Time
The non-linear transformation of this transition model can be parametrized by a feedforward neural network.
Learning Grid Cells as Vector Representation of Self-Position Coupled with Matrix Representation of Self-Motion
In this model, the 2D self-position of the agent is represented by a high-dimensional vector, and the 2D self-motion or displacement of the agent is represented by a matrix that transforms the vector.
A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models
In this paper, we review three families of probability models, namely, the discriminative models, the descriptive models, and the generative models.
Deformable Generator Networks: Unsupervised Disentanglement of Appearance and Geometry
We present a deformable generator model to disentangle the appearance and geometric information for both image and video data in a purely unsupervised manner.
Learning Descriptor Networks for 3D Shape Synthesis and Analysis
This paper proposes a 3D shape descriptor network, which is a deep convolutional energy-based model, for modeling volumetric shape patterns.
Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Sampling
Within each iteration of our learning algorithm, for each observed training image, we generate synthesized images at multiple grids by initializing the finite-step MCMC sampling from a minimal 1 x 1 version of the training image.
Cooperative Training of Descriptor and Generator Networks
Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finite-step MCMC that samples and trains the energy-based descriptor model.
