Search Results for author:
Found 17 papers, 14 papers with code
Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models
To address this challenge, we introduce the Cosmos-Drive-Dreams - a synthetic data generation (SDG) pipeline that aims to generate challenging scenarios to facilitate downstream tasks such as perception and driving policy training.
Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control
We introduce Cosmos-Transfer, a conditional world generation model that can generate world simulations based on multiple spatial control inputs of various modalities such as segmentation, depth, and edge.
GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control
Our results demonstrate more precise camera control than prior work, as well as state-of-the-art results in sparse-view novel view synthesis, even in challenging settings such as driving scenes and monocular dynamic video.
Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models
At the core of our approach is Difix, a single-step image diffusion model trained to enhance and remove artifacts in rendered novel views caused by underconstrained regions of the 3D representation.
InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Models
We present InfiniCube, a scalable method for generating unbounded dynamic 3D driving scenes with high fidelity and controllability.
SCube: Instant Large-Scale Scene Reconstruction using VoxSplats
We present SCube, a novel method for reconstructing large-scale 3D scenes (geometry, appearance, and semantics) from a sparse set of posed images.
fVDB: A Deep-Learning Framework for Sparse, Large-Scale, and High-Performance Spatial Intelligence
We present fVDB, a novel GPU-optimized framework for deep learning on large-scale 3D data.
XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies
We present XCube (abbreviated as $\mathcal{X}^3$), a novel generative model for high-resolution sparse 3D voxel grids with arbitrary attributes.
Blind Video Deflickering by Neural Filtering with a Flawed Atlas
Prior work usually requires specific guidance such as the flickering frequency, manual annotations, or extra consistent videos to remove the flicker.
Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Image
Novel view synthesis from a single image has recently attracted a lot of attention, and it has been primarily advanced by 3D deep learning and rendering techniques.
Retriever: Learning Content-Style Representation as a Token-Level Bipartite Graph
Last, an innovative link attention module serves as the decoder to reconstruct data from the decomposed content and style, with the help of the linking keys.
Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving
The ability to predict unseen vehicles is critical for safety in autonomous driving.
Learning Disentangled Representation by Exploiting Pretrained Generative Models: A Contrastive Learning View
Based on this observation, we argue that it is possible to mitigate the trade-off by $(i)$ leveraging the pretrained generative models with high generation quality, $(ii)$ focusing on discovering the traversal directions as factors for disentangled representation learning.
Rethinking Content and Style: Exploring Bias for Unsupervised Disentanglement
From the unsupervised disentanglement perspective, we rethink content and style and propose a formulation for unsupervised C-S disentanglement based on our assumption that different factors are of different importance and popularity for image reconstruction, which serves as a data bias.
Towards Building A Group-based Unsupervised Representation Disentanglement Framework
We then propose a model, based on existing VAE-based methods, to tackle the unsupervised learning problem of the framework.
Video Deblurring by Fitting to Test Data
Our key observation is that some frames in a video with motion blur are much sharper than others, and thus we can transfer the texture information in those sharp frames to blurry frames.
Music-oriented Dance Video Synthesis with Pose Perceptual Loss
We present a learning-based approach with pose perceptual loss for automatic music video generation.
