Search Results for author:
Found 36 papers, 18 papers with code
HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation
During inference, an image is generated by predicting all the tokens in the next (higher-resolution) scale, conditioned on all tokens in all previous (lower-resolution) scales.
The Dance of Atoms-De Novo Protein Design with Diffusion Model
The de novo design of proteins refers to creating proteins with specific structures and functions that do not naturally exist.
Personalized Text-to-Image Generation with Auto-Regressive Models
Personalized image synthesis has emerged as a pivotal application in text-to-image generation, enabling the creation of images featuring specific subjects in diverse contexts.
Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control
We introduce Cosmos-Transfer, a conditional world generation model that can generate world simulations based on multiple spatial control inputs of various modalities such as segmentation, depth, and edge.
Simulating the Real World: A Unified Survey of Multimodal Generative Models
Understanding and replicating the real world is a critical challenge in Artificial General Intelligence (AGI) research.
Cosmos World Foundation Model Platform for Physical AI
In this paper, we present the Cosmos World Foundation Model Platform to help developers build customized world models for their Physical AI setups.
HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation
HMAR reformulates next-scale prediction as a Markovian process, wherein prediction of each resolution scale is conditioned only on tokens in its immediate predecessor instead of the tokens in all predecessor resolutions.
3DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generation
Previous methods on controllable video generation primarily leverage 2D control signals to manipulate object motions and have achieved remarkable synthesis results.
Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding
In this paper, we propose a training-free probabilistic parallel decoding algorithm, Speculative Jacobi Decoding (SJD), to accelerate auto-regressive text-to-image generation.
EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generation
Current auto-regressive mesh generation methods suffer from issues such as incompleteness, insufficient detail, and poor generalization.
MotionCraft: Crafting Whole-Body Motion with Plug-and-Play Multimodal Controls
However, employing a unified model to achieve various generation tasks with different condition modalities presents two main challenges: motion distribution drifts across different tasks (e. g., co-speech gestures and text-driven daily actions) and the complex optimization of mixed conditions with varying granularities (e. g., text and audio).
T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation
In this work, we conduct the first systematic study on compositional text-to-video generation.
TextCraftor: Your Text Encoder Can be Image Quality Controller
Our findings reveal that, instead of replacing the CLIP text encoder used in Stable Diffusion with other large language models, we can enhance it through our proposed fine-tuning approach, TextCraftor, leading to substantial improvements in quantitative benchmarks and human assessments.
TC4D: Trajectory-Conditioned Text-to-4D Generation
We learn local deformations that conform to the global trajectory using supervision from a text-to-video model.
BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion
Image inpainting, the process of restoring corrupted images, has seen significant advancements with the advent of diffusion models (DMs).
E$^{2}$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation
One highly promising direction for enabling flexible real-time on-device image editing is utilizing data distillation by leveraging large-scale text-to-image diffusion models to generate paired datasets used for training generative adversarial networks (GANs).
HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting
In this paper, we propose an efficient yet effective framework, HumanGaussian, that generates high-quality 3D humans with fine-grained geometry and realistic appearance.
HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion
Our model enforces the joint learning of image appearance, spatial relationship, and geometry in a unified network, where each branch in the model complements to each other with both structural awareness and textural richness.
Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos
In the second stage, for each semantics, we randomly sample slots from the corresponding Gaussian distribution and perform masked feature aggregation within the semantic area to exploit temporal correspondence patterns for instance identification.
Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis
This paradigm extends the 2D image diffusion model to a volumetric version with a slightly increasing number of parameters and computation, offering a principled solution for generic cross-modality 3D medical image synthesis.
MonoHuman: Animatable Human Neural Field from Monocular Video
Recent works propose to graft a deformation network into the NeRF to further model the dynamics of the human neural field for animating vivid human motions.
Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation
In this work, we propose a novel diffusion-based framework, named Diffusion Co-Speech Gesture (DiffGesture), to effectively capture the cross-modal audio-to-gesture associations and preserve temporal coherence for high-fidelity audio-driven co-speech gesture generation.
Explicit and Implicit Knowledge Distillation via Unlabeled Data
Data-free knowledge distillation is a challenging model lightweight task for scenarios in which the original dataset is not available.
Audio-Driven Co-Speech Gesture Video Generation
Our key insight is that the co-speech gestures can be decomposed into common motion patterns and subtle rhythmic dynamics.
Static and Dynamic Concepts for Self-supervised Video Representation Learning
In this paper, we propose a novel learning scheme for self-supervised video representation learning.
Object-Compositional Neural Implicit Surfaces
This paper proposes a novel framework, ObjectSDF, to build an object-compositional neural implicit representation with high fidelity in 3D reconstruction and object representation.
Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation
To enhance the quality of synthesized gestures, we develop a contrastive learning strategy based on audio-text alignment for better audio representations.
Ranked #3 on
Gesture Generation
on TED Gesture Dataset
Resource allocation for reconfigurable intelligent surface aided broadcast channels
A two-user downlink network aided by a reconfigurable intelligent surface is considered.
Visual Sound Localization in the Wild by Cross-Modal Interference Erasing
Specifically, we observe that the previous practice of learning only a single audio representation is insufficient due to the additive nature of audio signals.
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation
Moreover, to enable portrait rendering in one unified neural radiance field, a Torso Deformation module is designed to stabilize the large-scale non-rigid torso motions.
A Two-Stage Stochastic Programming Model for Blood Supply Chain Management, Considering Facility Disruption and Service Level
In this paper, a blood supply chain network, where the occurrence of disruption might interrupt the flow of Red Blood Cells, is dealt with.
Interpreting Molecule Generative Models for Interactive Molecule Discovery
In this work, we develop a simple yet effective method to interpret the latent space of the learned generative models with various molecular properties for more interactive molecule generation and discovery.
Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization
The crux of self-supervised video representation learning is to build general features from unlabeled videos.
A Simulation-Optimization Technique for Service Level Analysis in Conjunction with Reorder Point Estimation and Lead-Time consideration: A Case Study in Sea Port
This study offers a step-by-step practical procedure from the analysis of the current status of the spare parts inventory system to advanced service-level analysis by virtue of simulation-optimization technique for a real-world case study associated with a seaport.
Motion Capture from Internet Videos
Therefore, we propose to capture human motion by jointly analyzing these Internet videos instead of using single videos separately.
Optimizing the Total Production and Maintenance Cost of an Integrated Multi-Product Process and Maintenance Planning (IPPMP) Model
Furthermore, a rational presumption is reflected in the problem statement in which the time and cost of PM are pertinent to the interval between the prior perfect repair and current PM.
