Search Results for author:
Found 10 papers, 5 papers with code
JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation
To further improve the performance of our unified model, we adopt two key strategies: (i) decoupling the understanding and generation encoders, and (ii) aligning their representations during unified training.
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation
In this paper, we introduce Janus, an autoregressive framework that unifies multimodal understanding and generation.
Ranked #134 on
Visual Question Answering
on MM-Vet
MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusion
For the first time, we leverage diffusion models as effective skeleton representation learners.
Correcting Diffusion-Based Perceptual Image Compression with Privileged End-to-End Decoder
We build a diffusion model and design a novel paradigm that combines the diffusion model and an end-to-end decoder, and the latter is responsible for transmitting the privileged information extracted at the encoder side.
Diffusion Enhancement for Cloud Removal in Ultra-Resolution Remote Sensing Imagery
The presence of cloud layers severely compromises the quality and effectiveness of optical remote sensing (RS) images.
Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution
Diffusion models, as a kind of powerful generative model, have given impressive results on image super-resolution (SR) tasks.
Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation
Language-guided image generation has achieved great success nowadays by using diffusion models.
MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
To generate joint audio-video pairs, we propose a novel Multi-Modal Diffusion model (i. e., MM-Diffusion), with two-coupled denoising autoencoders.
AI Illustrator: Translating Raw Descriptions into Images by Prompt-based Cross-Modal Generation
To address this issue, we propose a Prompt-based Cross-Modal Generation Framework (PCM-Frame) to leverage two powerful pre-trained models, including CLIP and StyleGAN.
Meta-Interpolation: Time-Arbitrary Frame Interpolation via Dual Meta-Learning
Extensive qualitative and quantitative evaluations, as well as ablation studies, demonstrate that, via introducing meta-learning in our framework in such a well-designed way, our method not only achieves superior performance to state-of-the-art frame interpolation approaches but also owns an extended capacity to support the interpolation at an arbitrary time-step.
