Search Results for author:
Found 19 papers, 10 papers with code
DreamO: A Unified Framework for Image Customization
Recently, extensive research on image customization (e. g., identity, subject, style, background, etc.)
DreamID: High-Fidelity and Fast diffusion-based Face Swapping via Triplet ID Group Learning
To address this issue, we leverage the accelerated diffusion model SD Turbo, reducing the inference steps to a single iteration, enabling efficient pixel-level end-to-end training with explicit Triplet ID Group supervision.
Phantom: Subject-consistent video generation via cross-modal alignment
The continuous development of foundational models for video generation is evolving into various applications, with subject-consistent video generation still in the exploratory stage.
DreamOmni: Unified Image Generation and Editing
We begin by analyzing existing frameworks and the requirements of downstream tasks, proposing a unified framework that integrates both T2I models and various editing tasks.
CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generation
To Inherit the advantages of MM-DiT, we use a separate set of network weights to process the layout, treating it as equally important as the image and text modalities.
TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation
This design enables direct access to both high-level semantic representations crucial for understanding tasks and fine-grained visual features essential for generation through shared indices.
DBF-Net: A Dual-Branch Network with Feature Fusion for Ultrasound Image Segmentation
To address this, we introduce UBBS-Net, a dual-branch deep neural network that learns the relationship between body and boundary for improved segmentation.
How to Bridge Spatial and Temporal Heterogeneity in Link Prediction? A Contrastive Method
Specifically, aiming at spatial heterogeneity, we develop a spatial feature modeling layer to capture the fine-grained topological distribution patterns from node- and edge-level representations, respectively.
Towards Bridging the Cross-modal Semantic Gap for Multi-modal Recommendation
Multi-modal recommendation greatly enhances the performance of recommender systems by modeling the auxiliary information from multi-modality contents.
VCoME: Verbal Video Composition with Multimodal Editing Effects
To address this task, we propose VCoME, a general framework that employs a large multimodal model to generate editing effects for video composition.
Development of Skip Connection in Deep Neural Networks for Computer Vision and Medical Image Analysis: A Survey
Deep learning has made significant progress in computer vision, specifically in image classification, object detection, and semantic segmentation.
Graphic Design with Large Multimodal Model
One existing practice is Graphic Layout Generation (GLG), which aims to layout sequential design elements.
DiffI2I: Efficient Diffusion Model for Image-to-Image Translation
Compared to traditional DMs, the compact IPR enables DiffI2I to obtain more accurate outcomes and employ a lighter denoising network and fewer iterations.
DiffIR: Efficient Diffusion Model for Image Restoration
Diffusion model (DM) has achieved SOTA performance by modeling the image synthesis process into a sequential application of a denoising network.
Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferring
In this paper, based on the CLIP model, we revisit temporal modeling in the context of image-to-video knowledge transferring, which is the key point for extending image-text pretrained models to the video domain.
Ranked #7 on
Video Retrieval
on MSR-VTT-1kA
(using extra training data)
Class Prototype-based Cleaner for Label Noise Learning
Semi-supervised learning based methods are current SOTA solutions to the noisy-label learning problem, which rely on learning an unsupervised label cleaner first to divide the training samples into a labeled set for clean data and an unlabeled set for noise data.
Ranked #4 on
Image Classification
on Clothing1M
(using extra training data)
Clover: Towards A Unified Video-Language Alignment and Fusion Model
We then introduce \textbf{Clover}\textemdash a Correlated Video-Language pre-training method\textemdash towards a universal Video-Language model for solving multiple video understanding tasks with neither performance nor efficiency compromise.
Ranked #1 on
Video Question Answering
on LSMDC-FiB
CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIP
Once trained, the transformer can generate coherent image tokens based on the text embedding extracted from the text encoder of CLIP upon an input text.
Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Report
To address this problem, we introduce the first Mobile AI challenge, where the target is to develop quantized deep learning-based camera scene classification solutions that can demonstrate a real-time performance on smartphones and IoT platforms.
