Search Results for author:
Found 31 papers, 14 papers with code
MergeVQ: A Unified Framework for Visual Generation and Representation with Disentangled Token Merging and Quantization
Masked Image Modeling (MIM) with Vector Quantization (VQ) has achieved great success in both self-supervised pre-training and image generation.
Improved Visual-Spatial Reasoning via R1-Zero-Like Training
Increasing attention has been placed on improving the reasoning capacities of multi-modal large language models (MLLMs).
H2VU-Benchmark: A Comprehensive Benchmark for Hierarchical Holistic Video Understanding
To tackle this challenge, we propose a hierarchical and holistic video understanding (H2VU) benchmark designed to evaluate both general video and online streaming video comprehension.
Layton: Latent Consistency Tokenizer for 1024-pixel Image Reconstruction and Generation by 256 Tokens
Experiments demonstrate Layton's superiority in high-fidelity reconstruction, with 10. 8 reconstruction Frechet Inception Distance on MSCOCO-2017 5K benchmark for 1024x1024 image reconstruction.
X2I: Seamless Integration of Multimodal Understanding into Diffusion Transformer via Attention Distillation
Compared to the teacher model, X2I shows a decrease in performance degradation of less than 1\% while gaining various multimodal understanding abilities, including multilingual to image, image to image, image-text to image, video to image, audio to image, and utilizing creative fusion to enhance imagery.
GenX: Mastering Code and Test Generation with Execution Feedback
Recent advancements in language modeling have enabled the translation of natural language into code, and the use of execution feedback to improve code generation.
PainterNet: Adaptive Image Inpainting with Actual-Token Attention and Diverse Mask Control
Recently, diffusion models have exhibited superior performance in the area of image inpainting.
HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluator
To address these challenges, we propose HEIE: a novel MLLM-Based Hierarchical Explainable image Implausibility Evaluator.
LLMI3D: Empowering LLM with 3D Perception from a Single 2D Image
Recent advancements in autonomous driving, augmented reality, robotics, and embodied intelligence have necessitated 3D perception algorithms.
GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models
Posters play a crucial role in marketing and advertising by enhancing visual communication and brand visibility, making significant contributions to industrial design.
TLCM: Training-efficient Latent Consistency Model for Image Generation with 2-8 Steps
Distilling latent diffusion models (LDMs) into ones that are fast to sample from is attracting growing research interest.
Probing Language Models for Pre-training Data Detection
Large Language Models (LLMs) have shown their impressive capabilities, while also raising concerns about the data contamination problems due to privacy issues and leakage of benchmark datasets in the pre-training phase.
LAPTOP-Diff: Layer Pruning and Normalized Distillation for Compressing Diffusion Models
To this end, we proposed the layer pruning and normalized distillation for compressing diffusion models (LAPTOP-Diff).
InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions
Instruction tuning effectively optimizes Large Language Models (LLMs) for downstream tasks.
SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation
In contrast to vanilla consistency distillation (CD) which distills the ordinary differential equation solvers-based sampling process of a pretrained teacher model into a student, SCott explores the possibility and validates the efficacy of integrating stochastic differential equation (SDE) solvers into CD to fully unleash the potential of the teacher.
Dream360: Diverse and Immersive Outdoor Virtual Scene Creation via Transformer-Based 360 Image Outpainting
To this end, we propose a transformer-based 360 image outpainting framework called Dream360, which can generate diverse, high-fidelity, and high-resolution panoramas from user-selected viewports, considering the spherical properties of 360 images.
PEA-Diffusion: Parameter-Efficient Adapter with Knowledge Distillation in non-English Text-to-Image Generation
In this paper, we are inspired to propose a simple plug-and-play language transfer method based on knowledge distillation.
Cross-lingual Text-to-Image Generation
Knowledge Distillation
+1
MCAD: Multi-teacher Cross-modal Alignment Distillation for efficient image-text retrieval
Due to the success of large-scale visual-language pretraining (VLP) models and the widespread use of image-text retrieval in industry areas, it is now critically necessary to reduce the model size and streamline their mobile-device deployment.
MoEController: Instruction-based Arbitrary Image Manipulation with Mixture-of-Expert Controllers
In this work, we propose a method with a mixture-of-expert (MOE) controllers to align the text-guided capacity of diffusion models with different kinds of human instructions, enabling our model to handle various open-domain image manipulation tasks with natural language instructions.
Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning
In this paper, we propose Subject-Diffusion, a novel open-domain personalized image generation model that, in addition to not requiring test-time fine-tuning, also only requires a single reference image to support personalized generation of single- or multi-subject in any domain.
Diffusion Personalization Tuning Free
Personalized Image Generation
+1
Prompt Space Optimizing Few-shot Reasoning Success with Large Language Models
To address this issue in prompt engineering, we propose a new and effective approach called Prompt Space.
Towards Language-guided Interactive 3D Generation: LLMs as Layout Interpreter with Generative Feedback
Generating and editing a 3D scene guided by natural language poses a challenge, primarily due to the complexity of specifying the positional relations and volumetric changes within the 3D space.
Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models
Our approach produces a more semantically accurate synthesis by constraining the attention regions of each token in the prompt to the image.
Edit Everything: A Text-Guided Generative System for Images Editing
We introduce a new generative system called Edit Everything, which can take image and text inputs and produce image outputs.
GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image Generation
Recent breakthroughs in the field of language-guided image generation have yielded impressive achievements, enabling the creation of high-quality and diverse images based on user instructions. Although the synthesis performance is fascinating, one significant limitation of current image generation models is their insufficient ability to generate text coherently within images, particularly for complex glyph structures like Chinese characters.
Optical Character Recognition (OCR)
parameter-efficient fine-tuning
+1
CompoNeRF: Text-guided Multi-object Compositional NeRF with Editable 3D Scene Layout
To tackle the issue of 'guidance collapse' and further enhance scene consistency, we propose a novel framework, dubbed CompoNeRF, by integrating an editable 3D scene layout with object-specific and scene-wide guidance mechanisms.
GammaE: Gamma Embeddings for Logical Queries on Knowledge Graphs
However, it remains challenging to model the negation and union operator.
Emily: Developing An Emotion-affective Open-Domain Chatbot with Knowledge Graph-based Persona
In this paper, we describe approaches for developing Emily, an emotion-affective open-domain chatbot.
KELM: Knowledge Enhanced Pre-Trained Language Representations with Message Passing on Hierarchical Relational Graphs
Incorporating factual knowledge into pre-trained language models (PLM) such as BERT is an emerging trend in recent NLP studies.
Ranked #11 on
Common Sense Reasoning
on ReCoRD
AugNet: End-to-End Unsupervised Visual Representation Learning with Image Augmentation
In our evaluations, the method outperforms all the state-of-the-art image retrieval algorithms on some out-of-domain image datasets.
DensE: An Enhanced Non-commutative Representation for Knowledge Graph Embedding with Adaptive Semantic Hierarchy
This design principle leads to several advantages of our method: (1) For composite relations, the corresponding diagonal relation matrices can be non-commutative, reflecting a predominant scenario in real world applications; (2) Our model preserves the natural interaction between relational operations and entity embeddings; (3) The scaling operation provides the modeling power for the intrinsic semantic hierarchical structure of entities; (4) The enhanced expressiveness of DensE is achieved with high computational efficiency in terms of both parameter size and training time; and (5) Modeling entities in Euclidean space instead of quaternion space keeps the direct geometrical interpretations of relational patterns.
Ranked #7 on
Link Prediction
on WN18
