Search Results for author:
Found 34 papers, 13 papers with code
Scattering Environment Aware Joint Multi-user Channel Estimation and Localization with Spatially Reused Pilots
This paper proposes a two-timescale approach for joint MU uplink channel estimation and localization in MIMO-OFDM systems, which fully captures the spatial characteristics of MUs.
A High-Quality Text-Rich Image Instruction Tuning Dataset via Hybrid Instruction Generation
Large multimodal models still struggle with text-rich images because of inadequate training data.
Numerical Pruning for Efficient Autoregressive Models
Transformers have emerged as the leading architecture in deep learning, proving to be versatile and highly effective across diverse domains beyond language and image processing.
SUGAR: Subject-Driven Video Customization in a Zero-Shot Manner
Compared to previous methods, SUGAR achieves state-of-the-art results in identity preservation, video dynamics, and video-text alignment for subject-driven video customization, demonstrating the effectiveness of our proposed method.
TTVD: Towards a Geometric Framework for Test-Time Adaptation Based on Voronoi Diagram
Deep learning models often struggle with generalization when deploying on real-world data, due to the common distributional shift to the training data.
LoRA-Contextualizing Adaptation of Large Multimodal Models for Long Document Understanding
In this work, we present a novel framework named LoRA-Contextualizing Adaptation of Large multimodal models (LoCAL), which broadens the capabilities of any LMM to support long-document understanding.
TextLap: Customizing Language Models for Text-to-Layout Planning
Automatic generation of graphical layouts is crucial for many real-world applications, including designing posters, flyers, advertisements, and graphical user interfaces.
Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Models
Video Large Language Models (Video-LLMs) have demonstrated remarkable capabilities in coarse-grained video understanding, however, they struggle with fine-grained temporal grounding.
MMR: Evaluating Reading Ability of Large Multimodal Models
Large multimodal models (LMMs) have demonstrated impressive capabilities in understanding various types of image, including text-rich images.
LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models
Large multimodal language models have demonstrated impressive capabilities in understanding and manipulating images.
Subspace Constrained Variational Bayesian Inference for Structured Compressive Sensing with a Dynamic Grid
We investigate the problem of recovering a structured sparse signal from a linear observation model with an uncertain dynamic grid in the sensing matrix.
ARTIST: Improving the Generation of Text-rich Images with Disentangled Diffusion Models and Large Language Models
Diffusion models have demonstrated exceptional capabilities in generating a broad spectrum of visual content, yet their proficiency in rendering text is still limited: they often generate inaccurate characters or words that fail to blend well with the underlying image.
Toffee: Efficient Million-Scale Dataset Construction for Subject-Driven Text-to-Image Generation
Our dataset is 5 times the size of previous largest dataset, yet our cost is tens of thousands of GPU hours lower.
TRINS: Towards Multimodal Language Models that Can Read
In this work, we introduce TRINS: a Text-Rich image INStruction dataset, with the objective of enhancing the reading ability of the multimodal large language model.
Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language Models
Our in-depth study advocates more careful use of ME in real-world scenarios.
Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints
Controllable layout generation refers to the process of creating a plausible visual arrangement of elements within a graphic design (e. g., document and web designs) with constraints representing design intentions.
Customization Assistant for Text-to-image Generation
Some existing methods do not require fine-tuning, while their performance are unsatisfactory.
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding
Instruction tuning unlocks the superior capability of Large Language Models (LLM) to interact with humans.
Enhancing Detail Preservation for Customized Text-to-Image Generation: A Regularization-Free Approach
However, generating images of novel concept provided by the user input image is still a challenging task.
Towards Building the Federated GPT: Federated Instruction Tuning
This repository offers a foundational framework for exploring federated fine-tuning of LLMs using heterogeneous instructions across diverse categories.
Learning to Estimate 3D Human Pose from Point Cloud
In this paper, we study the task of 3D human pose estimation from depth images.
Shifted Diffusion for Text-to-image Generation
Unlike the baseline diffusion model used in DALL-E 2, our method seamlessly encodes prior knowledge of the pre-trained CLIP model in its diffusion process by designing a new initialization distribution and a new transition step of the diffusion.
Ranked #3 on
Text-to-Image Generation
on Multi-Modal-CelebA-HQ
Lafite2: Few-shot Text-to-Image Generation
The low requirement of the proposed method yields high flexibility and usability: it can be beneficial to a wide range of settings, including the few-shot, semi-supervised and fully-supervised learning; it can be applied on different models including generative adversarial networks (GANs) and diffusion models.
Towards Language-Free Training for Text-to-Image Generation
One of the major challenges in training text-to-image generation models is the need of a large number of high-quality text-image pairs.
A Generic Approach for Enhancing GANs by Regularized Latent Optimization
With the rapidly growing model complexity and data volume, training deep generative models (DGMs) for better performance has becoming an increasingly more important challenge.
LAFITE: Towards Language-Free Training for Text-to-Image Generation
One of the major challenges in training text-to-image generation models is the need of a large number of high-quality image-text pairs.
Ranked #2 on
Text-to-Image Generation
on Multi-Modal-CelebA-HQ
Learning High-Dimensional Distributions with Latent Neural Fokker-Planck Kernels
Learning high-dimensional distributions is an important yet challenging problem in machine learning with applications in various domains.
Meta-Learning with Neural Tangent Kernels
We achieve this goal by 1) replacing the adaptation with a fast-adaptive regularizer in the RKHS; and 2) solving the adaptation analytically based on the NTK theory.
Meta-Learning in Reproducing Kernel Hilbert Space
Within this paradigm, we introduce two meta learning algorithms in RKHS, which no longer need an explicit inner-loop adaptation as in the MAML framework.
MixKD: Towards Efficient Distillation of Large-scale Language Models
Large-scale language models have recently demonstrated impressive empirical performance.
Learning Manifold Implicitly via Explicit Heat-Kernel Learning
Manifold learning is a fundamental problem in machine learning with numerous applications.
Graph Neural Networks with Composite Kernels
We then propose feature aggregation as the composition of the original neighbor-based kernel and a learnable kernel to encode feature similarities in a feature space.
Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitions
In this paper, we focus on skeleton-based action generation and propose to model smooth and diverse transitions on a latent space of action sequences with much lower dimensionality.
Ranked #4 on
Human action generation
on NTU RGB+D 2D
KernelNet: A Data-Dependent Kernel Parameterization for Deep Generative Modeling
Learning with kernels is an important concept in machine learning.
