Search Results for author:
Found 33 papers, 13 papers with code
Blocks as Probes: Dissecting Categorization Ability of Large Multimodal Models
Categorization, a core cognitive ability in humans that organizes objects based on common features, is essential to cognitive science as well as computer vision.
GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI
Large Vision-Language Models (LVLMs) are capable of handling diverse data types such as imaging, text, and physiological signals, and can be applied in various fields.
AppAgent v2: Advanced Agent for Flexible Mobile Interactions
In the deployment phase, RAG technology enables efficient retrieval and update from this knowledge base, thereby empowering the agent to perform tasks effectively and accurately.
LinguaLinker: Audio-Driven Portraits Animation with Implicit Facial Control Enhancement
This study delves into the intricacies of synchronizing facial dynamics with multilingual audio inputs, focusing on the creation of visually compelling, time-synchronized animations through diffusion-based techniques.
EMMA: Your Text-to-Image Diffusion Model Can Secretly Accept Multi-Modal Prompts
Recent advancements in image generation have enabled the creation of high-quality images from text conditions.
MeshXL: Neural Coordinate Field for Generative 3D Foundation Models
The polygon mesh representation of 3D data exhibits great flexibility, fast rendering speed, and storage efficiency, which is widely preferred in various applications.
ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment
Diffusion models have demonstrated remarkable performance in the domain of text-to-image generation.
Computing Threshold Circuits with Void Reactions in Step Chemical Reaction Networks
We introduce a new model of \emph{step} Chemical Reaction Networks (step CRNs), motivated by the step-wise addition of materials in standard lab procedures.
Generate Like Experts: Multi-Stage Font Generation by Incorporating Font Transfer Process into Diffusion Models
Based on this observation we generalize diffusion methods to model font generative process by separating the reverse diffusion process into three stages with different functions: The structure construction stage first generates the structure information for the target character based on the source image and the font transfer stage subsequently transforms the source font to the target font.
AppAgent: Multimodal Agents as Smartphone Users
Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks.
Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models
This paper presents Paint3D, a novel coarse-to-fine generative framework that is capable of producing high-resolution, lighting-less, and diverse 2K UV texture maps for untextured 3D meshes conditioned on text or image inputs.
FaceStudio: Put Your Face Everywhere in Seconds
This study investigates identity-preserving image synthesis, an intriguing task in image generation that seeks to maintain a subject's identity while adding a personalized, stylistic touch.
ChartLlama: A Multimodal LLM for Chart Understanding and Generation
Next, we introduce ChartLlama, a multi-modal large language model that we've trained using our created dataset.
SAM-Med3D
These issues can hardly be addressed by fine-tuning SAM on medical data because the original 2D structure of SAM neglects 3D spatial information.
Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering
In this paper, we propose a learning framework that trains models to predict geometry-preserving depth without requiring extra data or annotations.
Deformation Robust Text Spotting with Geometric Prior
Based on this database, we develop a deformation robust text spotting method (DR TextSpotter) to solve the recognition problem of complex deformation of characters in different fonts.
StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data
However, these datasets often exhibit domain bias, potentially constraining the generative capabilities of the models.
Ranked #78 on
Visual Question Answering
on MM-Vet
Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representation
We present a novel alignment-before-generation approach to tackle the challenging task of generating general 3D shapes based on 2D images or texts.
StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation
The recent advancements in image-text diffusion models have stimulated research interest in large-scale 3D generative models.
Neural Transformation Fields for Arbitrary-Styled Font Generation
Few-shot font generation (FFG), aiming at generating font images with a few samples, is an emerging topic in recent years due to the academic and commercial values.
Executing your Commands via Motion Diffusion in Latent Space
We study a challenging task, conditional human motion generation, which produces plausible human motion sequences according to various conditional inputs, such as action classes or textual descriptors.
Ranked #2 on
Motion Synthesis
on HumanAct12
Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report
Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks.
Learning Variational Motion Prior for Video-based Motion Capture
To improve the generalization capacity of prior space, we propose a transformer-based variational autoencoder pretrained over marker-based 3D mocap data, with a novel style-mapping block to boost the generation quality.
Hierarchical Normalization for Robust Monocular Depth Estimation
In this paper, we address monocular depth estimation with deep neural networks.
GenText: Unsupervised Artistic Text Generation via Decoupled Font and Texture Manipulation
Specifically, our current work incorporates three different stages, stylization, destylization, and font transfer, respectively, into a unified platform with a single powerful encoder network and two separate style generator networks, one for font transfer, the other for stylization and destylization.
Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior
In the asymmetric codec, we adopt a mixed multi-path residual block (MMRB) to gradually extract weak texture features of input images, which can better preserve the original facial features and avoid excessive fantasy.
Fine-grained Identity Preserving Landmark Synthesis for Face Reenactment
Recent face reenactment works are limited by the coarse reference landmarks, leading to unsatisfactory identity preserving performance due to the distribution gap between the manipulated landmarks and those sampled from a real person.
Shuffle Transformer with Feature Alignment for Video Face Parsing
This is a short technical report introducing the solution of the Team TCParser for Short-video Face Parsing Track of The 3rd Person in Context (PIC) Workshop and Challenge at CVPR 2021.
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer
In this work, we revisit the spatial shuffle as an efficient way to build connections among windows.
Ranked #45 on
Semantic Segmentation
on ADE20K val
Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report
While many solutions have been proposed for this task, they are usually very computationally expensive and thus are not applicable for on-device inference.
Software-Defined Edge Computing: A New Architecture Paradigm to Support IoT Data Analysis
We further present an experiment on data anomaly detection in this architecture, and the comparison between two architectures for ECG diagnosis.
A Survey on Complex Question Answering over Knowledge Base: Recent Advances and Challenges
Question Answering (QA) over Knowledge Base (KB) aims to automatically answer natural language questions via well-structured relation information between entities stored in knowledge bases.
Deep & Cross Network for Ad Click Predictions
Feature engineering has been the key to the success of many prediction models.
