Search Results for author:
Found 59 papers, 41 papers with code
HA-VLN: A Benchmark for Human-Aware Navigation in Discrete-Continuous Environments with Dynamic Multi-Human Interactions, Real-World Validation, and an Open Leaderboard
Vision-and-Language Navigation (VLN) systems often focus on either discrete (panoramic) or continuous (free-motion) paradigms alone, overlooking the complexities of human-populated, dynamic environments.
MaxSup: Overcoming Representation Collapse in Label Smoothing
Label Smoothing (LS) is widely adopted to curb overconfidence in neural network predictions and enhance generalization.
A Video-grounded Dialogue Dataset and Metric for Event-driven Activities
This paper presents VDAct, a dataset for a Video-grounded Dialogue on Event-driven Activities, alongside VDEval, a session-based context evaluation metric specially designed for the task.
UCDR-Adapter: Exploring Adaptation of Pre-Trained Vision-Language Models for Universal Cross-Domain Retrieval
Second, Target Prompt Generation creates dynamic prompts by attending to masked source prompts, enabling seamless adaptation to unseen domains and classes.
StableAnimator: High-Quality Identity-Preserving Human Image Animation
During inference, we propose a novel Hamilton-Jacobi-Bellman (HJB) equation-based optimization to further enhance the face quality.
ProMQA: Question Answering Dataset for Multimodal Procedural Activity Understanding
Multimodal systems have great potential to assist humans in procedural activities, where people follow instructions to achieve their goals.
Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarios
Hopefully, more researchers will be inspired and encouraged to improve the practicality and efficacy of DD.
POPoS: Improving Efficient and Robust Facial Landmark Detection with Parallel Optimal Position Search
(2) To enhance the pseudo-range accuracy of selected anchor points, a new loss function, named multilateration anchor loss, is proposed.
DPDEdit: Detail-Preserved Diffusion Models for Multimodal Fashion Image Editing
To address these challenges, we introduce a new multimodal fashion image editing architecture based on latent diffusion models, called Detail-Preserved Diffusion Models (DPDEdit).
FlexEdit: Marrying Free-Shape Masks to VLLM for Flexible Image Editing
Our approach employs a VLLM in comprehending the image content, mask, and user instructions.
SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognition
This paper presents our winning approach for the MER-NOISE and MER-OV tracks of the MER2024 Challenge on multimodal emotion recognition.
Combo: Co-speech holistic 3D human motion generation and efficient customizable adaptation in harmony
For the former, we propose to pre-train on data regarding a fixed identity with neutral emotion, and defer the incorporation of customizable conditions (identity and emotion) to fine-tuning stage, which is boosted by our novel X-Adapter for parameter-efficient fine-tuning.
SHIELD: LLM-Driven Schema Induction for Predictive Analytics in EV Battery Supply Chain Disruptions
We present SHIELD (Schema-based Hierarchical Induction for EV supply chain Disruption), a system integrating Large Language Models (LLMs) with domain expertise for EV battery supply chain risk assessment.
Prioritize Alignment in Dataset Distillation
To achieve this, existing methods use the agent model to extract information from the target dataset and embed it into the distilled dataset.
Music2P: A Multi-Modal AI-Driven Tool for Simplifying Album Cover Design
In today's music industry, album cover design is as crucial as the music itself, reflecting the artist's vision and brand.
Robust Adaptation of Foundation Models with Black-Box Visual Prompting
SPSA-GC efficiently estimates the gradient of PTM to update the Coordinator.
MetaDesigner: Advancing Artistic Typography Through AI-Driven, User-Centric, and Multilingual WordArt Synthesis
MetaDesigner introduces a transformative framework for artistic typography synthesis, powered by Large Language Models (LLMs) and grounded in a user-centric design paradigm.
Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactions
Vision-and-Language Navigation (VLN) aims to develop embodied agents that navigate based on human instructions.
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning
Accurate emotion perception is crucial for various applications, including human-computer interaction, education, and counseling.
MotionFollower: Editing Video Motion via Lightweight Score-Guided Diffusion
In this paper, we propose MotionFollower, a lightweight score-guided diffusion model for video motion editing.
UMETTS: A Unified Framework for Emotional Text-to-Speech Synthesis with Multimodal Prompts
Emotional Text-to-Speech (E-TTS) synthesis has garnered significant attention in recent years due to its potential to revolutionize human-computer interaction.
LEAF: Unveiling Two Sides of the Same Coin in Semi-supervised Facial Expression Recognition
LEAF introduces a hierarchical expression-aware aggregation strategy that operates at three levels: semantic, instance, and category.
Facial Expression Recognition
Facial Expression Recognition (FER)
MIPS at SemEval-2024 Task 3: Multimodal Emotion-Cause Pair Extraction in Conversations with Multimodal Language Models
This paper presents our winning submission to Subtask 2 of SemEval 2024 Task 3 on multimodal emotion cause analysis in conversations.
IVAC-P2L: Leveraging Irregular Repetition Priors for Improving Video Action Counting
Our research addresses the shortfall by introducing a novel approach to VAC, called Irregular Video Action Counting (IVAC).
DyRoNet: Dynamic Routing and Low-Rank Adapters for Autonomous Driving Streaming Perception
The advancement of autonomous driving systems hinges on the ability to achieve low-latency and high-accuracy perception.
FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audio
In this paper, we abstract the process of people hearing speech, extracting meaningful cues, and creating various dynamically audio-consistent talking faces, termed Listening and Imagining, into the task of high-fidelity diverse talking faces generation from a single audio.
WordArt Designer API: User-Driven Artistic Typography Synthesis with Large Language Models on ModelScope
This paper introduces the WordArt Designer API, a novel framework for user-driven artistic typography synthesis utilizing Large Language Models (LLMs) on ModelScope.
BlockGCN: Redefine Topology Awareness for Skeleton-Based Action Recognition
To remedy this we propose a two-fold strategy: (1) We introduce an innovative approach that encodes bone connectivity by harnessing the power of graph distances to describe the physical topology; we further incorporate action-specific topological representation via persistent homology analysis to depict systemic dynamics.
Ranked #6 on
Skeleton Based Action Recognition
on NTU RGB+D 120
Tracking with Human-Intent Reasoning
The perception component then generates the tracking results based on the embeddings.
Ranked #5 on
Referring Video Object Segmentation
on ReVOS
ProS: Prompting-to-simulate Generalized knowledge for Universal Cross-Domain Retrieval
Then, in Context-aware Simulator Learning stage, we train a Content-aware Prompt Simulator under a simulated test scenarios to produce the corresponding CaDP.
MotionEditor: Editing Video Motion via Content-Aware Diffusion
This mechanism enables the editing branch to query the key and value from the reconstruction branch in a decoupled manner, making the editing branch retain the original background and protagonist appearance.
Towards Calibrated Robust Fine-Tuning of Vision-Language Models
Improving out-of-distribution (OOD) generalization during in-distribution (ID) adaptation is a primary goal of robust fine-tuning of zero-shot models beyond naive fine-tuning.
Debunking Free Fusion Myth: Online Multi-view Anomaly Detection with Disentangled Product-of-Experts Modeling
Multi-view or even multi-modal data is appealing yet challenging for real-world applications.
WordArt Designer: User-Driven Artistic Typography Synthesis using Large Language Models
This paper introduces WordArt Designer, a user-driven framework for artistic typography synthesis, relying on the Large Language Model (LLM).
DCPT: Darkness Clue-Prompted Tracking in Nighttime UAVs
To address this, we propose a novel architecture called Darkness Clue-Prompted Tracking (DCPT) that achieves robust UAV tracking at night by efficiently learning to generate darkness clue prompts.
Refined Temporal Pyramidal Compression-and-Amplification Transformer for 3D Human Pose Estimation
Accurately estimating the 3D pose of humans in video sequences requires both accuracy and a well-structured architecture.
Ranked #10 on
3D Human Pose Estimation
on HumanEva-I
PoSynDA: Multi-Hypothesis Pose Synthesis Domain Adaptation for Robust 3D Human Pose Estimation
Typically, PoSynDA uses a diffusion-inspired structure to simulate 3D pose distribution in the target domain.
Improving Anomaly Segmentation with Multi-Granularity Cross-Domain Alignment
Anomaly segmentation plays a pivotal role in identifying atypical objects in images, crucial for hazard detection in autonomous driving systems.
KeyPosS: Plug-and-Play Facial Landmark Detection through GPS-Inspired True-Range Multilateration
By spearheading the integration of Multilateration with facial analysis, KeyPosS marks a paradigm shift in facial landmark detection.
Overcoming Topology Agnosticism: Enhancing Skeleton-Based Action Recognition through Redefined Skeletal Topology Awareness
As a remedy, we propose a threefold strategy: (1) We forge an innovative pathway that encodes bone connectivity by harnessing the power of graph distances.
Implicit Temporal Modeling with Learnable Alignment for Video Recognition
While modeling temporal information within straight through tube is widely adopted in literature, we find that simple frame alignment already provides enough essence without temporal attention.
Ranked #19 on
Action Classification
on Kinetics-400
ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules
We evaluate ChartReader on Chart-to-Table, ChartQA, and Chart-to-Text tasks, demonstrating its superiority over existing methods.
DAMO-StreamNet: Optimizing Streaming Perception in Autonomous Driving
Real-time perception, or streaming perception, is a crucial aspect of autonomous driving that has yet to be thoroughly explored in existing research.
HDFormer: High-order Directed Transformer for 3D Human Pose Estimation
Human pose estimation is a challenging task due to its structured data sequence nature.
Ranked #89 on
3D Human Pose Estimation
on Human3.6M
Hypergraph Transformer for Skeleton-based Action Recognition
Skeleton-based action recognition aims to recognize human actions given human joint coordinates with skeletal interconnections.
Ranked #12 on
Skeleton Based Action Recognition
on NTU RGB+D 120
ProContEXT: Exploring Progressive Context Transformer for Tracking
Existing Visual Object Tracking (VOT) only takes the target area in the first frame as a template.
Ranked #1 on
Video Object Tracking
on NT-VOT211
LongShortNet: Exploring Temporal and Semantic Features Fusion in Streaming Perception
Streaming perception is a critical task in autonomous driving that requires balancing the latency and accuracy of the autopilot system.
GSRFormer: Grounded Situation Recognition Transformer with Alternate Semantic Attention Refinement
In the second stage, we exploit transformer layers to unearth the potential semantic relations within both verbs and semantic roles.
Rethinking Spatial Invariance of Convolutional Networks for Object Counting
We evaluate our methods on 4 mainstream object counting networks (i. e., MCNN, CSRNet, SANet, and ResNet-50).
Ranked #1 on
Object Counting
on TRANCOS
Subspace Representation Learning for Few-shot Image Classification
In this paper, we propose a subspace representation learning (SRL) framework to tackle few-shot image classification tasks.
Generating Person Images with Appearance-aware Pose Stylizer
Generation of high-quality person images is challenging, due to the sophisticated entanglements among image factors, e. g., appearance, pose, foreground, background, local details, global structures, etc.
Improving the Learning of Multi-column Convolutional Neural Network for Crowd Counting
By minimizing the mutual information, each column is guided to learn features with different image scales.
Learning Spatial Awareness to Improve Crowd Counting
Although the Maximum Excess over SubArrays (MESA) loss has been previously proposed to address the above issues by finding the rectangular subregion whose predicted density map has the maximum difference from the ground truth, it cannot be solved by gradient descent, thus can hardly be integrated into the deep learning framework.
Ranked #5 on
Crowd Counting
on WorldExpo’10
Perceiving Physical Equation by Observing Visual Scenarios
To tackle this challenge, we present a novel pipeline comprised of an Observer Engine and a Physicist Engine by respectively imitating the actions of an observer and a physicist in the real world.
Stacked Pooling: Improving Crowd Counting by Boosting Scale Invariance
In this work, we explore the cross-scale similarity in crowd counting scenario, in which the regions of different scales often exhibit high visual similarity.
GNAS: A Greedy Neural Architecture Search Method for Multi-Attribute Learning
A key problem in deep multi-attribute learning is to effectively discover the inter-attribute correlation structures.
On the Selection of Anchors and Targets for Video Hyperlinking
A principle way of hyperlinking can be carried out by picking centers of clusters as anchors and from there reach out to targets within or outside of clusters with consideration of neighborhood complexity.
Video2Shop: Exact Matching Clothes in Videos to Online Shopping Images
For the video side, deep visual features are extracted from detected object regions in each frame, and further fed into a Long Short-Term Memory (LSTM) framework for sequence modeling, which captures the temporal dynamics in videos.
Multi-View Image Generation from a Single-View
This paper addresses a challenging problem -- how to generate multi-view cloth images from only a single view input.
