Search Results for author:
Found 57 papers, 29 papers with code
Emotion-Qwen: Training Hybrid Experts for Unified Emotion and General Vision-Language Understanding
While Large Multimodal Models (LMMs) have demonstrated significant progress in general vision-language (VL) tasks, their performance in emotion-specific scenarios remains limited.
Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models
Most existing emotion analysis emphasizes which emotion arises (e. g., happy, sad, angry) but neglects the deeper why.
MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process Errors Identification
Reasoning is an essential capacity for large language models (LLMs) to address complex tasks, where the identification of process errors is vital for improving this ability.
PEBench: A Fictitious Dataset to Benchmark Machine Unlearning for Multimodal Large Language Models
In recent years, Multimodal Large Language Models (MLLMs) have demonstrated remarkable advancements in tasks such as visual question answering, visual understanding, and reasoning.
Faster Vision Mamba is Rebuilt in Minutes via Merged Token Re-training
Re-training the token-reduced model enhances the performance of Mamba, by effectively rebuilding the key knowledge.
Is Graph Convolution Always Beneficial For Every Feature?
In prior studies, to assess the impacts of graph convolution on features, people proposed metrics based on feature homophily to measure feature consistency with the graph topology.
Rethinking Structure Learning For Graph Neural Networks
Surprisingly, our empirical observations and theoretical analysis show that no matter which type of graph structure construction methods are used, after feeding the same GSL bases to the newly constructed graph, there is no MI gain compared to the original GSL bases.
Mamba-Enhanced Text-Audio-Video Alignment Network for Emotion Recognition in Conversations
MaTAV is with the advantages of aligning unimodal features to ensure consistency across different modalities and handling long input sequences to better capture contextual multimodal information.
DPDEdit: Detail-Preserved Diffusion Models for Multimodal Fashion Image Editing
To address these challenges, we introduce a new multimodal fashion image editing architecture based on latent diffusion models, called Detail-Preserved Diffusion Models (DPDEdit).
Multimodal Multi-turn Conversation Stance Detection: A Challenge Dataset and Effective Model
To address this, we introduce a new multimodal multi-turn conversational stance detection dataset (called MmMtCSD).
FlexEdit: Marrying Free-Shape Masks to VLLM for Flexible Image Editing
Our approach employs a VLLM in comprehending the image content, mask, and user instructions.
SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognition
This paper presents our winning approach for the MER-NOISE and MER-OV tracks of the MER2024 Challenge on multimodal emotion recognition.
DSMix: Distortion-Induced Sensitivity Map Based Pre-training for No-Reference Image Quality Assessment
It applies cut and mix operations to diverse categories of synthetic distorted images, assigning confidence scores to class labels based on the aforementioned prior knowledge.
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning
Accurate emotion perception is crucial for various applications, including human-computer interaction, education, and counseling.
Dataset Growth
To tackle this challenge, we propose InfoGrowth, an efficient online algorithm for data cleaning and selection, resulting in a growing dataset that keeps up to date with awareness of cleanliness and diversity.
A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Training
Training diffusion models is always a computation-intensive task.
UMETTS: A Unified Framework for Emotional Text-to-Speech Synthesis with Multimodal Prompts
Emotional Text-to-Speech (E-TTS) synthesis has garnered significant attention in recent years due to its potential to revolutionize human-computer interaction.
Two in One Go: Single-stage Emotion Recognition with Decoupled Subject-context Transformer
Emotion recognition aims to discern the emotional state of subjects within an image, relying on subject-centric and contextual visual cues.
LEAF: Unveiling Two Sides of the Same Coin in Semi-supervised Facial Expression Recognition
LEAF introduces a hierarchical expression-aware aggregation strategy that operates at three levels: semantic, instance, and category.
Facial Expression Recognition
Facial Expression Recognition (FER)
MIPS at SemEval-2024 Task 3: Multimodal Emotion-Cause Pair Extraction in Conversations with Multimodal Language Models
This paper presents our winning submission to Subtask 2 of SemEval 2024 Task 3 on multimodal emotion cause analysis in conversations.
Invisible Gas Detection: An RGB-Thermal Cross Attention Network and A New Benchmark
The widespread use of various chemical gases in industrial processes necessitates effective measures to prevent their leakage during transportation and storage, given their high toxicity.
A Challenge Dataset and Effective Models for Conversational Stance Detection
Previous stance detection studies typically concentrate on evaluating stances within individual instances, thereby exhibiting limitations in effectively modeling multi-party discussions concerning the same specific topic, as naturally transpire in authentic social media interactions.
CaT-GNN: Enhancing Credit Card Fraud Detection via Causal Temporal Graph Neural Networks
Credit card fraud poses a significant threat to the economy.
3D Landmark Detection on Human Point Clouds: A Benchmark and A Dual Cascade Point Transformer Framework
Comparative evaluations with popular point-based methods on HPoint103 and the public dataset DHP19 demonstrate the dramatic outperformance of our D-CPT.
MIMIC: Mask Image Pre-training with Mix Contrastive Fine-tuning for Facial Expression Recognition
In addition, when compared with the domain disparity existing between face datasets and FER datasets, the divergence between general datasets and FER datasets is more pronounced.
The Snowflake Hypothesis: Training Deep GNN with One Node One Receptive field
Despite Graph Neural Networks demonstrating considerable promise in graph representation learning tasks, GNNs predominantly face significant issues with over-fitting and over-smoothing as they go deeper as models of computer vision realm.
Rail Detection: An Efficient Row-based Network and A New Benchmark
Inspired by the growth of lane detection, we propose a rail database and a row-based rail detection method.
IncepFormer: Efficient Inception Transformer with Pyramid Pooling for Semantic Segmentation
Semantic segmentation usually benefits from global contexts, fine localisation information, multi-scale features, etc.
Ranked #160 on
Image Classification
on ImageNet
(GFLOPs metric)
AU-Aware Vision Transformers for Biased Facial Expression Recognition
Studies have proven that domain bias and label bias exist in different Facial Expression Recognition (FER) datasets, making it hard to improve the performance of a specific dataset by adding other datasets.
AU-Supervised Convolutional Vision Transformers for Synthetic Facial Expression Recognition
In Learing from Synthetic Data(LSD) task, facial expression recognition (FER) methods aim to learn the representation of expression from the artificially generated data and generalise to real data.
Video-based Smoky Vehicle Detection with A Coarse-to-Fine Framework
Moreover, we present a new Coarse-to-fine Deep Smoky vehicle detection (CoDeS) framework for efficient smoky vehicle detection.
Video Frame Interpolation Based on Deformable Kernel Region
In order to solve this problem, we revisit the deformable convolution for video interpolation, which can break the fixed grid restrictions on the kernel region, making the distribution of reference points more suitable for the shape of the object, and thus warp a more accurate interpolation frame.
Stereo Matching with Cost Volume based Sparse Disparity Propagation
Stereo matching is crucial for binocular stereo vision.
Self-Ensemling for 3D Point Cloud Domain Adaption
In SEN, a student network is kept in a collaborative manner with supervised learning and self-supervised learning, and a teacher network conducts temporal consistency to learn useful representations and ensure the quality of point clouds reconstruction.
Spatial and Temporal Networks for Facial Expression Recognition in the Wild Videos
The paper describes our proposed methodology for the seven basic expression classification track of Affective Behavior Analysis in-the-wild (ABAW) Competition 2021.
An Efficient Training Approach for Very Large Scale Face Recognition
This method adopts Dynamic Class Pool (DCP) for storing and updating the identities features dynamically, which could be regarded as a substitute for the FC layer.
Ranked #1 on
Face Verification
on IJB-C
(training dataset metric)
Affordance Transfer Learning for Human-Object Interaction Detection
The proposed method can thus be used to 1) improve the performance of HOI detection, especially for the HOIs with unseen objects; and 2) infer the affordances of novel objects.
Ranked #2 on
Affordance Recognition
on HICO-DET(Unknown Concepts)
Detecting Human-Object Interaction via Fabricated Compositional Learning
With the proposed object fabricator, we are able to generate large-scale HOI samples for rare and unseen categories to alleviate the open long-tailed issues in HOI detection.
Ranked #4 on
Affordance Recognition
on HICO-DET
Unsupervised Person Re-Identification with Multi-Label Learning Guided Self-Paced Clustering
The multi-label learning module leverages a memory feature bank and assigns each image with a multi-label vector based on the similarities between the image and feature bank.
Exploring Emotion Features and Fusion Strategies for Audio-Video Emotion Recognition
The audio-video based emotion recognition aims to classify a given video into basic emotions.
Facial Expression Recognition (FER)
Video Emotion Recognition
AU-Guided Unsupervised Domain Adaptive Facial Expression Recognition
The domain diversities including inconsistent annotation and varied image collection conditions inevitably exist among different facial expression recognition (FER) datasets, which pose an evident challenge for adapting the FER model trained on one dataset to another one.
Facial Expression Recognition
Facial Expression Recognition (FER)
+1
Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition
To this end, we propose an Attention-Driven Dynamic Graph Convolutional Network (ADD-GCN) to dynamically generate a specific graph for each image.
Ranked #24 on
Multi-Label Classification
on MS-COCO
Suppressing Mislabeled Data via Grouping and Self-Attention
Specifically, this plug-and-play AFM first leverages a \textit{group-to-attend} module to construct groups and assign attention weights for group-wise samples, and then uses a \textit{mixup} module with the attention weights to interpolate massive noisy-suppressed samples.
Visual Compositional Learning for Human-Object Interaction Detection
The integration of decomposition and composition enables VCL to share object and verb features among different HOI samples and images, and to generate new interaction samples and new types of HOI, and thus largely alleviates the long-tail distribution problem and benefits low-shot or zero-shot HOI detection.
Ranked #3 on
Affordance Recognition
on HICO-DET(Unknown Concepts)
TTPP: Temporal Transformer with Progressive Prediction for Efficient Action Anticipation
Video action anticipation aims to predict future action categories from observed frames.
Suppressing Uncertainties for Large-Scale Facial Expression Recognition
Annotating a qualitative large-scale facial expression dataset is extremely difficult due to the uncertainties caused by ambiguous facial expressions, low-quality facial images, and the subjectiveness of annotators.
Facial Expression Recognition
Facial Expression Recognition (FER)
A Comprehensive Study on Temporal Modeling for Online Action Detection
Online action detection (OAD) is a practical yet challenging task, which has attracted increasing attention in recent years.
Learning Category Correlations for Multi-label Image Recognition with Graph Networks
In this paper, instead of using a pre-defined graph which is inflexible and may be sub-optimal for multi-label classification, we propose the A-GCN, which leverages the popular Graph Convolutional Networks with an Adaptive label correlation graph to model label dependencies.
Product Image Recognition with Guidance Learning and Noisy Supervision
Instead of collecting product images by labor-and time-intensive image capturing, we take advantage of the web and download images from the reviews of several e-commerce websites where the images are casually captured by consumers.
Bootstrap Model Ensemble and Rank Loss for Engagement Intensity Regression
Based on our winner solution last year, we mainly explore head features and body features with a bootstrap strategy and two novel loss functions in this paper.
Frame attention networks for facial expression recognition in videos
The feature embedding module is a deep Convolutional Neural Network (CNN) which embeds face images into feature vectors.
Ranked #3 on
Facial Expression Recognition (FER)
on CK+
(Accuracy (7 emotion) metric)
Facial Expression Recognition
Facial Expression Recognition (FER)
Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition
Extensive experiments show that our RAN and region biased loss largely improve the performance of FER with occlusion and variant pose.
Ranked #2 on
Facial Expression Recognition (FER)
on SFEW
Facial Expression Recognition
Facial Expression Recognition (FER)
Multi-region two-stream R-CNN for action detection
We propose a multi-region two-stream R-CNN model for action detection in realistic videos.
Ranked #2 on
Action Detection
on UCF Sports
Frankenstein: Learning Deep Face Representations using Small Data
To train such networks, very large training sets are needed with millions of labeled images.
Multi-View Super Vector for Action Recognition
Kernel average is then applied on these components to produce recognition result.
Bag of Visual Words and Fusion Methods for Action Recognition: Comprehensive Study and Good Practice
Many efforts have been made in each step independently in different scenarios and their effect on action recognition is still unknown.
A Study on Unsupervised Dictionary Learning and Feature Encoding for Action Classification
Many efforts have been devoted to develop alternative methods to traditional vector quantization in image domain such as sparse coding and soft-assignment.
