Search Results for author:
Found 54 papers, 29 papers with code
A Label-Aware Autoregressive Framework for Cross-Domain NER
In doing so, label information contained in the embedding vectors can be effectively transferred to the target domain, and Bi-LSTM can further model the label relationship among different domains by pre-train and then fine-tune setting.
Cross-Domain Named Entity Recognition
named-entity-recognition
+2
Towards Efficient Partially Relevant Video Retrieval with Active Moment Discovering
Partially relevant video retrieval (PRVR) is a practical yet challenging task in text-to-video retrieval, where videos are untrimmed and contain much background content.
A Survey on fMRI-based Brain Decoding for Reconstructing Multimodal Stimuli
This survey systematically reviews recent progress in fMRI-based brain decoding, focusing on stimulus reconstruction from passive brain signals.
EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answering
We introduce EgoTextVQA, a novel and rigorously constructed benchmark for egocentric QA assistance involving scene text.
AugRefer: Advancing 3D Visual Grounding via Cross-Modal Augmentation and Spatial Relation-based Referring
3D visual grounding (3DVG), which aims to correlate a natural language description with the target object within a 3D scene, is a significant yet challenging task.
Linguistics-Vision Monotonic Consistent Network for Sign Language Production
Sign Language Production (SLP) aims to generate sign videos corresponding to spoken language sentences, where the conversion of sign Glosses to Poses (G2P) is the key step.
MOL-Mamba: Enhancing Molecular Representation with Structural & Electronic Insights
Molecular representation learning plays a crucial role in various downstream tasks, such as molecular property prediction and drug design.
Prototypical Calibrating Ambiguous Samples for Micro-Action Recognition
In addition, we propose a new prototypical diversity amplification loss to strengthen the model's capacity by amplifying the differences between different prototypes.
Ranked #1 on
Micro-Action Recognition
on MA-52
Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Production
Sign-IDD incorporates a novel Iconicity Disentanglement (ID) module to bridge the gap between relative positions among joints.
Dense Audio-Visual Event Localization under Cross-Modal Consistency and Multi-Temporal Granularity Collaboration
Specifically, the CMCC module contains two branches: a cross-modal interaction branch and a temporal consistency-gated branch.
ASAP: Advancing Semantic Alignment Promotes Multi-Modal Manipulation Detecting and Grounding
We present ASAP, a new framework for detecting and grounding multi-modal media manipulation (DGM4). Upon thorough examination, we observe that accurate fine-grained cross-modal semantic alignment between the image and text is vital for accurately manipulation detection and grounding.
Multimodal Class-aware Semantic Enhancement Network for Audio-Visual Video Parsing
However, each segment may contain multiple events, resulting in semantically mixed holistic features that can lead to semantic interference during intra- or cross-modal interactions: the event semantics of one segment may incorporate semantics of unrelated events from other segments.
Patch-level Sounding Object Tracking for Audio-Visual Question Answering
The M-KPT and S-KPT modules are performed in parallel for each temporal segment, allowing balanced tracking of salient and sounding objects.
Repetitive Action Counting with Hybrid Temporal Relation Modeling
Repetitive Action Counting (RAC) aims to count the number of repetitive actions occurring in videos.
Ranked #4 on
Repetitive Action Counting
on RepCount
Moderating the Generalization of Score-based Generative Model
To fill this gap, we first examine the current 'gold standard' in Machine Unlearning (MU), i. e., re-training the model after removing the undesirable training data, and find it does not work in SGMs.
Towards Pixel-Level Prediction for Gaze Following: Benchmark and Approach
Following the gaze of other people and analyzing the target they are looking at can help us understand what they are thinking, and doing, and predict the actions that may follow.
Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observation
To address this, we propose a novel framework, Sign-D2C, that employs a conditional diffusion model to synthesize contextually smooth transition frames, enabling the seamless construction of continuous sign language sequences.
Towards Open-Vocabulary Audio-Visual Event Localization
In this paper, we advance the field by introducing the Open-Vocabulary Audio-Visual Event Localization (OV-AVEL) problem, which requires localizing audio-visual events and predicting explicit categories for both seen and unseen data at inference.
Grounding is All You Need? Dual Temporal Grounding for Video Dialog
In the realm of video dialog response generation, the understanding of video content and the temporal nuances of conversation history are paramount.
Scene-Text Grounding for Text-Based Video Question Answering
In this paper, we propose to study Grounded TextVideoQA by forcing models to answer questions and spatio-temporally localize the relevant scene-text regions, thus decoupling QA from scenetext recognition and promoting research towards interpretable QA.
Prototype Learning for Micro-gesture Classification
In this paper, we briefly introduce the solution developed by our team, HFUT-VUT, for the track of Micro-gesture Classification in the MiGA challenge at IJCAI 2024.
Label-anticipated Event Disentanglement for Audio-Visual Video Parsing
Audio-Visual Video Parsing (AVVP) task aims to detect and temporally locate events within audio and visual modalities.
PsycoLLM: Enhancing LLM for Psychological Understanding and Evaluation
Additionally, to compare the performance of PsycoLLM with other LLMs, we develop a comprehensive psychological benchmark based on authoritative psychological counseling examinations in China, which includes assessments of professional ethics, theoretical proficiency, and case analysis.
MMAD: Multi-label Micro-Action Detection in Videos
This paper specifically focuses on a subset of body actions known as micro-actions, which are subtle, low-intensity body movements with promising applications in human emotion analysis.
Micro-gesture Online Recognition using Learnable Query Points
The Micro-gesture Online Recognition task involves identifying the category and locating the start and end times of micro-gestures in video clips.
Joint Spatial-Temporal Modeling and Contrastive Learning for Self-supervised Heart Rate Measurement
This paper briefly introduces the solutions developed by our team, HFUT-VUT, for Track 1 of self-supervised heart rate measurement in the 3rd Vision-based Remote Physiological Signal Sensing (RePSS) Challenge hosted at IJCAI 2024.
Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labeling
The Audio-Visual Video Parsing task aims to identify and temporally localize the events that occur in either or both the audio and visual streams of audible videos.
The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report
In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking.
Unified Static and Dynamic Network: Efficient Temporal Filtering for Video Grounding
Inspired by the activity-silent and persistent activity mechanisms in human visual perception biology, we design a Unified Static and Dynamic Network (UniSDNet), to learn the semantic association between the video and text/audio queries in a cross-modal environment for efficient video grounding.
Training A Small Emotional Vision Language Model for Visual Art Comprehension
This paper develops small vision language models to understand visual art, which, given an art work, aims to identify its emotion category and explain this prediction with natural language.
Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture
To this end, we present FD4MM, a new paradigm of Frequency Decoupling for Motion Magnification with a Multi-level Isomorphic Architecture to capture multi-level high-frequency details and a stable low-frequency structure (motion field) in video space.
Benchmarking Micro-action Recognition: Dataset, Methods, and Applications
It offers insights into the feelings and intentions of individuals and is important for human-oriented applications such as emotion recognition and psychological assessment.
Ranked #2 on
Micro-Action Recognition
on MA-52
Data-Free Quantization via Pseudo-label Filtering
Quantization for model compression can efficiently reduce the network complexity and storage requirement but the original training data is necessary to remedy the performance loss caused by quantization.
Object-aware Adaptive-Positivity Learning for Audio-Visual Question Answering
These selected pairs are constrained to have larger similarity values than the mismatched pairs.
Audio-visual Question Answering
Audio-Visual Question Answering (AVQA)
+4
EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer
Then, we introduce a novel dynamic filter that eliminates noise cues and preserves critical features in the motion magnification and amplification generation phases.
Exploring Sparse Spatial Relation in Graph Inference for Text-Based VQA
The associations between these repetitive objects are superfluous for answer reasoning; (2) two spatially distant OCR tokens detected in the image frequently have weak semantic dependencies for answer reasoning; and (3) the co-existence of nearby objects and tokens may be indicative of important visual cues for predicting answers.
Dual-Path Temporal Map Optimization for Make-up Temporal Video Grounding
Compared with the general video grounding task, MTVG focuses on meticulous actions and changes on the face.
Exploiting Diverse Feature for Multimodal Sentiment Analysis
Specifically, we achieve the results of 0. 8492 and 0. 8439 for MuSe-Personalisation in terms of arousal and valence CCC.
Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos
Specifically, the proposed Dual-TL uses a Spatial TokenLearner (S-TL) to explore associations in different facial ROIs, which promises the rPPG prediction far away from noisy ROI disturbances.
M&M: Tackling False Positives in Mammography with a Multi-view and Multi-instance Learning Sparse Detector
Deep-learning-based object detection methods show promise for improving screening mammography, but high rates of false positives can hinder their effectiveness in clinical practice.
ViGT: Proposal-free Video Grounding with Learnable Token in Transformer
First, we employed a sharing feature encoder to project both video and query into a joint feature space before performing cross-modal co-attention (i. e., video-to-query attention and query-to-video attention) to highlight discriminative features in each modality.
Data Augmentation for Human Behavior Analysis in Multi-Person Conversations
In this paper, we present the solution of our team HFUT-VUT for the MultiMediate Grand Challenge 2023 at ACM Multimedia 2023.
Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification
In this paper, we briefly introduce the solution of our team HFUT-VUT for the Micros-gesture Classification in the MiGA challenge at IJCAI 2023.
Ranked #1 on
Micro-gesture Recognition
on iMiGUE
Improving Audio-Visual Video Parsing with Pseudo Visual Labels
We perform extensive experiments on the LLP dataset and demonstrate that our method can generate high-quality segment-level pseudo labels with the help of our newly proposed loss and the label denoising strategy.
Audio-Visual Segmentation with Semantics
To deal with these problems, we propose a new baseline method that uses a temporal pixel-wise audio-visual interaction module to inject audio semantics as guidance for the visual segmentation process.
Global Temporal Difference Network for Action Recognition
In the global aggregation module, the global prior knowledge is learned by aggregating the visual feature sequence of video into a global vector.
Ranked #62 on
Action Recognition
on Something-Something V2
Contrastive Positive Sample Propagation along the Audio-Visual Event Line
Visual and audio signals often coexist in natural environments, forming audio-visual events (AVEs).
MEGCF: Multimodal Entity Graph Collaborative Filtering for Personalized Recommendation
This paper aims at solving the mismatch problem between MFE and UIM, so as to generate high-quality embedding representations and better model multimodal user preferences.
Joint Multi-grained Popularity-aware Graph Convolution Collaborative Filtering for Recommendation
In this work, we propose to model multi-grained popularity features and jointly learn them together with high-order connectivity, to match the differentiation of user preferences exhibited in popularity features.
Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers
PF-ViT aims to separate and recognize the disturbance-agnostic emotion from a static facial image via generating its corresponding poker face, without the need for paired images.
Ranked #7 on
Facial Expression Recognition (FER)
on FER+
Audio-Visual Segmentation
To deal with the AVS problem, we propose a novel method that uses a temporal pixel-wise audio-visual interaction module to inject audio semantics as guidance for the visual segmentation process.
SAPAG: A Self-Adaptive Privacy Attack From Gradients
Distributed learning such as federated learning or collaborative learning enables model training on decentralized data from users and only collects local gradients, where data is processed close to its sources for data privacy.
Recurrent Relational Memory Network for Unsupervised Image Captioning
Unsupervised image captioning with no annotations is an emerging challenge in computer vision, where the existing arts usually adopt GAN (Generative Adversarial Networks) models.
Iterative Context-Aware Graph Inference for Visual Dialog
Visual dialog is a challenging task that requires the comprehension of the semantic dependencies among implicit visual and textual contexts.
Ranked #12 on
Visual Dialog
on VisDial v0.9 val
