Search Results for author:
Found 21 papers, 15 papers with code
WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training
We further construct a large Chinese multi-source image-text dataset called RUC-CAS-WenLan for pre-training our BriVL model.
Ranked #1 on
Image Retrieval
on RUC-CAS-WenLan
MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning
The first Multimodal Emotion Recognition Challenge (MER 2023) was successfully held at ACM Multimedia.
M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database
In this work, we propose a Multi-modal Multi-scene Multi-label Emotional Dialogue dataset, M3ED, which contains 990 dyadic emotional dialogues from 56 different TV series, a total of 9, 082 turns and 24, 449 utterances.
Cultural Vocal Bursts Intensity Prediction
Emotion Recognition
MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation
Emotion recognition in conversation (ERC) is a crucial component in affective dialogue systems, which helps the system understand users' emotions and generate empathetic responses.
Missing Modality Imagination Network for Emotion Recognition with Uncertain Missing Modalities
However, in real-world applications, we often encounter the problem of missing modality, and which modalities will be missing is uncertain.
SummPip: Unsupervised Multi-Document Summarization with Sentence Graph Compression
Obtaining training data for multi-document summarization (MDS) is time consuming and resource-intensive, so recent neural models can only be trained for limited domains.
Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities
Multimodal emotion recognition leverages complementary information across modalities to gain performance.
Towards Relation Extraction From Speech
We explore speech relation extraction via two approaches: the pipeline approach conducting text-based extraction with a pretrained ASR module, and the end2end approach via a new proposed encoder-decoder model, or what we called SpeechRE.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Multi-Task Learning Framework for Emotion Recognition in-the-wild
This paper presents our system for the Multi-Task Learning (MTL) Challenge in the 4th Affective Behavior Analysis in-the-wild (ABAW) competition.
Generating Synthetic Speech from SpokenVocab for Speech Translation
Training end-to-end speech translation (ST) systems requires sufficiently large-scale data, which is unavailable for most language pairs and domains.
It is Not as Good as You Think! Evaluating Simultaneous Machine Translation on Interpretation Data
Most existing simultaneous machine translation (SiMT) systems are trained and evaluated on offline translation corpora.
M-Adapter: Modality Adaptation for End-to-End Speech-to-Text Translation
End-to-end speech-to-text translation models are often initialized with pre-trained speech encoder and pre-trained text decoder.
Investigating Pre-trained Audio Encoders in the Low-Resource Condition
Pre-trained speech encoders have been central to pushing state-of-the-art results across various speech understanding and generation tasks.
Self-supervised Rewiring of Pre-trained Speech Encoders: Towards Faster Fine-tuning with Less Labels in Speech Processing
Pre-trained speech Transformers have facilitated great success across various speech processing tasks.
MEmoBERT: Pre-training Model with Prompt-based Learning for Multimodal Emotion Recognition
Multimodal emotion recognition study is hindered by the lack of labelled corpora in terms of scale and diversity, due to the high annotation cost and label ambiguity.
DialogueEIN: Emotion Interaction Network for Dialogue Affective Analysis
Emotion Recognition in Conversation (ERC) has attracted increasing attention in the affective computing research field.
RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise
Pre-trained speech Transformers in speech translation (ST) have facilitated state-of-the-art (SotA) results; yet, using such encoders is computationally expensive.
NAIST-SIC-Aligned: an Aligned English-Japanese Simultaneous Interpretation Corpus
Research has been limited due to the lack of a large-scale training corpus.
Simultaneous Machine Translation with Large Language Models
The results show that LLM outperforms dedicated MT models in terms of BLEU and LAAL metrics.
Towards Event Extraction from Speech with Contextual Clues
While text-based event extraction has been an active research area and has seen successful application in many domains, extracting semantic events from speech directly is an under-explored problem.
Double Mixture: Towards Continual Event Detection from Speech
To address the challenges of catastrophic forgetting and effective disentanglement, we propose a novel method, 'Double Mixture.'
