Gated Multimodal Fusion with Contrastive Learning for Turn-taking Prediction in Human-robot Dialogue

no code implementations • 18 Apr 2022 • Jiudong Yang, Peiying Wang, Yi Zhu, Mingchao Feng, Meng Chen, Xiaodong He

Turn-taking, aiming to decide when the next speaker can start talking, is an essential component in building human-robot spoken dialogue systems.

Contrastive Learning Data Augmentation +2

Building Robust Spoken Language Understanding by Cross Attention between Phoneme Sequence and ASR Hypothesis

no code implementations • 22 Mar 2022 • Zexun Wang, Yuquan Le, Yi Zhu, Yuming Zhao, Mingchao Feng, Meng Chen, Xiaodong He

Building Spoken Language Understanding (SLU) robust to Automatic Speech Recognition (ASR) errors is an essential issue for various voice-enabled virtual assistants.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4