Search Results for author:
Found 25 papers, 5 papers with code
The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction
This pioneering effort aims to set the first benchmark for the AVTSE task, offering fresh insights into enhancing the ac-curacy of back-end speech recognition systems through AVTSE in challenging and real acoustic environments.
Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023
Three different structures based on attention-guided feature gathering (AFG) are designed for deep feature fusion.
The USTC-NERCSLIP Systems for the CHiME-7 DASR Challenge
This technical report details our submission system to the CHiME-7 DASR Challenge, which focuses on speaker diarization and speech recognition under complex multi-speaker settings.
Improving Audio-Visual Speech Recognition by Lip-Subword Correlation Based Visual Pre-training and Cross-Modal Fusion Encoder
In this paper, we propose two novel techniques to improve audio-visual speech recognition (AVSR) under a pre-training and fine-tuning training framework.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+2
Semi-supervised multi-channel speaker diarization with cross-channel attention
Most neural speaker diarization systems rely on sufficient manual training data labels, which are hard to collect under real-world scenarios.
Learning a Structural Causal Model for Intuition Reasoning in Conversation
Reasoning, a crucial aspect of NLP research, has not been adequately addressed by prevailing models including Large Language Model.
On the Importance of Backbone to the Adversarial Robustness of Object Detectors
We argue that using adversarially pre-trained backbone networks is essential for enhancing the adversarial robustness of object detectors.
Affective Reasoning at Utterance Level in Conversations: A Causal Discovery Approach
It is a novel causal discovery method showing how to discover causal relationships in a conversation via designing a common skeleton and generating a substitute for implicit causes.
Towards Causal Representation Learning and Deconfounding from Indefinite Data
We implement the above designs as a dynamic variational inference model, tailored to learn causal representation from indefinite data under latent confounding.
An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuits
Then, inspired by the large number of connections between cortical regions and the thalamus, the model fuses the auditory and visual information in a thalamic subnetwork through top-down connections.
Ranked #1 on
Speech Separation
on VoxCeleb2
Dual adaptive training of photonic neural networks
Photonic neural network (PNN) is a remarkable analog artificial intelligence (AI) accelerator that computes with photons instead of electrons to feature low latency, high energy efficiency, and high parallelism.
Progressive Multi-Scale Self-Supervised Learning for Speech Recognition
Self-supervised learning (SSL) models have achieved considerable improvements in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improved Speech Pre-Training with Supervision-Enhanced Acoustic Unit
Speech pre-training has shown great success in learning useful and general latent representations from large-scale unlabeled data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Optical multi-task learning using multi-wavelength diffractive deep neural networks
By encoding multi-task inputs into multi-wavelength channels, the system can increase the computing throughput and significantly alle-viate the competition to perform multiple tasks in parallel with high accuracy.
Deep Learning Based Audio-Visual Multi-Speaker DOA Estimation Using Permutation-Free Loss Function
In this paper, we propose a deep learning based multi-speaker direction of arrival (DOA) estimation with audio and visual signals by using permutation-free loss function.
Optical Neural Ordinary Differential Equations
Increasing the layer number of on-chip photonic neural networks (PNNs) is essential to improve its model performance.
A Review and Roadmap of Deep Learning Causal Discovery in Different Variable Paradigms
Understanding causality helps to structure interventions to achieve specific goals and enables predictions under interventions.
Learning a General Clause-to-Clause Relationships for Enhancing Emotion-Cause Pair Extraction
To learn it applicably, we propose a general clause-level encoding model named EA-GAT comprising E-GAT and Activation Sort.
Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation
Tremendous efforts have been made on instance segmentation but the mask quality is still not satisfactory.
Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus
Federated learning (FL) is a promising distributed learning solution that only exchanges model parameters without revealing raw data.
Lip-reading with Hierarchical Pyramidal Convolution and Self-Attention
In this paper, we propose a novel deep learning architecture to improving word-level lip-reading.
Modified EP MIMO Detection Algorithm with Deep Learning Parameters Selection
According to the influence of the moment matching and parameter selection for the performance of the EP MIMO detector, we propose a modified EP MIMO detector (MEPD).
Correlating Subword Articulation with Lip Shapes for Embedding Aware Audio-Visual Speech Enhancement
We first extract visual embedding from lip frames using a pre-trained phone or articulation place recognizer for visual-only EASE (VEASE).
What does the language of foods say about us?
Second, we investigate the effect of socioeconomic factors (income, poverty, and education) on predicting state-level T2DM rates.
Assessment of central serous chorioretinopathy (CSC) depicted on color fundus photographs using deep Learning
To investigate whether and to what extent central serous chorioretinopathy (CSC) depicted on color fundus photographs can be assessed using deep learning technology.
