Search Results for author:
Found 33 papers, 9 papers with code
A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition
In this paper, we investigate this contrasting phenomenon from the perspective of modality bias and reveal that an excessive modality bias on the audio caused by dropout is the underlying reason.
Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning
Our framework evaluates the specificity of attributes by considering the diversity of objects they apply to and their related context.
Towards Causal Relationship in Indefinite Data: Baseline Model and New Datasets
These highpoints make the probabilistic model capable of overcoming challenges brought by the coexistence of multi-structure data and multi-value representations and pave the way for the extension of latent confounders.
Discrete Messages Improve Communication Efficiency among Isolated Intelligent Agents
Individuals, despite having varied life experiences and learning processes, can communicate effectively through languages.
CASR: Refining Action Segmentation via Marginalizing Frame-levle Causal Relationships
CASR works out by reducing the difference in the causal adjacency matrix between we constructed and pre-segmentation results of backbone models.
A Review and Roadmap of Deep Causal Model from Different Causal Structures and Representations
The fusion of causal models with deep learning introducing increasingly intricate data sets, such as the causal associations within images or between textual components, has surfaced as a focal research area.
SSL Framework for Causal Inconsistency between Structures and Representations
The cross-pollination of deep learning and causal discovery has catalyzed a burgeoning field of research seeking to elucidate causal relationships within non-statistical data forms like images, videos, and text.
Large Language Models in Finance: A Survey
In this paper, we provide a practical survey focused on two key aspects of utilizing LLMs for financial tasks: existing solutions and guidance for adoption.
The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction
This pioneering effort aims to set the first benchmark for the AVTSE task, offering fresh insights into enhancing the ac-curacy of back-end speech recognition systems through AVTSE in challenging and real acoustic environments.
Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023
Three different structures based on attention-guided feature gathering (AFG) are designed for deep feature fusion.
The USTC-NERCSLIP Systems for the CHiME-7 DASR Challenge
This technical report details our submission system to the CHiME-7 DASR Challenge, which focuses on speaker diarization and speech recognition under complex multi-speaker scenarios.
Improving Audio-Visual Speech Recognition by Lip-Subword Correlation Based Visual Pre-training and Cross-Modal Fusion Encoder
In this paper, we propose two novel techniques to improve audio-visual speech recognition (AVSR) under a pre-training and fine-tuning training framework.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+2
Semi-supervised multi-channel speaker diarization with cross-channel attention
Most neural speaker diarization systems rely on sufficient manual training data labels, which are hard to collect under real-world scenarios.
Learning a Structural Causal Model for Intuition Reasoning in Conversation
Reasoning, a crucial aspect of NLP research, has not been adequately addressed by prevailing models including Large Language Model.
On the Importance of Backbone to the Adversarial Robustness of Object Detectors
We argue that using adversarially pre-trained backbone networks is essential for enhancing the adversarial robustness of object detectors.
Towards Causal Representation Learning and Deconfounding from Indefinite Data
We implement the above designs as a dynamic variational inference model, tailored to learn causal representation from indefinite data under latent confounding.
How to Enhance Causal Discrimination of Utterances: A Case on Affective Reasoning
noise terms into the conversation process, thereby constructing a structural causal model (SCM).
An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuits
Then, inspired by the large number of connections between cortical regions and the thalamus, the model fuses the auditory and visual information in a thalamic subnetwork through top-down connections.
Ranked #3 on
Speech Separation
on VoxCeleb2
Dual adaptive training of photonic neural networks
Photonic neural network (PNN) is a remarkable analog artificial intelligence (AI) accelerator that computes with photons instead of electrons to feature low latency, high energy efficiency, and high parallelism.
Progressive Multi-Scale Self-Supervised Learning for Speech Recognition
Self-supervised learning (SSL) models have achieved considerable improvements in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Improved Speech Pre-Training with Supervision-Enhanced Acoustic Unit
Speech pre-training has shown great success in learning useful and general latent representations from large-scale unlabeled data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Optical multi-task learning using multi-wavelength diffractive deep neural networks
By encoding multi-task inputs into multi-wavelength channels, the system can increase the computing throughput and significantly alle-viate the competition to perform multiple tasks in parallel with high accuracy.
Deep Learning Based Audio-Visual Multi-Speaker DOA Estimation Using Permutation-Free Loss Function
In this paper, we propose a deep learning based multi-speaker direction of arrival (DOA) estimation with audio and visual signals by using permutation-free loss function.
Optical Neural Ordinary Differential Equations
Increasing the layer number of on-chip photonic neural networks (PNNs) is essential to improve its model performance.
A Review and Roadmap of Deep Learning Causal Discovery in Different Variable Paradigms
Understanding causality helps to structure interventions to achieve specific goals and enables predictions under interventions.
Learning a General Clause-to-Clause Relationships for Enhancing Emotion-Cause Pair Extraction
To learn it applicably, we propose a general clause-level encoding model named EA-GAT comprising E-GAT and Activation Sort.
Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation
Tremendous efforts have been made on instance segmentation but the mask quality is still not satisfactory.
Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus
Federated learning (FL) is a promising distributed learning solution that only exchanges model parameters without revealing raw data.
Lip-reading with Hierarchical Pyramidal Convolution and Self-Attention
In this paper, we propose a novel deep learning architecture to improving word-level lip-reading.
Modified EP MIMO Detection Algorithm with Deep Learning Parameters Selection
According to the influence of the moment matching and parameter selection for the performance of the EP MIMO detector, we propose a modified EP MIMO detector (MEPD).
Correlating Subword Articulation with Lip Shapes for Embedding Aware Audio-Visual Speech Enhancement
We first extract visual embedding from lip frames using a pre-trained phone or articulation place recognizer for visual-only EASE (VEASE).
What does the language of foods say about us?
Second, we investigate the effect of socioeconomic factors (income, poverty, and education) on predicting state-level T2DM rates.
Assessment of central serous chorioretinopathy (CSC) depicted on color fundus photographs using deep Learning
To investigate whether and to what extent central serous chorioretinopathy (CSC) depicted on color fundus photographs can be assessed using deep learning technology.
