Search Results for author:
Found 43 papers, 14 papers with code
Unveiling Language Skills under Circuits
Our experiments on various datasets confirm the correspondence between our identified skill paths and language skills, and validate three longstanding hypotheses: 1) Language skills are identifiable through circuit dissection; 2) Simple language skills reside in shallow layers, whereas complex language skills are found in deeper layers; 3) Complex language skills are formed on top of simpler language skills.
Deep CLAS: Deep Contextual Listen, Attend and Spell
In this work, we propose deep CLAS to use contextual information better.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Lightweight Transducer Based on Frame-Level Criterion
This enables the lightweight transducer achieving similar results to transducer.
Ranked #7 on
Speech Recognition
on AISHELL-1
Data Processing Techniques for Modern Multimodal Models
we provide an comprehensive review of common data processing techniques used in modern multimodal model training with a focus on diffusion models and multimodal large language models (MLLMs).
Large Language Model Agent in Financial Trading: A Survey
In this survey, we provide a comprehensive review of the current research on using LLMs as agents in financial trading.
Individualized Federated Learning for Traffic Prediction with Error Driven Aggregation
Low-latency traffic prediction is vital for smart city traffic management.
Balance of Number of Embedding and their Dimensions in Vector Quantization
The dimensionality of the embedding and the number of available embeddings ( also called codebook size) are critical factors influencing the performance of Vector Quantization(VQ), a discretization process used in many models such as the Vector Quantized Variational Autoencoder (VQ-VAE) architecture.
Super-resolution imaging using super-oscillatory diffractive neural networks
Here, we propose an optical super-oscillatory diffractive neural network, i. e., SODNN, that can achieve super-resolved spatial resolution for imaging beyond the diffraction limit with superior performance over existing methods.
Quantifying Emergence in Large Language Models
Empirical results show that (1) our method gives consistent measurements which align with existing observations based on performance metrics, validating the effectiveness of our emergence quantification; (2) our proposed metric uncovers novel emergence patterns such as the correlations between the variance of our metric and the number of ``shots'' in ICL, which further suggests a new way of interpreting hallucinations in LLMs; (3) we offer a potential solution towards estimating the emergence of larger and closed-resource LMs via smaller LMs like GPT-2.
RFL-CDNet: Towards Accurate Change Detection via Richer Feature Learning
The C2FG module aims to seamlessly integrate the side prediction from the previous coarse-scale into the current fine-scale prediction in a coarse-to-fine manner, while LF module assumes that the contribution of each stage and each spatial location is independent, thus designing a learnable module to fuse multiple predictions.
Ranked #13 on
Change Detection
on WHU-CD
A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition
In this paper, we investigate this contrasting phenomenon from the perspective of modality bias and reveal that an excessive modality bias on the audio caused by dropout is the underlying reason.
Context-based and Diversity-driven Specificity in Compositional Zero-Shot Learning
Our framework evaluates the specificity of attributes by considering the diversity of objects they apply to and their related context.
Towards Causal Relationship in Indefinite Data: Baseline Model and New Datasets
These highpoints make the probabilistic model capable of overcoming challenges brought by the coexistence of multi-structure data and multi-value representations and pave the way for the extension of latent confounders.
Discrete Messages Improve Communication Efficiency among Isolated Intelligent Agents
Individuals, despite having varied life experiences and learning processes, can communicate effectively through languages.
CASR: Refining Action Segmentation via Marginalizing Frame-levle Causal Relationships
CASR works out by reducing the difference in the causal adjacency matrix between we constructed and pre-segmentation results of backbone models.
A Review and Roadmap of Deep Causal Model from Different Causal Structures and Representations
The fusion of causal models with deep learning introducing increasingly intricate data sets, such as the causal associations within images or between textual components, has surfaced as a focal research area.
SSL Framework for Causal Inconsistency between Structures and Representations
The cross-pollination of deep learning and causal discovery has catalyzed a burgeoning field of research seeking to elucidate causal relationships within non-statistical data forms like images, videos, and text.
Large Language Models in Finance: A Survey
In this paper, we provide a practical survey focused on two key aspects of utilizing LLMs for financial tasks: existing solutions and guidance for adoption.
The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction
This pioneering effort aims to set the first benchmark for the AVTSE task, offering fresh insights into enhancing the ac-curacy of back-end speech recognition systems through AVTSE in challenging and real acoustic environments.
Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023
Three different structures based on attention-guided feature gathering (AFG) are designed for deep feature fusion.
The USTC-NERCSLIP Systems for the CHiME-7 DASR Challenge
This technical report details our submission system to the CHiME-7 DASR Challenge, which focuses on speaker diarization and speech recognition under complex multi-speaker scenarios.
Improving Audio-Visual Speech Recognition by Lip-Subword Correlation Based Visual Pre-training and Cross-Modal Fusion Encoder
In this paper, we propose two novel techniques to improve audio-visual speech recognition (AVSR) under a pre-training and fine-tuning training framework.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+2
Semi-supervised multi-channel speaker diarization with cross-channel attention
Most neural speaker diarization systems rely on sufficient manual training data labels, which are hard to collect under real-world scenarios.
Learning a Structural Causal Model for Intuition Reasoning in Conversation
Reasoning, a crucial aspect of NLP research, has not been adequately addressed by prevailing models including Large Language Model.
On the Importance of Backbone to the Adversarial Robustness of Object Detectors
We argue that using adversarially pre-trained backbone networks is essential for enhancing the adversarial robustness of object detectors.
How to Enhance Causal Discrimination of Utterances: A Case on Affective Reasoning
noise terms into the conversation process, thereby constructing a structural causal model (SCM).
Towards Causal Representation Learning and Deconfounding from Indefinite Data
We implement the above designs as a dynamic variational inference model, tailored to learn causal representation from indefinite data under latent confounding.
An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuits
Then, inspired by the large number of connections between cortical regions and the thalamus, the model fuses the auditory and visual information in a thalamic subnetwork through top-down connections.
Ranked #3 on
Speech Separation
on VoxCeleb2
Dual adaptive training of photonic neural networks
Photonic neural network (PNN) is a remarkable analog artificial intelligence (AI) accelerator that computes with photons instead of electrons to feature low latency, high energy efficiency, and high parallelism.
Improved Speech Pre-Training with Supervision-Enhanced Acoustic Unit
Speech pre-training has shown great success in learning useful and general latent representations from large-scale unlabeled data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Progressive Multi-Scale Self-Supervised Learning for Speech Recognition
Self-supervised learning (SSL) models have achieved considerable improvements in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Optical multi-task learning using multi-wavelength diffractive deep neural networks
By encoding multi-task inputs into multi-wavelength channels, the system can increase the computing throughput and significantly alle-viate the competition to perform multiple tasks in parallel with high accuracy.
Deep Learning Based Audio-Visual Multi-Speaker DOA Estimation Using Permutation-Free Loss Function
In this paper, we propose a deep learning based multi-speaker direction of arrival (DOA) estimation with audio and visual signals by using permutation-free loss function.
Optical Neural Ordinary Differential Equations
Increasing the layer number of on-chip photonic neural networks (PNNs) is essential to improve its model performance.
A Review and Roadmap of Deep Learning Causal Discovery in Different Variable Paradigms
Understanding causality helps to structure interventions to achieve specific goals and enables predictions under interventions.
Learning a General Clause-to-Clause Relationships for Enhancing Emotion-Cause Pair Extraction
To learn it applicably, we propose a general clause-level encoding model named EA-GAT comprising E-GAT and Activation Sort.
Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation
Tremendous efforts have been made on instance segmentation but the mask quality is still not satisfactory.
Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus
Federated learning (FL) is a promising distributed learning solution that only exchanges model parameters without revealing raw data.
Lip-reading with Hierarchical Pyramidal Convolution and Self-Attention
In this paper, we propose a novel deep learning architecture to improving word-level lip-reading.
Modified EP MIMO Detection Algorithm with Deep Learning Parameters Selection
According to the influence of the moment matching and parameter selection for the performance of the EP MIMO detector, we propose a modified EP MIMO detector (MEPD).
Correlating Subword Articulation with Lip Shapes for Embedding Aware Audio-Visual Speech Enhancement
We first extract visual embedding from lip frames using a pre-trained phone or articulation place recognizer for visual-only EASE (VEASE).
What does the language of foods say about us?
Second, we investigate the effect of socioeconomic factors (income, poverty, and education) on predicting state-level T2DM rates.
Assessment of central serous chorioretinopathy (CSC) depicted on color fundus photographs using deep Learning
To investigate whether and to what extent central serous chorioretinopathy (CSC) depicted on color fundus photographs can be assessed using deep learning technology.
