Search Results for author:
Found 59 papers, 15 papers with code
A Tree-Structured Decoder for Image-to-Markup Generation
Recent encoder-decoder approaches typically employ string decoders to convert images into serialized strings for image-to-markup.
Continuous Modeling of the Denoising Process for Speech Enhancement Based on Deep Learning
The starting state is noisy speech and the ending state is clean speech.
Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding with Sequence-to-Sequence Architecture
We propose a novel neural speaker diarization system using memory-aware multi-speaker embedding with sequence-to-sequence architecture (NSD-MS2S), which integrates the strengths of memory-aware multi-speaker embedding (MA-MSE) and sequence-to-sequence (Seq2Seq) architecture, leading to improvement in both efficiency and performance.
The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction
This pioneering effort aims to set the first benchmark for the AVTSE task, offering fresh insights into enhancing the ac-curacy of back-end speech recognition systems through AVTSE in challenging and real acoustic environments.
Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023
Three different structures based on attention-guided feature gathering (AFG) are designed for deep feature fusion.
The USTC-NERCSLIP Systems for the CHiME-7 DASR Challenge
This technical report details our submission system to the CHiME-7 DASR Challenge, which focuses on speaker diarization and speech recognition under complex multi-speaker settings.
Improving Audio-Visual Speech Recognition by Lip-Subword Correlation Based Visual Pre-training and Cross-Modal Fusion Encoder
In this paper, we propose two novel techniques to improve audio-visual speech recognition (AVSR) under a pre-training and fine-tuning training framework.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+2
Count, Decode and Fetch: A New Approach to Handwritten Chinese Character Error Correction
This poses a challenge when dealing with an unseen misspelled character, as the decoder may generate an IDS sequence that matches a seen character instead.
Semi-supervised multi-channel speaker diarization with cross-channel attention
Most neural speaker diarization systems rely on sufficient manual training data labels, which are hard to collect under real-world scenarios.
Variance-Preserving-Based Interpolation Diffusion Models for Speech Enhancement
The goal of this study is to implement diffusion models for speech enhancement (SE).
HRDoc: Dataset and Baseline Method Toward Hierarchical Reconstruction of Document Structures
Moreover, we proposed an encoder-decoder-based hierarchical document structure parsing system (DSPS) to tackle this problem.
SEMv2: Table Separation Line Detection Based on Conditional Convolution
Table structure recognition is an indispensable element for enabling machines to comprehend tables.
Multimodal Tree Decoder for Table of Contents Extraction in Document Images
Next, to parse the hierarchical relationship between the heading entities, a tree-structured decoder is designed.
Gradient and Channel Aware Dynamic Scheduling for Over-the-Air Computation in Federated Edge Learning Systems
To further improve the efficiency of wireless data aggregation and model learning, over-the-air computation (AirComp) is emerging as a promising solution by using the superposition characteristics of wireless channels.
Deep Learning Based Audio-Visual Multi-Speaker DOA Estimation Using Permutation-Free Loss Function
In this paper, we propose a deep learning based multi-speaker direction of arrival (DOA) estimation with audio and visual signals by using permutation-free loss function.
Convergence Theory of Generalized Distributed Subgradient Method with Random Quantization
Compared with perfect data, quantization poses fundamental challenges on loss of data accuracy, which further impacts the convergence of the algorithms.
Multimodal Pre-training Based on Graph Attention Network for Document Understanding
Its main task is to automatically read, understand, and analyze documents.
A study on joint modeling and data augmentation of multi-modalities for audio-visual scene classification
In this paper, we propose two techniques, namely joint modeling and data augmentation, to improve system performances for audio-visual scene classification (AVSC).
A Study of Designing Compact Audio-Visual Wake Word Spotting System Based on Iterative Fine-Tuning in Neural Network Pruning
Audio-only-based wake word spotting (WWS) is challenging under noisy conditions due to environmental interference in signal transmission.
The USTC-Ximalaya system for the ICASSP 2022 multi-channel multi-party meeting transcription (M2MeT) challenge
We propose two improvements to target-speaker voice activity detection (TS-VAD), the core component in our proposed speaker diarization system that was submitted to the 2022 Multi-Channel Multi-Party Meeting Transcription (M2MeT) challenge.
Underwater Differential Game: Finite-Time Target Hunting Task with Communication Delay
Simulations show that underwater disturbances have a large impact on the system considering communication delay.
SDN-based Resource Allocation in Edge and Cloud Computing Systems: An Evolutionary Stackelberg Differential Game Approach
Based on this dynamic access model, a Stackelberg differential game based cloud computing resource sharing mechanism is proposed to facilitate the resource trading between the cloud computing service provider (CCP) and different edge computing service providers (ECPs).
Target-speaker Voice Activity Detection with Improved I-Vector Estimation for Unknown Number of Speaker
Target-speaker voice activity detection (TS-VAD) has recently shown promising results for speaker diarization on highly overlapped speech.
Split, embed and merge: An accurate table structure recognizer
However, due to the complexity and diversity in their structure and style, it is very difficult to parse the tabular data into the structured format which machines can understand easily, especially for complex tables.
Ranked #7 on
Table Recognition
on PubTabNet
Separation Guided Speaker Diarization in Realistic Mismatched Conditions
We propose a separation guided speaker diarization (SGSD) approach by fully utilizing a complementarity of speech separation and speaker clustering.
A Lottery Ticket Hypothesis Framework for Low-Complexity Device-Robust Neural Acoustic Scene Classification
We propose a novel neural model compression strategy combining data augmentation, knowledge transfer, pruning, and quantization for device-robust acoustic scene classification (ASC).
USTC-NELSLIP System Description for DIHARD-III Challenge
This system description describes our submission system to the Third DIHARD Speech Diarization Challenge.
Lip-reading with Hierarchical Pyramidal Convolution and Self-Attention
In this paper, we propose a novel deep learning architecture to improving word-level lip-reading.
Exploring Emotion Features and Fusion Strategies for Audio-Video Emotion Recognition
The audio-video based emotion recognition aims to classify a given video into basic emotions.
Facial Expression Recognition (FER)
Video Emotion Recognition
The Third DIHARD Diarization Challenge
DIHARD III was the third in a series of speaker diarization challenges intended to improve the robustness of diarization systems to variability in recording equipment, noise conditions, and conversational domain.
Frequency Gating: Improved Convolutional Neural Networks for Speech Enhancement in the Time-Frequency Domain
One of the strengths of traditional convolutional neural networks (CNNs) is their inherent translational invariance.
A Two-Stage Approach to Device-Robust Acoustic Scene Classification
To improve device robustness, a highly desirable key feature of a competitive data-driven acoustic scene classification (ASC) system, a novel two-stage system based on fully convolutional neural networks (CNNs) is proposed.
Ranked #1 on
Acoustic Scene Classification
on TAU Urban Acoustic Scenes 2019
(using extra training data)
Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis
Multi-speaker speech recognition of unsegmented recordings has diverse applications such as meeting transcription and automatic subtitle generation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
A Noise-Aware Memory-Attention Network Architecture for Regression-Based Speech Enhancement
We propose a novel noise-aware memory-attention network (NAMAN) for regression-based speech enhancement, aiming at improving quality of enhanced speech in unseen noise conditions.
Correlating Subword Articulation with Lip Shapes for Embedding Aware Audio-Visual Speech Enhancement
We first extract visual embedding from lip frames using a pre-trained phone or articulation place recognizer for visual-only EASE (VEASE).
On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression
In this paper, we exploit the properties of mean absolute error (MAE) as a loss function for the deep neural network (DNN) based vector-to-vector regression.
Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network Based Vector-to-Vector Regression
In this paper, we show that, in vector-to-vector regression utilizing deep neural networks (DNNs), a generalized loss of mean absolute error (MAE) between the predicted and expected feature vectors is upper bounded by the sum of an approximation error, an estimation error, and an optimization error.
An Acoustic Segment Model Based Segment Unit Selection Approach to Acoustic Scene Classification with Partial Utterances
In contrast to building scene models with whole utterances, the ASM-removed sub-utterances, i. e., acoustic utterances without stop acoustic segments, are then used as inputs to the AlexNet-L back-end for final classification.
Device-Robust Acoustic Scene Classification Based on Two-Stage Categorization and Data Augmentation
On Task 1b development data set, we achieve an accuracy of 96. 7\% with a model size smaller than 500KB.
Stroke Constrained Attention Network for Online Handwritten Mathematical Expression Recognition
For single-modal HMER, SCAN first employs a CNN-GRU encoder to extract point-level features from input traces in online mode and employs a CNN encoder to extract pixel-level features from input images in offline mode, then use stroke constrained information to convert them into online and offline stroke-level features.
Joint Architecture and Knowledge Distillation in CNN for Chinese Text Recognition
Finally, the knowledge distillation with multiple losses is adopted to improve performance of the compact CNN.
Speaker detection in the wild: Lessons learned from JSALT 2019
This paper presents the problems and solutions addressed at the JSALT workshop when using a single microphone for speaker detection in adverse scenarios.
Audio and Speech Processing Sound
Challenge of Spatial Cognition for Deep Learning
Given the success of the deep convolutional neural networks (DCNNs) in applications of visual recognition and classification, it would be tantalizing to test if DCNNs can also learn spatial concepts, such as straightness, convexity, left/right, front/back, relative size, aspect ratio, polygons, etc., from varied visual examples of these concepts that are simple and yet vital for spatial reasoning.
Adaptive Period Embedding for Representing Oriented Objects in Aerial Images
While almost all previous object detectors for aerial images directly regress the angle of objects, they use complex rules to calculate the angle, and their performance is limited by the rule design.
Ranked #35 on
Object Detection In Aerial Images
on DOTA
The Second DIHARD Diarization Challenge: Dataset, task, and baselines
This paper introduces the second DIHARD challenge, the second in a series of speaker diarization challenges intended to improve the robustness of diarization systems to variation in recording equipment, noise conditions, and conversational domain.
Deep Neural Network Embeddings with Gating Mechanisms for Text-Independent Speaker Verification
In this paper, gating mechanisms are applied in deep neural network (DNN) training for x-vector-based text-independent speaker verification.
Multi-Task Learning with High-Order Statistics for X-vector based Text-Independent Speaker Verification
The x-vector based deep neural network (DNN) embedding systems have demonstrated effectiveness for text-independent speaker verification.
Deep Fusion: An Attention Guided Factorized Bilinear Pooling for Audio-video Emotion Recognition
Although there is no consensus on a definition, human emotional states usually can be apperceived by auditory and visual systems.
Writer-Aware CNN for Parsimonious HMM-Based Offline Handwritten Chinese Text Recognition
Recently, the hybrid convolutional neural network hidden Markov model (CNN-HMM) has been introduced for offline handwritten Chinese text recognition (HCTR) and has achieved state-of-the-art performance.
TextMountain: Accurate Scene Text Detection via Instance Segmentation
In inference stage, each pixel at the mountain foot needs to search the path to the mountaintop and this process can be efficiently completed in parallel, yielding the efficiency of our method compared with others.
Parsimonious HMMs for Offline Handwritten Chinese Text Recognition
Recently, hidden Markov models (HMMs) have achieved promising results for offline handwritten Chinese text recognition.
DenseRAN for Offline Handwritten Chinese Character Recognition
Recently, great success has been achieved in offline handwritten Chinese character recognition by using deep learning methods.
Attention Based Fully Convolutional Network for Speech Emotion Recognition
In this paper, we present a novel attention based fully convolutional network for speech emotion recognition.
Sliding Line Point Regression for Shape Robust Scene Text Detection
Specifically, we first generate the smallest rectangular box including the text with region proposal network (RPN), then isometrically regress the points on the edge of text by using the vertically and horizontally sliding lines.
Ranked #17 on
Scene Text Detection
on SCUT-CTW1500
Trajectory-based Radical Analysis Network for Online Handwritten Chinese Character Recognition
The RNN decoder aims at generating the caption by detecting radicals and spatial structures through an attention model.
Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition
Handwritten mathematical expression recognition is a challenging problem due to the complicated two-dimensional structures, ambiguous handwriting input and variant scales of handwritten math symbols.
A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition
In this study, we present a novel end-to-end approach based on the encoder-decoder framework with the attention mechanism for online handwritten mathematical expression recognition (OHMER).
Radical analysis network for zero-shot learning in printed Chinese character recognition
Chinese characters have a huge set of character categories, more than 20, 000 and the number is still increasing as more and more novel characters continue being created.
Multi-Objective Learning and Mask-Based Post-Processing for Deep Neural Network Based Speech Enhancement
We propose a multi-objective framework to learn both secondary targets not directly related to the intended task of speech enhancement (SE) and the primary target of the clean log-power spectra (LPS) features to be used directly for constructing the enhanced speech signals.
Sound
