Search Results for author:
Found 31 papers, 4 papers with code
The Sogou-TIIC Speech Translation System for IWSLT 2018
Our final average result on speech translation is 31. 02 BLEU.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Nonlinear spiked covariance matrices and signal propagation in deep neural networks
Many recent works have studied the eigenvalue spectrum of the Conjugate Kernel (CK) defined by the nonlinear feature map of a feedforward neural network.
StreamVoice: Streamable Context-Aware Language Modeling for Real-time Zero-Shot Voice Conversion
Specifically, to enable streaming capability, StreamVoice employs a fully causal context-aware LM with a temporal-independent acoustic predictor, while alternately processing semantic and acoustic features at each time step of autoregression which eliminates the dependence on complete source speech.
FedRec+: Enhancing Privacy and Addressing Heterogeneity in Federated Recommendation Systems
Preserving privacy and reducing communication costs for edge users pose significant challenges in recommendation systems.
VITS-Based Singing Voice Conversion Leveraging Whisper and multi-scale F0 Modeling
This paper introduces the T23 team's system submitted to the Singing Voice Conversion Challenge 2023.
Proposing an intelligent mesh smoothing method with graph neural networks
With a lightweight model, GMSNet can effectively smoothing mesh nodes with varying degrees and remain unaffected by the order of input data.
MSM-VC: High-fidelity Source Style Transfer for Non-Parallel Voice Conversion by Multi-scale Style Modeling
In addition to conveying the linguistic content from source speech to converted speech, maintaining the speaking style of source speech also plays an important role in the voice conversion (VC) task, which is essential in many scenarios with highly expressive source speech, such as dubbing and data augmentation.
Text-to-Video: a Two-stage Framework for Zero-shot Identity-agnostic Talking-head Generation
In the second stage, an audio-driven talking head generation method is employed to produce compelling videos privided the audio generated in the first stage.
Auxiliary-Tasks Learning for Physics-Informed Neural Network-Based Partial Differential Equations Solving
To alleviate these issues, we proposed auxiliary-task learning-based physics-informed neural networks (ATL-PINNs), which provide four different auxiliary-task learning modes and investigate their performance compared with original PINNs.
LM-VC: Zero-shot Voice Conversion via Speech Generation based on Language Models
An intuitive approach is to follow AudioLM - Tokenizing speech into semantic and acoustic tokens respectively by HuBERT and SoundStream, and converting source semantic tokens to target acoustic tokens conditioned on acoustic tokens of the target speaker.
ST-PINN: A Self-Training Physics-Informed Neural Network for Partial Differential Equations
To address the issue of low accuracy and convergence problems of existing PINNs, we propose a self-training physics-informed neural network, ST-PINN.
Faithful and Efficient Explanations for Neural Networks via Neural Tangent Kernel Surrogate Models
A second trend has been to utilize kernel functions in various explain-by-example or data attribution tasks.
Multi-level Temporal-channel Speaker Retrieval for Zero-shot Voice Conversion
Specifically, to flexibly adapt to the dynamic-variant speaker characteristic in the temporal and channel axis of the speech, we propose a novel fine-grained speaker modeling method, called temporal-channel retrieval (TCR), to find out when and where speaker information appears in speech.
FedEBA+: Towards Fair and Effective Federated Learning via Entropy-Based Model
Ensuring fairness is a crucial aspect of Federated Learning (FL), which enables the model to perform consistently across all clients.
Delivering Speaking Style in Low-resource Voice Conversion with Multi-factor Constraints
Conveying the linguistic content and maintaining the source speech's speaking style, such as intonation and emotion, is essential in voice conversion (VC).
Overparameterized random feature regression with nearly orthogonal data
Our analysis shows high-probability non-asymptotic concentration results for the training errors, cross-validations, and generalization errors of RFRR centered around their respective values for a kernel ridge regression (KRR).
Expressive-VC: Highly Expressive Voice Conversion with Attention Fusion of Bottleneck and Perturbation Features
We further fuse the linguistic and para-linguistic features through an attention mechanism, where speaker-dependent prosody features are adopted as the attention query, which result from a prosody encoder with target speaker embedding and normalized pitch and energy of source speech as input.
Streaming Voice Conversion Via Intermediate Bottleneck Features And Non-streaming Teacher Guidance
In this paper, we propose to use intermediate bottleneck features (IBFs) to replace PPGs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
An Improved Structured Mesh Generation Method Based on Physics-informed Neural Networks
To improve the prediction accuracy of the neural network, we also introduce a novel auxiliary line strategy and an efficient network model during meshing.
TorchNTK: A Library for Calculation of Neural Tangent Kernels of PyTorch Models
We introduce torchNTK, a python library to calculate the empirical neural tangent kernel (NTK) of neural network models in the PyTorch framework.
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation
We study the first gradient descent step on the first-layer parameters $\boldsymbol{W}$ in a two-layer neural network: $f(\boldsymbol{x}) = \frac{1}{\sqrt{N}}\boldsymbol{a}^\top\sigma(\boldsymbol{W}^\top\boldsymbol{x})$, where $\boldsymbol{W}\in\mathbb{R}^{d\times N}, \boldsymbol{a}\in\mathbb{R}^{N}$ are randomly initialized, and the training objective is the empirical MSE loss: $\frac{1}{n}\sum_{i=1}^n (f(\boldsymbol{x}_i)-y_i)^2$.
Causal Disentanglement for Semantics-Aware Intent Learning in Recommendation
Traditional recommendation models trained on observational interaction data have generated large impacts in a wide range of applications, it faces bias problems that cover users' true intent and thus deteriorate the recommendation effectiveness.
IQDUBBING: Prosody modeling based on discrete self-supervised speech representation for expressive voice conversion
Specifically, prosody vector is first extracted from pre-trained VQ-Wav2Vec model, where rich prosody information is embedded while most speaker and environment information are removed effectively by quantization.
Multi-speaker Multi-style Text-to-speech Synthesis With Single-speaker Single-style Training Data Scenarios
Moreover, the explicit prosody features used in the prosody predicting module can increase the diversity of synthetic speech by adjusting the value of prosody features.
One-shot Voice Conversion For Style Transfer Based On Speaker Adaptation
One-shot style transfer is a challenging task, since training on one utterance makes model extremely easy to over-fit to training data and causes low speaker similarity and lack of expressiveness.
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks
As an application, we show that random feature regression induced by the empirical kernel achieves the same asymptotic performance as its limiting kernel regression under the ultra-wide regime.
Hilbert Sinkhorn Divergence for Optimal Transport
Sinkhorn divergence has become a very popular metric to compare probability distributions in optimal transport.
Enriching Source Style Transfer in Recognition-Synthesis based Non-Parallel Voice Conversion
Specifically, prosodic features are used to explicit model prosody, while VAE and reference encoder are used to implicitly model prosody, which take Mel spectrum and bottleneck feature as input respectively.
WNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognition
Recently, attention-based encoder-decoder (AED) end-to-end (E2E) models have drawn more and more attention in the field of automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks
We study the eigenvalue distributions of the Conjugate Kernel and Neural Tangent Kernel associated to multi-layer feedforward neural networks.
Polynomial Representation for Persistence Diagram
In this work, we discover a set of general polynomials that vanish on vectorized PDs and extract the task-adapted feature representation from these polynomials.
