Search Results for author:
Found 45 papers, 6 papers with code
The Sogou-TIIC Speech Translation System for IWSLT 2018
Our final average result on speech translation is 31. 02 BLEU.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Federated Unlearning with Gradient Descent and Conflict Mitigation
A steepest descent direction for unlearning is then calculated in the condition of being non-conflicting with other clients' gradients and closest to the target client's gradient.
Optimal Exact Recovery in Semi-Supervised Learning: A Study of Spectral Methods and Graph Convolutional Networks
Venturing into the transductive learning landscape, we, for the first time, pinpoint the information-theoretical threshold for the exact recovery of all test nodes in CSBM.
Rate, Explain and Cite (REC): Enhanced Explanation and Attribution in Automatic Evaluation by Large Language Models
LLMs have demonstrated impressive proficiency in generating coherent and high-quality text, making them valuable across a range of text-generation tasks.
UFT: Unifying Fine-Tuning of SFT and RLHF/DPO/UNA through a Generalized Implicit Reward Function
Due to the differing nature and objective functions of SFT and alignment, catastrophic forgetting has become a significant issue.
FedMABA: Towards Fair Federated Learning through Multi-Armed Bandits Allocation
The increasing concern for data privacy has driven the rapid development of federated learning (FL), a privacy-preserving collaborative paradigm.
GNNRL-Smoothing: A Prior-Free Reinforcement Learning Model for Mesh Smoothing
We verified the proposed model on both 2D and 3D meshes.
UNA: Unifying Alignments of RLHF/PPO, DPO and KTO by a Generalized Implicit Reward Function
DPO proposes a mapping between an optimal policy and a reward, greatly simplifying the training process of RLHF.
StreamVoice+: Evolving into End-to-end Streaming Zero-shot Voice Conversion
StreamVoice+ integrates a semantic encoder and a connector with the original StreamVoice framework, now trained using a non-streaming ASR.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Universality of kernel random matrices and kernel regression in the quadratic regime
In this regime, under certain conditions on the data distribution, the kernel random matrix involved in KRR exhibits behavior akin to that of a linear kernel.
McGAN: Generating Manufacturable Designs by Embedding Manufacturing Rules into Conditional Generative Adversarial Network
Generative design (GD) methods aim to automatically generate a wide variety of designs that satisfy functional or aesthetic design requirements.
A Comprehensive Survey of LLM Alignment Techniques: RLHF, RLAIF, PPO, DPO and More
With advancements in self-supervised learning, the availability of trillions tokens in a pre-training corpus, instruction fine-tuning, and the development of large Transformers with billions of parameters, large language models (LLMs) are now capable of generating factual and coherent responses to human queries.
PAFT: A Parallel Training Paradigm for Effective LLM Fine-Tuning
This paper introduces PAFT, a new PArallel training paradigm for effective LLM Fine-Tuning, which independently performs SFT and preference alignment (e. g., DPO and ORPO, etc.)
Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descent
It enables full parameter training in FL with only selected block updates and uploads, thereby reducing communication, computation, and memory costs.
DualVC 3: Leveraging Language Model Generated Pseudo Context for End-to-end Low Latency Streaming Voice Conversion
With speaker-independent semantic tokens to guide the training of the content encoder, the dependency on ASR is removed and the model can operate under extremely small chunks, with cascading errors eliminated.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Nonlinear spiked covariance matrices and signal propagation in deep neural networks
Many recent works have studied the eigenvalue spectrum of the Conjugate Kernel (CK) defined by the nonlinear feature map of a feedforward neural network.
StreamVoice: Streamable Context-Aware Language Modeling for Real-time Zero-Shot Voice Conversion
Specifically, to enable streaming capability, StreamVoice employs a fully causal context-aware LM with a temporal-independent acoustic predictor, while alternately processing semantic and acoustic features at each time step of autoregression which eliminates the dependence on complete source speech.
FedRec+: Enhancing Privacy and Addressing Heterogeneity in Federated Recommendation Systems
Preserving privacy and reducing communication costs for edge users pose significant challenges in recommendation systems.
VITS-Based Singing Voice Conversion Leveraging Whisper and multi-scale F0 Modeling
This paper introduces the T23 team's system submitted to the Singing Voice Conversion Challenge 2023.
Proposing an intelligent mesh smoothing method with graph neural networks
With a lightweight model, GMSNet can effectively smoothing mesh nodes with varying degrees and remain unaffected by the order of input data.
MSM-VC: High-fidelity Source Style Transfer for Non-Parallel Voice Conversion by Multi-scale Style Modeling
In addition to conveying the linguistic content from source speech to converted speech, maintaining the speaking style of source speech also plays an important role in the voice conversion (VC) task, which is essential in many scenarios with highly expressive source speech, such as dubbing and data augmentation.
Text-to-Video: a Two-stage Framework for Zero-shot Identity-agnostic Talking-head Generation
In the second stage, an audio-driven talking head generation method is employed to produce compelling videos privided the audio generated in the first stage.
Auxiliary-Tasks Learning for Physics-Informed Neural Network-Based Partial Differential Equations Solving
To alleviate these issues, we proposed auxiliary-task learning-based physics-informed neural networks (ATL-PINNs), which provide four different auxiliary-task learning modes and investigate their performance compared with original PINNs.
LM-VC: Zero-shot Voice Conversion via Speech Generation based on Language Models
An intuitive approach is to follow AudioLM - Tokenizing speech into semantic and acoustic tokens respectively by HuBERT and SoundStream, and converting source semantic tokens to target acoustic tokens conditioned on acoustic tokens of the target speaker.
ST-PINN: A Self-Training Physics-Informed Neural Network for Partial Differential Equations
To address the issue of low accuracy and convergence problems of existing PINNs, we propose a self-training physics-informed neural network, ST-PINN.
Faithful and Efficient Explanations for Neural Networks via Neural Tangent Kernel Surrogate Models
A second trend has been to utilize kernel functions in various explain-by-example or data attribution tasks.
Multi-level Temporal-channel Speaker Retrieval for Zero-shot Voice Conversion
Specifically, to flexibly adapt to the dynamic-variant speaker characteristic in the temporal and channel axis of the speech, we propose a novel fine-grained speaker modeling method, called temporal-channel retrieval (TCR), to find out when and where speaker information appears in speech.
FedEBA+: Towards Fair and Effective Federated Learning via Entropy-Based Model
Ensuring fairness is a crucial aspect of Federated Learning (FL), which enables the model to perform consistently across all clients.
Delivering Speaking Style in Low-resource Voice Conversion with Multi-factor Constraints
Conveying the linguistic content and maintaining the source speech's speaking style, such as intonation and emotion, is essential in voice conversion (VC).
Overparameterized random feature regression with nearly orthogonal data
Our analysis shows high-probability non-asymptotic concentration results for the training errors, cross-validations, and generalization errors of RFRR centered around their respective values for a kernel ridge regression (KRR).
Expressive-VC: Highly Expressive Voice Conversion with Attention Fusion of Bottleneck and Perturbation Features
We further fuse the linguistic and para-linguistic features through an attention mechanism, where speaker-dependent prosody features are adopted as the attention query, which result from a prosody encoder with target speaker embedding and normalized pitch and energy of source speech as input.
Streaming Voice Conversion Via Intermediate Bottleneck Features And Non-streaming Teacher Guidance
In this paper, we propose to use intermediate bottleneck features (IBFs) to replace PPGs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
An Improved Structured Mesh Generation Method Based on Physics-informed Neural Networks
To improve the prediction accuracy of the neural network, we also introduce a novel auxiliary line strategy and an efficient network model during meshing.
TorchNTK: A Library for Calculation of Neural Tangent Kernels of PyTorch Models
We introduce torchNTK, a python library to calculate the empirical neural tangent kernel (NTK) of neural network models in the PyTorch framework.
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation
We study the first gradient descent step on the first-layer parameters $\boldsymbol{W}$ in a two-layer neural network: $f(\boldsymbol{x}) = \frac{1}{\sqrt{N}}\boldsymbol{a}^\top\sigma(\boldsymbol{W}^\top\boldsymbol{x})$, where $\boldsymbol{W}\in\mathbb{R}^{d\times N}, \boldsymbol{a}\in\mathbb{R}^{N}$ are randomly initialized, and the training objective is the empirical MSE loss: $\frac{1}{n}\sum_{i=1}^n (f(\boldsymbol{x}_i)-y_i)^2$.
Causal Disentanglement for Semantics-Aware Intent Learning in Recommendation
Traditional recommendation models trained on observational interaction data have generated large impacts in a wide range of applications, it faces bias problems that cover users' true intent and thus deteriorate the recommendation effectiveness.
IQDUBBING: Prosody modeling based on discrete self-supervised speech representation for expressive voice conversion
Specifically, prosody vector is first extracted from pre-trained VQ-Wav2Vec model, where rich prosody information is embedded while most speaker and environment information are removed effectively by quantization.
Multi-speaker Multi-style Text-to-speech Synthesis With Single-speaker Single-style Training Data Scenarios
Moreover, the explicit prosody features used in the prosody predicting module can increase the diversity of synthetic speech by adjusting the value of prosody features.
One-shot Voice Conversion For Style Transfer Based On Speaker Adaptation
One-shot style transfer is a challenging task, since training on one utterance makes model extremely easy to over-fit to training data and causes low speaker similarity and lack of expressiveness.
Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks
As an application, we show that random feature regression induced by the empirical kernel achieves the same asymptotic performance as its limiting kernel regression under the ultra-wide regime.
Hilbert Sinkhorn Divergence for Optimal Transport
Sinkhorn divergence has become a very popular metric to compare probability distributions in optimal transport.
Enriching Source Style Transfer in Recognition-Synthesis based Non-Parallel Voice Conversion
Specifically, prosodic features are used to explicit model prosody, while VAE and reference encoder are used to implicitly model prosody, which take Mel spectrum and bottleneck feature as input respectively.
WNARS: WFST based Non-autoregressive Streaming End-to-End Speech Recognition
Recently, attention-based encoder-decoder (AED) end-to-end (E2E) models have drawn more and more attention in the field of automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks
We study the eigenvalue distributions of the Conjugate Kernel and Neural Tangent Kernel associated to multi-layer feedforward neural networks.
Polynomial Representation for Persistence Diagram
In this work, we discover a set of general polynomials that vanish on vectorized PDs and extract the task-adapted feature representation from these polynomials.
