Search Results for author:
Found 42 papers, 17 papers with code
EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Prompting
In this work, we propose EmoVoice, a novel emotion-controllable TTS model that exploits large language models (LLMs) to enable fine-grained freestyle natural language emotion control, and a phoneme boost variant design that makes the model output phoneme tokens and audio tokens in parallel to enhance content consistency, inspired by chain-of-thought (CoT) and modality-of-thought (CoM) techniques.
Accurate Expert Predictions in MoE Inference via Cross-Layer Gate
Large Language Models (LLMs) have demonstrated impressive performance across various tasks, and their application in edge scenarios has attracted significant attention.
Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline
Although offloading techniques utilise memory from the CPU and disk and parallelise the I/O and computation for efficiency, the computation for each expert in MoE models is often less than the I/O, resulting in numerous bubbles in the pipeline.
MinMo: A Multimodal Large Language Model for Seamless Voice Interaction
Previous models for voice interactions are categorized as native and aligned.
CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models
By training on a large-scale multilingual dataset, CosyVoice 2 achieves human-parity naturalness, minimal response latency, and virtually lossless synthesis quality in the streaming mode.
ChatTracker: Enhancing Visual Tracking Performance via Chatting with Multimodal Large Language Model
Recently, Vision-Language~(VL) trackers have proposed to utilize additional natural language descriptions to enhance versatility in various applications.
Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gap
While automatic speech recognition (ASR) systems have achieved remarkable performance with large-scale datasets, their efficacy remains inadequate in low-resource settings, encompassing dialects, accents, minority languages, and long-tail hotwords, domains with significant practical relevance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
SEFraud: Graph-based Self-Explainable Fraud Detection via Interpretative Mask Learning
While considerable efforts have been devoted to designing adequate fraud detectors, the interpretability of their results has often been overlooked.
MaLa-ASR: Multimedia-Assisted LLM-Based ASR
As more and more information-rich data like video become available, utilizing multi-modal auxiliary information to enhance audio tasks has sparked widespread research interest.
An Embarrassingly Simple Approach for LLM with Strong ASR Capacity
We found that delicate designs are not necessary, while an embarrassingly simple composition of off-the-shelf speech encoder, LLM, and the only trainable linear projector is competent for the ASR task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
BA-SOT: Boundary-Aware Serialized Output Training for Multi-Talker ASR
The recently proposed serialized output training (SOT) simplifies multi-talker automatic speech recognition (ASR) by generating speaker transcriptions separated by a special token.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
CASA-ASR: Context-Aware Speaker-Attributed ASR
In addition, a two-pass decoding strategy is further proposed to fully leverage the contextual modeling ability resulting in a better recognition performance.
A Comparative Study on Multichannel Speaker-Attributed Automatic Speech Recognition in Multi-party Meetings
Speaker-attributed automatic speech recognition (SA-ASR) in multi-party meeting scenarios is one of the most valuable and challenging ASR task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
A GPU-specialized Inference Parameter Server for Large-Scale Deep Recommendation Models
Recommendation systems are of crucial importance for a variety of modern apps and web services, such as news feeds, social networks, e-commerce, search, etc.
Merlin HugeCTR: GPU-accelerated Recommender System Training and Inference
In this talk, we introduce Merlin HugeCTR.
Unsupervisedly Prompting AlphaFold2 for Few-Shot Learning of Accurate Folding Landscape and Protein Structure Prediction
Data-driven predictive methods which can efficiently and accurately transform protein sequences into biologically active structures are highly valuable for scientific research and medical development.
A Universal PINNs Method for Solving Partial Differential Equations with a Point Source
In recent years, deep learning technology has been used to solve partial differential equations (PDEs), among which the physics-informed neural networks (PINNs)method emerges to be a promising method for solving both forward and inverse PDE problems.
PSP: Million-level Protein Sequence Dataset for Protein Structure Prediction
We provide in addition the benchmark training procedure for SOTA protein structure prediction model on this dataset.
A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings
Therefore, we propose the second approach, WD-SOT, to address alignment errors by introducing a word-level diarization model, which can get rid of such timestamp alignment dependency.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering
To alleviate the data scarcity problem in training question answering systems, recent works propose additional intermediate pre-training for dense passage retrieval (DPR).
Meta-Auto-Decoder for Solving Parametric Partial Differential Equations
Many important problems in science and engineering require solving the so-called parametric partial differential equations (PDEs), i. e., PDEs with different physical parameters, boundary conditions, shapes of computation domains, etc.
Solving Partial Differential Equations with Point Source Based on Physics-Informed Neural Networks
In recent years, deep learning technology has been used to solve partial differential equations (PDEs), among which the physics-informed neural networks (PINNs) emerges to be a promising method for solving both forward and inverse PDE problems.
You Ought to Look Around: Precise, Large Span Action Detection
In this paper, we propose a method called YOLA (You Ought to Look Around) which includes three parts: 1) a robust backbone SPN-I3D for extracting spatio-temporal features.
Towards More Effective and Economic Sparsely-Activated Model
To increase the number of activated experts without an increase in computational cost, we propose SAM (Switch and Mixture) routing, an efficient hierarchical routing mechanism that activates multiple experts in a same device (GPU).
iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compression
We also propose a lossless compression algorithm based on iVPF.
Answer Complex Questions: Path Ranker Is All You Need
Currently, the most popular method for open-domain Question Answering (QA) adopts "Retriever and Reader" pipeline, where the retriever extracts a list of candidate documents from a large set of documents followed by a ranker to rank the most relevant documents and the reader extracts answer from the candidates.
SKFAC: Training Neural Networks With Faster Kronecker-Factored Approximate Curvature
For the fully connected layers, by utilizing the low-rank property of Kronecker factors of Fisher information matrix, our method only requires inverting a small matrix to approximate the curvature with desirable accuracy.
Vertex-Centric Visual Programming for Graph Neural Networks
Graph neural networks (GNNs) have achieved remarkable performance in many graph analytics tasks such as node classification, link prediction and graph clustering.
SKFAC:Training Neural Networks with Faster Kronecker-Factored Approximate Curvature
For the fully connected layers, by utilizing the low-rank property of Kronecker factors of Fisher information matrix, our method only requires inverting a small matrix to approximate the curvature with desirable accuracy.
THOR, Trace-based Hardware-adaptive layer-ORiented Natural Gradient Descent Computation
It is well-known that second-order optimizer can accelerate the training of deep neural networks, however, the huge computation cost of second-order optimization makes it impractical to apply in real practice.
DGCL: an efficient communication library for distributed GNN training
Graph neural networks (GNNs) have gained increasing popularity in many areas such as e-commerce, social networks and bio-informatics.
Seastar: vertex-centric programming for graph neural networks
Graph neural networks (GNNs) have achieved breakthrough performance in graph analytics such as node classification, link prediction and graph clustering.
Elastic Deep Learning in Multi-Tenant GPU Clusters
We study how to support elasticity, that is, the ability to dynamically adjust the parallelism (i. e., the number of GPUs), for deep neural network (DNN) training in a GPU cluster.
WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit
In this paper, we propose an open source, production first, and production ready speech recognition toolkit called WeNet in which a new two-pass approach is implemented to unify streaming and non-streaming end-to-end (E2E) speech recognition in a single model.
EPP-MVSNet: Epipolar-Assembling Based Depth Prediction for Multi-View Stereo
As a result, we achieve promising results on all datasets and the highest F-Score on the online TNT intermediate benchmark.
Ranked #9 on
Point Clouds
on Tanks and Temples
AsymptoticNG: A regularized natural gradient optimization algorithm with look-ahead strategy
Optimizers that further adjust the scale of gradient, such as Adam, Natural Gradient (NG), etc., despite widely concerned and used by the community, are often found poor generalization performance, compared with Stochastic Gradient Descent (SGD).
Unified Streaming and Non-streaming Two-pass End-to-end Model for Speech Recognition
In this paper, we present a novel two-pass approach to unify streaming and non-streaming end-to-end (E2E) speech recognition in a single model.
Ranked #12 on
Speech Recognition
on AISHELL-1
Eigenvalue-corrected Natural Gradient Based on a New Approximation
Using second-order optimization methods for training deep neural networks (DNNs) has attracted many researchers.
A Trace-restricted Kronecker-Factored Approximation to Natural Gradient
There have been many attempts to use second-order optimization methods for training deep neural networks.
The SLT 2021 children speech recognition challenge: Open datasets, rules and baselines
Automatic speech recognition (ASR) has been significantly advanced with the use of deep learning and big data.
Sound Audio and Speech Processing
The energy technique for the six-step BDF method
In combination with the Grenander--Szeg\"o theorem, we observe that a relaxed positivity condition on multipliers, milder than the basic %fundamental requirement of the Nevanlinna--Odeh multipliers that the sum of the absolute values of their components is strictly less than $1$, makes the energy technique applicable to the stability analysis of BDF methods for parabolic equations with selfadjoint elliptic part.
Numerical Analysis Numerical Analysis
TensorOpt: Exploring the Tradeoffs in Distributed DNN Training with Auto-Parallelism
A good parallelization strategy can significantly improve the efficiency or reduce the cost for the distributed training of deep neural networks (DNNs).
