Search Results for author:
Found 14 papers, 7 papers with code
AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs
In this work, we extend the instruction-tuned Llama-2 model with end-to-end general-purpose speech processing and reasoning abilities while maintaining the wide range of original LLM capabilities, without using any carefully curated paired data.
Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Model
In this work, we propose the use of an adaptive masking approach in two scenarios for pruning a multilingual ASR model efficiently, each resulting in sparse monolingual models or a sparse multilingual model (named as Dynamic ASR Pathways).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
TODM: Train Once Deploy Many Efficient Supernet-Based RNN-T Compression For On-device ASR Models
Results demonstrate that our TODM Supernet either matches or surpasses the performance of manually tuned models by up to a relative of 3% better in word error rate (WER), while efficiently keeping the cost of training many models at a small constant.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Prompting Large Language Models with Speech Recognition Abilities
Furthermore, we perform ablation studies to investigate whether the LLM can be completely frozen during training to maintain its original capabilities, scaling up the audio encoder, and increasing the audio encoder striding to generate fewer embeddings.
Abstractive Text Summarization
Automatic Speech Recognition
+3
Multi-Head State Space Model for Speech Recognition
State space models (SSMs) have recently shown promising results on small-scale sequence and language modelling tasks, rivalling and outperforming many attention-based approaches.
Ranked #8 on
Speech Recognition
on LibriSpeech test-clean
Anchored Speech Recognition with Neural Transducers
In this paper, we investigate anchored speech recognition to make neural transducers robust to background speech.
Understanding Non-linearity in Graph Neural Networks from the Bayesian-Inference Perspective
Graph neural networks (GNNs) have shown superiority in many prediction tasks over graphs due to their impressive capability of capturing nonlinear relations in graph-structured data.
Federated Domain Adaptation for ASR with Full Self-Supervision
Cross-device federated learning (FL) protects user privacy by collaboratively training a model on user devices, therefore eliminating the need for collecting, storing, and manually labeling user data.
Graph Belief Propagation Networks
With the wide-spread availability of complex relational data, semi-supervised node classification in graphs has become a central machine learning problem.
A Unifying Generative Model for Graph Learning Algorithms: Label Propagation, Graph Convolutions, and Combinations
Semi-supervised learning on graphs is a widely applicable problem in network science and machine learning.
Residual Correlation in Graph Neural Network Regression
A graph neural network transforms features in each vertex's neighborhood into a vector representation of the vertex.
Neural Jump Stochastic Differential Equations
Many time series are effectively generated by a combination of deterministic continuous flows along with discrete jumps sparked by stochastic events.
Graph-based Semi-Supervised & Active Learning for Edge Flows
The first strategy selects edges to minimize the reconstruction error bound and works well on flows that are approximately divergence-free.
Detecting Core-Periphery Structure in Spatial Networks
The core-periphery structure, which decompose a network into a densely-connected core and a sparsely-connected periphery, constantly emerges from spatial networks such as traffic, biological and social networks.
Social and Information Networks Physics and Society
