Search Results for author:
Found 14 papers, 1 papers with code
AudioChatLlama: Towards General-Purpose Speech Abilities for LLMs
In this work, we extend the instruction-tuned Llama-2 model with end-to-end general-purpose speech processing and reasoning abilities while maintaining the wide range of original LLM capabilities, without using any carefully curated paired data.
End-to-End Speech Recognition Contextualization with Large Language Models
Overall, we demonstrate that by only adding a handful number of trainable parameters via adapters, we can unlock contextualized speech recognition capability for the pretrained LLM while keeping the same text-only input functionality.
Prompting Large Language Models with Speech Recognition Abilities
Furthermore, we perform ablation studies to investigate whether the LLM can be completely frozen during training to maintain its original capabilities, scaling up the audio encoder, and increasing the audio encoder striding to generate fewer embeddings.
Abstractive Text Summarization
Automatic Speech Recognition
+3
SynthVSR: Scaling Up Visual Speech Recognition With Synthetic Supervision
Furthermore, when combined with large-scale pseudo-labeled audio-visual data SynthVSR yields a new state-of-the-art VSR WER of 16. 9% using publicly available data only, surpassing the recent state-of-the-art approaches trained with 29 times more non-public machine-transcribed video data (90, 000 hours).
Egocentric Audio-Visual Noise Suppression
In this paper, we first demonstrate that egocentric visual information is helpful for noise suppression.
KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos
In this paper, we describe KT-Speech-Crawler: an approach for automatic dataset construction for speech recognition by crawling YouTube videos.
Incorporating End-to-End Speech Recognition Models for Sentiment Analysis
We argue that using ground-truth transcriptions during training and evaluation phases leads to a significant discrepancy in performance compared to real-world conditions, as the spoken text has to be recognized on the fly and can contain speech recognition mistakes.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks
Speech emotion recognition (SER) is an important aspect of effective human-robot collaboration and received a lot of attention from the research community.
EmoRL: Continuous Acoustic Emotion Classification using Deep Reinforcement Learning
Acoustically expressed emotions can make communication with a robot more efficient.
Reusing Neural Speech Representations for Auditory Emotion Recognition
Acoustic emotion recognition aims to categorize the affective state of the speaker and is still a difficult task for machine learning models.
GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection
Given the text of a tweet and its emotion category (anger, joy, fear, and sadness), the participants were asked to build a system that assigns emotion intensity values.
Automatically augmenting an emotion dataset improves classification using audio
In this work, we tackle a problem of speech emotion classification.
The OMG-Emotion Behavior Dataset
This paper is the basis paper for the accepted IJCNN challenge One-Minute Gradual-Emotion Recognition (OMG-Emotion) by which we hope to foster long-emotion classification using neural models for the benefit of the IJCNN community.
Human-Computer Interaction
GradAscent at EmoInt-2017: Character and Word Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection
Given the text of a tweet and its emotion category (anger, joy, fear, and sadness), the participants were asked to build a system that assigns emotion intensity values.
