Search Results for author:
Found 24 papers, 7 papers with code
Towards Learning a Universal Non-Semantic Representation of Speech
The ultimate goal of transfer learning is to reduce labeled data requirements by exploiting a pre-existing embedding model trained for different datasets or tasks.
MusicLM: Generating Music From Text
We introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody backed by a distorted guitar riff".
Ranked #8 on
Text-to-Music Generation
on MusicCaps
MuLan: A Joint Embedding of Music Audio and Natural Language
Music tagging and content-based retrieval systems have traditionally been constructed using pre-defined ontologies covering a rigid set of music attributes or text queries.
Attention Bottlenecks for Multimodal Fusion
Humans perceive the world by concurrently processing and fusing high-dimensional inputs from multiple modalities such as vision and audio.
Ranked #2 on
Action Classification
on Kinetics-Sounds
CNN Architectures for Large-Scale Audio Classification
Convolutional Neural Networks (CNNs) have proven very effective in image classification and show promise for audio.
Self-Supervised Learning from Automatically Separated Sound Scenes
Real-world sound scenes consist of time-varying collections of sound sources, each generating characteristic sound events that are mixed together in audio recordings.
A segmental framework for fully-unsupervised large-vocabulary speech recognition
We also show that the discovered clusters can be made less speaker- and gender-specific by using an unsupervised autoencoder-like feature extractor to learn better frame-level features (prior to embedding).
Unsupervised Learning of Semantic Audio Representations
Even in the absence of any explicit semantic annotation, vast collections of audio recordings provide valuable information for learning the categorical structure of sounds.
Ranked #41 on
Audio Classification
on AudioSet
Scalable Out-of-Sample Extension of Graph Embeddings Using Deep Neural Networks
Several popular graph embedding techniques for representation learning and dimensionality reduction rely on performing computationally expensive eigendecompositions to derive a nonlinear transformation of the input data space.
Unsupervised word segmentation and lexicon discovery using acoustic word embeddings
In settings where only unlabelled speech data is available, speech technology needs to be developed without transcriptions, pronunciation dictionaries, or language modelling text.
Evaluating Low-Level Speech Features Against Human Perceptual Data
We introduce a method for measuring the correspondence between low-level speech features and human perception, using a cognitive model of speech perception implemented directly on speech recordings.
Automatic Speech Recognition (ASR)
Representation Learning
+1
Bridging the gap between speech technology and natural language processing: an evaluation toolbox for term discovery systems
The unsupervised discovery of linguistic terms from either continuous phoneme transcriptions or from raw speech has seen an increasing interest in the past years both from a theoretical and a practical standpoint.
Coincidence, Categorization, and Consolidation: Learning to Recognize Sounds with Minimal Supervision
Humans do not acquire perceptual abilities in the way we train machines.
Improving Universal Sound Separation Using Sound Classification
Deep learning approaches have recently achieved impressive performance on both audio source separation and sound classification.
Addressing Missing Labels in Large-Scale Sound Event Recognition Using a Teacher-Student Framework With Loss Masking
The study of label noise in sound event recognition has recently gained attention with the advent of larger and noisier datasets.
Into the Wild with AudioScope: Unsupervised Audio-Visual Separation of On-Screen Sounds
For evaluation and semi-supervised experiments, we collected human labels for presence of on-screen and off-screen sounds on a small subset of clips.
Sparse, Efficient, and Semantic Mixture Invariant Training: Taming In-the-Wild Unsupervised Sound Separation
The best performance is achieved using larger numbers of output sources, enabled by our efficient MixIT loss, combined with sparsity losses to prevent over-separation.
BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition
We summarize the results of a host of efforts using giant automatic speech recognition (ASR) models pre-trained using large, diverse unlabeled datasets containing approximately a million hours of audio.
Ranked #1 on
Speech Recognition
on Common Voice
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Universal Paralinguistic Speech Representations Using Self-Supervised Conformers
Many speech applications require understanding aspects beyond the words being spoken, such as recognizing emotion, detecting whether the speaker is wearing a mask, or distinguishing real from synthetic speech.
Text-Driven Separation of Arbitrary Sounds
The second model, SoundFilter, takes a mixed source audio clip as an input and separates it based on a conditioning vector from the shared text-audio representation defined by SoundWords, making the model agnostic to the conditioning modality.
MAQA: A Multimodal QA Benchmark for Negation
Multimodal learning can benefit from the representation power of pretrained Large Language Models (LLMs).
V2Meow: Meowing to the Visual Beat via Video-to-Music Generation
Video-to-music generation demands both a temporally localized high-quality listening experience and globally aligned video-acoustic signatures.
Dataset balancing can hurt model performance
We find, however, that while balancing improves performance on the public AudioSet evaluation data it simultaneously hurts performance on an unpublished evaluation set collected under the same conditions.
