Search Results for author:
Found 21 papers, 13 papers with code
Guided Masked Self-Distillation Modeling for Distributed Multimedia Sensor Event Analysis
This is because the information obtained from a single sensor is often missing or fragmented in such an environment; observations from multiple locations and modalities should be integrated to analyze events comprehensively.
Masked Modeling Duo: Towards a Universal Audio Pre-training Framework
This study proposes Masked Modeling Duo (M2D), an improved masked prediction SSL, which learns by predicting representations of masked input signals that serve as training signals.
Refining Knowledge Transfer on Audio-Image Temporal Agreement for Audio-Text Cross Retrieval
Experimental results show that method (i) improves the audio-text retrieval performance by selecting the nearest image that aligns with the audio information and transferring the learned knowledge.
Audio Difference Captioning Utilizing Similarity-Discrepancy Disentanglement
We proposed Audio Difference Captioning (ADC) as a new extension task of audio captioning for describing the semantic differences between input pairs of similar but slightly different audio clips.
Masked Modeling Duo for Speech: Specializing General-Purpose Audio Representation to Speech using Denoising Distillation
Self-supervised learning general-purpose audio representations have demonstrated high performance in a variety of tasks.
First-shot anomaly sound detection for machine condition monitoring: A domain generalization baseline
This paper provides a baseline system for First-shot-compliant unsupervised anomaly detection (ASD) for machine condition monitoring.
Masked Modeling Duo: Learning Representations by Encouraging Both Networks to Model the Input
We propose a new method, Masked Modeling Duo (M2D), that learns representations directly while obtaining training signals using only masked patches.
Ranked #1 on
Speaker Identification
on VoxCeleb1
(using extra training data)
ConceptBeam: Concept Driven Target Speech Extraction
We use it to bridge modality-dependent information, i. e., the speech segments in the mixture, and the specified, modality-independent concept.
Introducing Auxiliary Text Query-modifier to Content-based Audio Retrieval
While the range of conventional content-based audio retrieval is limited to audio that is similar to the query audio, the proposed method can adjust the retrieval range by adding an embedding of the auxiliary text query-modifier to the embedding of the query sample audio in a shared latent space.
Composing General Audio Representation by Fusing Multilayer Features of a Pre-trained Model
This approach improves the utility of frequency and channel information in downstream processes, and combines the effectiveness of middle and late layer features for different tasks.
Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation
In this paper, we seek to learn audio representations from the input itself as supervision using a pretext task of auto-encoding of masked spectrogram patches, Masked Spectrogram Modeling (MSM, a variant of Masked Image Modeling applied to audio spectrogram).
BYOL for Audio: Exploring Pre-trained General-purpose Audio Representations
In this study, we hypothesize that representations effective for general audio tasks should provide multiple aspects of robust features of the input sound.
SoundBeam: Target sound extraction conditioned on sound-class labels and enrollment clues for increased performance and continuous learning
We can achieve this with a neural network that extracts the target SEs by conditioning it on clues representing the target SE classes.
Multi-view and Multi-modal Event Detection Utilizing Transformer-based Multi-sensor fusion
We tackle a challenging task: multi-view and multi-modal event detection that detects events in a wide-range real environment by utilizing data from distributed cameras and microphones and their weak labels.
Echo-aware Adaptation of Sound Event Localization and Detection in Unknown Environments
Our goal is to develop a sound event localization and detection (SELD) system that works robustly in unknown environments.
ToyADMOS2: Another dataset of miniature-machine operating sounds for anomalous sound detection under domain shift conditions
This paper proposes a new large-scale dataset called "ToyADMOS2" for anomaly detection in machine operating sounds (ADMOS).
BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation
Inspired by the recent progress in self-supervised learning for computer vision that generates supervision using data augmentations, we explore a new general-purpose audio representation learning approach.
Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval
Then, the caption of the audio input is generated by using a pre-trained language model while referring to the guidance captions.
Effects of Word-frequency based Pre- and Post- Processings for Audio Captioning
The system we used for Task 6 (Automated Audio Captioning)of the Detection and Classification of Acoustic Scenes and Events(DCASE) 2020 Challenge combines three elements, namely, dataaugmentation, multi-task learning, and post-processing, for audiocaptioning.
The NTT DCASE2020 Challenge Task 6 system: Automated Audio Captioning with Keywords and Sentence Length Estimation
This technical report describes the system participating to the Detection and Classification of Acoustic Scenes and Events (DCASE) 2020 Challenge, Task 6: automated audio captioning.
Ranked #4 on
Audio captioning
on Clotho
Crossmodal Voice Conversion
We use the latent code of an input face image encoded by the face encoder as the auxiliary input into the speech converter and train the speech converter so that the original latent code can be recovered from the generated speech by the voice encoder.
