Vibravox: A Dataset of French Speech Captured with Body-conduction Audio Sensors

Vibravox is a dataset compliant with the General Data Protection Regulation (GDPR) containing audio recordings using five different body-conduction audio sensors : two in-ear microphones, two bone conduction vibration pickups and a laryngophone. The data set also includes audio data from an airborne microphone used as a reference. The Vibravox corpus contains 38 hours of speech samples and physiological sounds recorded by 188 participants under different acoustic conditions imposed by an high order ambisonics 3D spatializer. Annotations about the recording conditions and linguistic transcriptions are also included in the corpus. We conducted a series of experiments on various speech-related tasks, including speech recognition, speech enhancement and speaker verification. These experiments were carried out using state-of-the-art models to evaluate and compare their performances on signals captured by the different audio sensors offered by the Vibravox dataset, with the aim of gaining a better grasp of their individual characteristics.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Automatic Phoneme Recognition VibraVox (forehead accelerometer) medium wav2vec2.0 Test PER 0.046 # 1
Bandwidth Extension VibraVox (forehead accelerometer) Configurable EBEN (M=4, P=4, Q=4) STOI 0.855 # 1
Noresqua-MOS 4.250 # 1
PER (wav2vec2) 0.091 # 1
EER (ECAPA2) 0.0183 # 1
Speaker Verification VibraVox (forehead accelerometer) ECAPA2 Test EER 0.009 # 1
Test min-DCF 0.06 # 1
Speaker Verification VibraVox (headset microphone) ECAPA2 Test EER 0.0026 # 1
Test min-DCF 0.02 # 1
Automatic Phoneme Recognition VibraVox (headset microphone) medium wav2vec2.0 Test PER 0.028 # 1
Bandwidth Extension VibraVox (rigid in-ear microphone) Configurable EBEN (M=4, P=2, Q=4) STOI 0.877 # 1
Noresqua-MOS 4.285 # 1
PER (wav2vec2) 0.084 # 1
EER (ECAPA2) 0.0364 # 1
Speaker Verification VibraVox (rigid in-ear microphone) ECAPA2 Test EER 0.0316 # 1
Test min-DCF 0.21 # 1
Automatic Phoneme Recognition VibraVox (rigid in-ear microphone) medium wav2vec2.0 Test PER 0.045 # 1
Speaker Verification VibraVox (soft in-ear microphone) ECAPA2 Test EER 0.0172 # 1
Test min-DCF 0.10 # 1
Automatic Phoneme Recognition VibraVox (soft in-ear microphone) medium wav2vec2.0 Test PER 0.041 # 1
Bandwidth Extension VibraVox (soft in-ear microphone) Configurable EBEN (M=4, P=2, Q=4) STOI 0.868 # 1
Noresqua-MOS 4.331 # 1
PER (wav2vec2) 0.087 # 1
EER (ECAPA2) 0.0488 # 1
Speaker Verification VibraVox (temple vibration pickup) ECAPA2 Test EER 0.08 # 1
Test min-DCF 0.58 # 1
Automatic Phoneme Recognition VibraVox (temple vibration pickup) medium wav2vec2.0 Test PER 0.142 # 1
Bandwidth Extension VibraVox (temple vibration pickup) Configurable EBEN (M=4, P=1, Q=4) STOI 0.763 # 1
Noresqua-MOS 3.632 # 1
PER (wav2vec2) 0.391 # 1
EER (ECAPA2) 0.1622 # 1
Speaker Verification VibraVox (throat microphone) ECAPA2 Test EER 0.0353 # 1
Test min-DCF 0.20 # 1
Bandwidth Extension VibraVox (throat microphone) Configurable EBEN (M=4, P=2, Q=4) STOI 0.834 # 1
Noresqua-MOS 3.862 # 1
PER (wav2vec2) 0.179 # 1
EER (ECAPA2) 0.0847 # 1
Automatic Phoneme Recognition VibraVox (throat microphone) medium wav2vec2.0 Test PER 0.073 # 1

Methods