Search Results for author: Shinnosuke Takamichi

Found 27 papers, 7 papers with code

Do learned speech symbols follow Zipf's law?

no code implementations18 Sep 2023 Shinnosuke Takamichi, Hiroki Maeda, Joonyong Park, Daisuke Saito, Hiroshi Saruwatari

In this study, we investigate whether speech symbols, learned through deep learning, follow Zipf's law, akin to natural language symbols.

How Generative Spoken Language Modeling Encodes Noisy Speech: Investigation from Phonetics to Syntactics

no code implementations1 Jun 2023 Joonyong Park, Shinnosuke Takamichi, Tomohiko Nakamura, Kentaro Seki, Detai Xin, Hiroshi Saruwatari

We examine the speech modeling potential of generative spoken language modeling (GSLM), which involves using learned symbols derived from data rather than phonemes for speech analysis and synthesis.

Language Modelling Resynthesis

CALLS: Japanese Empathetic Dialogue Speech Corpus of Complaint Handling and Attentive Listening in Customer Center

no code implementations23 May 2023 Yuki Saito, Eiji Iimori, Shinnosuke Takamichi, Kentaro Tachibana, Hiroshi Saruwatari

We present CALLS, a Japanese speech corpus that considers phone calls in a customer center as a new domain of empathetic spoken dialogue.

Speech Synthesis

ChatGPT-EDSS: Empathetic Dialogue Speech Synthesis Trained from ChatGPT-derived Context Word Embeddings

no code implementations23 May 2023 Yuki Saito, Shinnosuke Takamichi, Eiji Iimori, Kentaro Tachibana, Hiroshi Saruwatari

We focus on ChatGPT's reading comprehension and introduce it to EDSS, a task of synthesizing speech that can empathize with the interlocutor's emotion.

Chatbot Reading Comprehension +2

Learning to Speak from Text: Zero-Shot Multilingual Text-to-Speech with Unsupervised Text Pretraining

1 code implementation30 Jan 2023 Takaaki Saeki, Soumi Maiti, Xinjian Li, Shinji Watanabe, Shinnosuke Takamichi, Hiroshi Saruwatari

While neural text-to-speech (TTS) has achieved human-like natural synthetic speech, multilingual TTS systems are limited to resource-rich languages due to the need for paired text and studio-quality audio data.

Language Modelling

JaCappella Corpus: A Japanese a Cappella Vocal Ensemble Corpus

1 code implementation29 Nov 2022 Tomohiko Nakamura, Shinnosuke Takamichi, Naoko Tanji, Satoru Fukayama, Hiroshi Saruwatari

These songs were arranged from out-of-copyright Japanese children's songs and have six voice parts (lead vocal, soprano, alto, tenor, bass, and vocal percussion).

Vocal ensemble separation

Acoustic Modeling for End-to-End Empathetic Dialogue Speech Synthesis Using Linguistic and Prosodic Contexts of Dialogue History

no code implementations16 Jun 2022 Yuto Nishimura, Yuki Saito, Shinnosuke Takamichi, Kentaro Tachibana, Hiroshi Saruwatari

To train the empathetic DSS model effectively, we investigate 1) a self-supervised learning model pretrained with large speech corpora, 2) a style-guided training using a prosody embedding of the current utterance to be predicted by the dialogue context embedding, 3) a cross-modal attention to combine text and speech modalities, and 4) a sentence-wise embedding to achieve fine-grained prosody modeling rather than utterance-wise modeling.

Self-Supervised Learning Sentence +2

STUDIES: Corpus of Japanese Empathetic Dialogue Speech Towards Friendly Voice Agent

no code implementations28 Mar 2022 Yuki Saito, Yuto Nishimura, Shinnosuke Takamichi, Kentaro Tachibana, Hiroshi Saruwatari

We describe our methodology to construct an empathetic dialogue speech corpus and report the analysis results of the STUDIES corpus.

Low-Latency Incremental Text-to-Speech Synthesis with Distilled Context Prediction Network

no code implementations22 Sep 2021 Takaaki Saeki, Shinnosuke Takamichi, Hiroshi Saruwatari

Although this method achieves comparable speech quality to that of a method that waits for the future context, it entails a huge amount of processing for sampling from the language model at each time step.

Knowledge Distillation Language Modelling +2

Onoma-to-wave: Environmental sound synthesis from onomatopoeic words

no code implementations11 Feb 2021 Yuki Okamoto, Keisuke Imoto, Shinnosuke Takamichi, Ryosuke Yamanishi, Takahiro Fukumori, Yoichi Yamashita

We also propose a method of environmental sound synthesis using onomatopoeic words and sound event labels.

Sound Audio and Speech Processing

RWCP-SSD-Onomatopoeia: Onomatopoeic Word Dataset for Environmental Sound Synthesis

no code implementations9 Jul 2020 Yuki Okamoto, Keisuke Imoto, Shinnosuke Takamichi, Ryosuke Yamanishi, Takahiro Fukumori, Yoichi Yamashita

We believe that using onomatopoeic words will enable us to control the fine time-frequency structure of synthesized sounds.

Sound Audio and Speech Processing

HumanGAN: generative adversarial network with human-based discriminator and its evaluation in speech perception modeling

no code implementations25 Sep 2019 Kazuki Fujii, Yuki Saito, Shinnosuke Takamichi, Yukino Baba, Hiroshi Saruwatari

To model the human-acceptable distribution, we formulate a backpropagation-based generator training algorithm by regarding human perception as a black-boxed discriminator.

Generative Adversarial Network

V2S attack: building DNN-based voice conversion from automatic speaker verification

no code implementations5 Aug 2019 Taiki Nakamura, Yuki Saito, Shinnosuke Takamichi, Yusuke Ijima, Hiroshi Saruwatari

The experimental evaluation compares converted voices between the proposed method that does not use the targeted speaker's voice data and the standard VC that uses the data.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

DNN-based Speaker Embedding Using Subjective Inter-speaker Similarity for Multi-speaker Modeling in Speech Synthesis

no code implementations19 Jul 2019 Yuki Saito, Shinnosuke Takamichi, Hiroshi Saruwatari

Although conventional DNN-based speaker embedding such as a $d$-vector can be applied to multi-speaker modeling in speech synthesis, it does not correlate with the subjective inter-speaker similarity and is not necessarily appropriate speaker representation for open speakers whose speech utterances are not included in the training data.

Speech Synthesis

Generative Moment Matching Network-based Random Modulation Post-filter for DNN-based Singing Voice Synthesis and Neural Double-tracking

no code implementations9 Feb 2019 Hiroki Tamaru, Yuki Saito, Shinnosuke Takamichi, Tomoki Koriyama, Hiroshi Saruwatari

To address this problem, we use a GMMN to model the variation of the modulation spectrum of the pitch contour of natural singing voices and add a randomized inter-utterance variation to the pitch contour generated by conventional DNN-based singing voice synthesis.

Singing Voice Synthesis

Phase reconstruction from amplitude spectrograms based on von-Mises-distribution deep neural network

2 code implementations10 Jul 2018 Shinnosuke Takamichi, Yuki Saito, Norihiro Takamune, Daichi Kitamura, Hiroshi Saruwatari

This paper presents a deep neural network (DNN)-based phase reconstruction from amplitude spectrograms.

Sound Audio and Speech Processing

JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis

1 code implementation28 Oct 2017 Ryosuke Sonobe, Shinnosuke Takamichi, Hiroshi Saruwatari

Thanks to improvements in machine learning techniques including deep learning, a free large-scale speech corpus that can be shared between academic institutions and commercial companies has an important role.

BIG-bench Machine Learning Speech Synthesis

Statistical Parametric Speech Synthesis Incorporating Generative Adversarial Networks

4 code implementations23 Sep 2017 Yuki Saito, Shinnosuke Takamichi, Hiroshi Saruwatari

In the proposed framework incorporating the GANs, the discriminator is trained to distinguish natural and generated speech parameters, while the acoustic models are trained to minimize the weighted sum of the conventional minimum generation loss and an adversarial loss for deceiving the discriminator.

Speech Synthesis Voice Conversion

Sampling-based speech parameter generation using moment-matching networks

no code implementations12 Apr 2017 Shinnosuke Takamichi, Tomoki Koriyama, Hiroshi Saruwatari

To give synthetic speech natural inter-utterance variation, this paper builds DNN acoustic models that make it possible to randomly sample speech parameters.

Speech Synthesis

Voice Conversion Using Sequence-to-Sequence Learning of Context Posterior Probabilities

no code implementations10 Apr 2017 Hiroyuki Miyoshi, Yuki Saito, Shinnosuke Takamichi, Hiroshi Saruwatari

Conventional VC using shared context posterior probabilities predicts target speech parameters from the context posterior probabilities estimated from the source speech parameters.

speech-recognition Speech Recognition +2

Cannot find the paper you are looking for? You can Submit a new open access paper.