Search Results for author:
Found 125 papers, 23 papers with code
Segmentation for Efficient Supervised Language Annotation with an Explicit Cost-Utility Tradeoff
In this paper, we study the problem of manually correcting automatic annotations of natural language in as efficient a manner as possible.
Collection of a Simultaneous Translation Corpus for Comparative Analysis
This makes it possible to compare translation data with simultaneous interpretation data.
Towards Multilingual Conversations in the Medical Domain: Development of Multilingual Medical Data and A Network-based ASR System
This paper outlines the recent development on multilingual medical data and multilingual speech recognition system for network-based speech-to-speech translation in the medical domain.
Semantic Parsing of Ambiguous Input through Paraphrasing and Verification
We propose a new method for semantic parsing of ambiguous and ungrammatical input, such as search queries.
Neural Reranking Improves Subjective Quality of Machine Translation: NAIST at WAT2015
We further perform a detailed analysis of reasons for this increase, finding that the main contributions of the neural models lie in improvement of the grammatical correctness of the output, as opposed to improvements in lexical choice of content words.
Optimizing Computer-Assisted Transcription Quality with Iterative User Interfaces
Our goal is to improve the human transcription quality via appropriate user interface design.
Construction of Japanese Audio-Visual Emotion Database and Its Application in Emotion Recognition
Emotional aspects play a vital role in making human communication a rich and dynamic experience.
Incorporating Discrete Translation Lexicons into Neural Machine Translation
Neural machine translation (NMT) often makes mistakes in translating low-frequency content words that are essential to understanding the meaning of the sentence.
Neural Machine Translation via Binary Code Prediction
In this paper, we propose a new method for calculating the output layer in neural machine translation systems.
Compressing Recurrent Neural Network with Tensor Train
Recurrent Neural Network (RNN) are a popular choice for modeling temporal and sequential tasks and achieve many state-of-the-art performance on various complex problems.
Local Monotonic Attention Mechanism for End-to-End Speech and Language Processing
In this paper, we propose a novel attention mechanism that has local and monotonic properties.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Analysis of the Effect of Dependency Information on Predicate-Argument Structure Analysis and Zero Anaphora Resolution
This paper investigates and analyzes the effect of dependency information on predicate-argument structure analysis (PASA) and zero anaphora resolution (ZAR) for Japanese, and shows that a straightforward approach of PASA and ZAR works effectively even if dependency information was not available.
Gated Recurrent Neural Tensor Network
Our proposed RNNs, which are called a Long-Short Term Memory Recurrent Neural Tensor Network (LSTMRNTN) and Gated Recurrent Unit Recurrent Neural Tensor Network (GRURNTN), are made by combining the LSTM and GRU RNN models with the tensor product.
An Empirical Study of Mini-Batch Creation Strategies for Neural Machine Translation
Training of neural machine translation (NMT) models usually uses mini-batches for efficiency purposes.
Listening while Speaking: Speech Chain by Deep Learning
In this paper, we take a step further and develop a closed-loop speech chain model based on deep learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Information Navigation System with Discovering User Interests
We demonstrate an information navigation system for sightseeing domains that has a dialogue interface for discovering user interests for tourist activities.
Transcribing Against Time
We investigate the problem of manually correcting errors from an automatic speech transcript in a cost-sensitive fashion.
Attention-based Wav2Text with Feature Transfer Learning
In this paper, we construct the first end-to-end attention-based encoder-decoder model to process directly from raw speech waveform to the text transcription.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Sequence-to-Sequence ASR Optimization via Reinforcement Learning
Despite the success of sequence-to-sequence approaches in automatic speech recognition (ASR) systems, the models still suffer from several problems, mainly due to the mismatch between the training and inference conditions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Acquisition and Assessment of Semantic Content for the Generation of Elaborateness and Indirectness in Spoken Dialogue Systems
Defining all possible system actions in a dialogue system by hand is a tedious work.
Cultural Vocal Bursts Intensity Prediction
Spoken Dialogue Systems
Improving Neural Machine Translation through Phrase-based Forced Decoding
Compared to traditional statistical machine translation (SMT), neural machine translation (NMT) often sacrifices adequacy for the sake of fluency.
A Simple and Strong Baseline: NAIST-NICT Neural Machine Translation System for WAT2017 English-Japanese Translation Task
This paper describes the details about the NAIST-NICT machine translation system for WAT2017 English-Japanese Scientific Paper Translation Task.
Structured-based Curriculum Learning for End-to-end English-Japanese Speech Translation
Sequence-to-sequence attentional-based neural network architectures have been shown to provide a powerful model for machine translation and speech recognition.
Interactive Image Manipulation with Natural Language Instruction Commands
We propose an interactive image-manipulation system with natural language instruction, which can generate a target image from a source image and an instruction that describes the difference between the source and the target image.
Tensor Decomposition for Compressing Recurrent Neural Network
In the machine learning fields, Recurrent Neural Network (RNN) has become a popular architecture for sequential data modeling.
Machine Speech Chain with One-shot Speaker Adaptation
In the speech chain loop mechanism, ASR also benefits from the ability to further learn an arbitrary speaker's characteristics from the generated speech waveform, resulting in a significant improvement in the recognition rate.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Guiding Neural Machine Translation with Retrieved Translation Pieces
Specifically, for an input sentence, we use a search engine to retrieve sentence pairs whose source sides are similar with the input sentence, and then collect $n$-grams that are both in the retrieved target sentences and aligned with words that match in the source sentences, which we call "translation pieces".
Multi-Source Neural Machine Translation with Missing Data
This study focuses on the use of incomplete multilingual corpora in multi-encoder NMT and mixture of NMT experts and examines a very simple implementation where missing source translations are replaced by a special symbol <NULL>.
Unsupervised Counselor Dialogue Clustering for Positive Emotion Elicitation in Neural Dialogue System
Positive emotion elicitation seeks to improve user{'}s emotional state through dialogue system interaction, where a chat-based scenario is layered with an implicit goal to address user{'}s emotional needs.
Multi-scale Alignment and Contextual History for Attention Mechanism in Sequence-to-sequence Model
In this paper, we propose two ideas to improve sequence-to-sequence model performance by enhancing the attention module.
Training Neural Machine Translation using Word Embedding-based Loss
The proposed loss function encourages an NMT decoder to generate words close to their references in the embedding space; this helps the decoder to choose similar acceptable words when the actual best candidates are not included in the vocabulary due to its size limitation.
Multi-Source Neural Machine Translation with Data Augmentation
By using information from these multiple sources, these systems achieve large gains in accuracy.
End-to-End Feedback Loss in Speech Chain Framework via Straight-Through Estimator
In our previous work, we applied a speech chain mechanism as a semi-supervised learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Another Diversity-Promoting Objective Function for Neural Dialogue Generation
Although generation-based dialogue systems have been widely researched, the response generations by most existing systems have very low diversities.
Optimization of Information-Seeking Dialogue Strategy for Argumentation-Based Dialogue System
It is required that these systems have sufficient supporting information to argue their claims rationally; however, the systems often do not have enough of such information in realistic situations.
VQVAE Unsupervised Unit Discovery and Multi-scale Code2Spec Inverter for Zerospeech Challenge 2019
Our proposed approach significantly improved the intelligibility (in CER), the MOS, and discrimination ABX scores compared to the official ZeroSpeech 2019 baseline or even the topline.
An Incremental Turn-Taking Model For Task-Oriented Dialog Systems
To identify the point of maximal understanding in an ongoing utterance, we a) implement an incremental Dialog State Tracker which is updated on a token basis (iDST) b) re-label the Dialog State Tracking Challenge 2 (DSTC2) dataset and c) adapt it to the incremental turn-taking experimental scenario.
Listening while Speaking and Visualizing: Improving ASR through Multimodal Chain
Previously, a machine speech chain, which is based on sequence-to-sequence deep learning, was proposed to mimic speech perception and production behavior.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Conversational Response Re-ranking Based on Event Causality and Role Factored Tensor Event Embedding
We propose a novel method for selecting coherent and diverse responses for a given dialogue context.
Make Skeleton-based Action Recognition Model Smaller, Faster and Better
Although skeleton-based action recognition has achieved great success in recent years, most of the existing methods may suffer from a large model size and slow execution speed.
Ranked #1 on
Hand Gesture Recognition
on DHG-14
Neural Conversation Model Controllable by Given Dialogue Act Based on Adversarial Learning and Label-aware Objective
We introduce an adversarial learning framework for the task of generating conditional responses with a new objective to a discriminator, which explicitly distinguishes sentences by using labels.
Speech-to-speech Translation between Untranscribed Unknown Languages
Second, we train a sequence-to-sequence model that directly maps the source language speech to the target language's discrete representation.
Deja-vu: Double Feature Presentation and Iterated Loss in Deep Transformer Networks
As our motivation is to allow acoustic models to re-examine their input features in light of partial hypotheses we introduce intermediate model heads and loss function.
Using Panoramic Videos for Multi-person Localization and Tracking in a 3D Panoramic Coordinate
3D panoramic multi-person localization and tracking are prominent in many applications, however, conventional methods using LiDAR equipment could be economically expensive and also computationally inefficient due to the processing of point cloud data.
Ranked #1 on
Multi-Object Tracking
on MOT15_3D
(using extra training data)
Simultaneous Neural Machine Translation using Connectionist Temporal Classification
Simultaneous machine translation is a variant of machine translation that starts the translation process before the end of an input.
Caption Generation of Robot Behaviors based on Unsupervised Learning of Action Segments
Two reasons make it challenging to apply existing sequence-to-sequence models to this mapping: 1) it is hard to prepare a large-scale dataset for any kind of robots and their environment, and 2) there is a gap between the number of samples obtained from robot action observations and generated word sequences of captions.
Emotional Speech Corpus for Persuasive Dialogue System
Expressing emotion is known as an efficient way to persuade one{'}s dialogue partner to accept one{'}s claim or proposal.
Transformer VQ-VAE for Unsupervised Unit Discovery and Speech Synthesis: ZeroSpeech 2020 Challenge
In this paper, we report our submitted system for the ZeroSpeech 2020 challenge on Track 2019.
NAIST's Machine Translation Systems for IWSLT 2020 Conversational Speech Translation Task
This paper describes NAIST{'}s NMT system submitted to the IWSLT 2020 conversational speech translation task.
Automatic Machine Translation Evaluation using Source Language Inputs and Cross-lingual Language Model
Our experiments show that our proposed method using Cross-lingual Language Model (XLM) trained with a translation language modeling (TLM) objective achieves a higher correlation with human judgments than a baseline method that uses only hypothesis and reference sentences.
Reflection-based Word Attribute Transfer
For transferring king into queen in this analogy-based manner, we subtract a difference vector man - woman based on the knowledge that king is male.
ReMOTS: Self-Supervised Refining Multi-Object Tracking and Segmentation
We aim to improve the performance of Multiple Object Tracking and Segmentation (MOTS) by refinement.
Ranked #1 on
Multi-Object Tracking
on MOTS20
Multi-Object Tracking
Multi-Object Tracking and Segmentation
+2
Image Captioning with Visual Object Representations Grounded in the Textual Modality
We present our work in progress exploring the possibilities of a shared embedding space between textual and visual modality.
Incremental Machine Speech Chain Towards Enabling Listening while Speaking in Real-time
By contrast, humans can listen to what hey speak in real-time, and if there is a delay in hearing, they won't be able to continue speaking.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Cross-Lingual Machine Speech Chain for Javanese, Sundanese, Balinese, and Bataks Speech Recognition and Synthesis
We then develop ASR and TTS of ethnic languages by utilizing Indonesian ASR and TTS in a cross-lingual machine speech chain framework with only text or only speech data removing the need for paired speech-text data of those ethnic languages.
Sequence-to-Sequence Learning via Attention Transfer for Incremental Speech Recognition
One main reason is because the model needs to decide the incremental steps and learn the transcription that aligns with the current short speech segment.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Augmenting Images for ASR and TTS through Single-loop and Dual-loop Multimodal Chain Framework
Previous research has proposed a machine speech chain to enable automatic speech recognition (ASR) and text-to-speech synthesis (TTS) to assist each other in semi-supervised learning and to avoid the need for a large amount of paired speech and text data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Simultaneous Speech-to-Speech Translation System with Neural Incremental ASR, MT, and TTS
This paper presents a newly developed, simultaneous neural speech-to-speech translation system and its evaluation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Improving Spoken Language Understanding by Wisdom of Crowds
Spoken language understanding (SLU), which converts user requests in natural language to machine-interpretable expressions, is becoming an essential task.
Incorporating Noisy Length Constraints into Transformer with Length-aware Positional Encodings
Since length constraints with exact target sentence lengths degrade translation performance, we add random noise within a certain window size to the length constraints in the PE during the training.
ARTA: Collection and Classification of Ambiguous Requests and Thoughtful Actions
In order to train the classification model on such training data, we applied the positive/unlabeled (PU) learning method, which assumes that only a part of the data is labeled with positive examples.
Using Perturbed Length-aware Positional Encoding for Non-autoregressive Neural Machine Translation
Non-autoregressive neural machine translation (NAT) usually employs sequence-level knowledge distillation using autoregressive neural machine translation (AT) as its teacher model.
Simultaneous Neural Machine Translation with Constituent Label Prediction
Simultaneous translation is a task in which translation begins before the speaker has finished speaking, so it is important to decide when to start the translation process.
Applying Syntax$\unicode{x2013}$Prosody Mapping Hypothesis and Prosodic Well-Formedness Constraints to Neural Sequence-to-Sequence Speech Synthesis
End-to-end text-to-speech synthesis (TTS), which generates speech sounds directly from strings of texts or phonemes, has improved the quality of speech synthesis over the conventional TTS.
Representing 'how you say' with 'what you say': English corpus of focused speech and text reflecting corresponding implications
As a type of paralinguistic information, English speech uses sentence stress, the heaviest prominence within a sentence, to convey emphasis.
Speech Segmentation Optimization using Segmented Bilingual Speech Corpus for End-to-end Speech Translation
We also propose a hybrid method that combines VAD and the above speech segmentation method.
Improved Consistency Training for Semi-Supervised Sequence-to-Sequence ASR via Speech Chain Reconstruction and Self-Transcribing
The existing paradigm of semi-supervised S2S ASR utilizes SpecAugment as data augmentation and requires a static teacher model to produce pseudo transcripts for untranscribed speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Speech Artifact Removal from EEG Recordings of Spoken Word Production with Tensor Decomposition
Research about brain activities involving spoken word production is considerably underdeveloped because of the undiscovered characteristics of speech artifacts, which contaminate electroencephalogram (EEG) signals and prevent the inspection of the underlying cognitive processes.
USB: A Unified Semi-supervised Learning Benchmark for Classification
We further provide the pre-trained versions of the state-of-the-art neural models for CV tasks to make the cost affordable for further tuning.
Actor-identified Spatiotemporal Action Detection --- Detecting Who Is Doing What in Videos
By using MOT, the spatiotemporal boundary of each actor is obtained and assigned to a unique actor identity.
Subspace Representations for Soft Set Operations and Sentence Similarities
In the field of natural language processing (NLP), continuous vector representations are crucial for capturing the semantic meanings of individual words.
E2E Refined Dataset
Although the well-known MR-to-text E2E dataset has been used by many researchers, its MR-text pairs include many deletion/insertion/substitution errors.
Average Token Delay: A Latency Metric for Simultaneous Translation
In this work, we propose a novel latency evaluation metric called Average Token Delay (ATD) that focuses on the end timings of partial translations in simultaneous translation.
Instance-level Heterogeneous Domain Adaptation for Limited-labeled Sketch-to-Photo Retrieval
Therefore, previous works pre-train their models on rich-labeled photo retrieval data (i. e., source domain) and then fine-tune them on the limited-labeled sketch-to-photo retrieval data (i. e., target domain).
Ranked #1 on
Image Retrieval
on PKU-Reid
SpeeChain: A Speech Toolkit for Large-Scale Machine Speech Chain
This paper introduces SpeeChain, an open-source Pytorch-based toolkit designed to develop the machine speech chain for large-scale use.
Evaluating the Robustness of Discrete Prompts
To address this question, we conduct a systematic study of the robustness of discrete prompts by applying carefully designed perturbations into an application using AutoPrompt and then measure their performance in two Natural Language Inference (NLI) datasets.
Whats New? Identifying the Unfolding of New Events in Narratives
We publish the annotated dataset, annotation materials, and machine learning baseline models for the task of new event extraction for narrative understanding.
Modeling Multiple User Interests using Hierarchical Knowledge for Conversational Recommender System
A conversational recommender system (CRS) is a practical application for item recommendation through natural language conversation.
Sketch-based Medical Image Retrieval
As a result, our SBMIR system enabled users to overcome previous challenges, including image retrieval based on fine-grained image characteristics, image retrieval without example images, and image retrieval for isolated samples.
NAIST-SIC-Aligned: an Aligned English-Japanese Simultaneous Interpretation Corpus
Research has been limited due to the lack of a large-scale training corpus.
Improving Speech Translation Accuracy and Time Efficiency with Fine-tuned wav2vec 2.0-based Speech Segmentation
In this study, we extended SHAS to improve ST translation accuracy and efficiency by splitting speech into shorter segments that correspond to sentences.
Arukikata Travelogue Dataset
We have constructed Arukikata Travelogue Dataset and released it free of charge for academic research.
Inter-connection: Effective Connection between Pre-trained Encoder and Decoder for Speech Translation
In this study, we propose an inter-connection mechanism that aggregates the information from each layer of the speech pre-trained model by weighted sums and inputs into the decoder.
Tagged End-to-End Simultaneous Speech Translation Training using Simultaneous Interpretation Data
In this paper, we propose an effective way to train a SimulST model using mixed data of SI and offline.
Computational analyses of linguistic features with schizophrenic and autistic traits along with formal thought disorders
We confirmed that an FTD-related subscale, odd speech, was significantly correlated with both the total SPQ and SRS scores, although they themselves were not correlated significantly.
Average Token Delay: A Duration-aware Latency Metric for Simultaneous Translation
In this work, we propose a novel latency evaluation metric for simultaneous translation called \emph{Average Token Delay} (ATD) that focuses on the duration of partial translations.
Response Generation for Cognitive Behavioral Therapy with Large Language Models: Comparative Study with Socratic Questioning
By comparing systems that use LLM-generated responses with those that do not, we investigate the impact of generated responses on subjective evaluations such as mood change, cognitive change, and dialogue quality (e. g., empathy).
TransLLaMa: LLM-based Simultaneous Translation System
Decoder-only large language models (LLMs) have recently demonstrated impressive capabilities in text generation and reasoning.
Findings of the IWSLT 2022 Evaluation Campaign
The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation.
Simultaneous Neural Machine Translation with Prefix Alignment
Simultaneous translation is a task that requires starting translation before the speaker has finished speaking, so we face a trade-off between latency and accuracy.
Pseudo Ambiguous and Clarifying Questions Based on Sentence Structures Toward Clarifying Question Answering System
Ambiguous questions are generated by eliminating a part of a sentence considering the sentence structure.
Is This Translation Error Critical?: Classification-Based Human and Automatic Machine Translation Evaluation Focusing on Critical Errors
Our classification-based approach focuses on such errors using several error type labels, for practical machine translation evaluation in an age of neural machine translation.
Large-Scale English-Japanese Simultaneous Interpretation Corpus: Construction and Analyses with Sentence-Aligned Data
This paper describes the construction of a new large-scale English-Japanese Simultaneous Interpretation (SI) corpus and presents the results of its analysis.
NAIST Simultaneous Speech-to-Text Translation System for IWSLT 2022
This paper describes NAIST’s simultaneous speech translation systems developed for IWSLT 2022 Evaluation Campaign.
Multi-paraphrase Augmentation to Leverage Neural Caption Translation
In this paper, we investigate and utilize neural paraphrasing to improve translation quality in neural MT (NMT), which has not yet been much explored.
Multi-Source Cross-Lingual Constituency Parsing
Pretrained multilingual language models have become a key part of cross-lingual transfer for many natural language processing tasks, even those without bilingual information.
Named Entity-Factored Transformer for Proper Noun Translation
The proposed method applies the Named Entity (NE) fea-ture vector to Factored Transformer for accurate proper noun translation.
FINDINGS OF THE IWSLT 2021 EVALUATION CAMPAIGN
The evaluation campaign of the International Conference on Spoken Language Translation (IWSLT 2021) featured this year four shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Multilingual speech translation, (iv) Low-resource speech translation.
NAIST English-to-Japanese Simultaneous Translation System for IWSLT 2021 Simultaneous Text-to-text Task
This paper describes NAIST’s system for the English-to-Japanese Simultaneous Text-to-text Translation Task in IWSLT 2021 Evaluation Campaign.
On Knowledge Distillation for Translating Erroneous Speech Transcriptions
Recent studies argue that knowledge distillation is promising for speech translation (ST) using end-to-end models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Using Spoken Word Posterior Features in Neural Machine Translation
In this paper, a neural sequence-to-sequence ASR is used as feature processing that is trained to produce word posterior features given spoken utterances.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Multilingual Machine Translation Evaluation Metrics Fine-tuned on Pseudo-Negative Examples for WMT 2021 Metrics Task
This paper describes our submission to the WMT2021 shared metrics task.
