Search Results for author: Sho Takase

Found 33 papers, 14 papers with code

Word-level Perturbation Considering Word Length and Compositional Subwords

1 code implementation Findings (ACL) 2022 Tatsuya Hiraoka, Sho Takase, Kei Uchiumi, Atsushi Keyaki, Naoaki Okazaki

We present two simple modifications for word-level perturbation: Word Replacement considering Length (WR-L) and Compositional Word Replacement (CWR). In conventional word replacement, a word in an input is replaced with a word sampled from the entire vocabulary, regardless of the length and context of the target word. WR-L considers the length of a target word by sampling words from the Poisson distribution. CWR considers the compositional candidates by restricting the source of sampling to related words that appear in subword regularization. Experimental results showed that the combination of WR-L and CWR improved the performance of text classification and machine translation.

Machine Translation text-classification +2

Nearest Neighbor Non-autoregressive Text Generation

no code implementations26 Aug 2022 Ayana Niwa, Sho Takase, Naoaki Okazaki

In addition, the proposed method outperforms an NAR baseline on the WMT'14 En-De dataset.

Machine Translation Text Generation +1

Are Neighbors Enough? Multi-Head Neural n-gram can be Alternative to Self-attention

no code implementations27 Jul 2022 Mengsay Loem, Sho Takase, Masahiro Kaneko, Naoaki Okazaki

Impressive performance of Transformer has been attributed to self-attention, where dependencies between entire input in a sequence are considered at every position.

Position

B2T Connection: Serving Stability and Performance in Deep Transformers

1 code implementation1 Jun 2022 Sho Takase, Shun Kiyono, Sosuke Kobayashi, Jun Suzuki

Recent Transformers tend to be Pre-LN because, in Post-LN with deep Transformers (e. g., those with ten or more layers), the training is often unstable, resulting in useless models.

Text Generation

Interpretability for Language Learners Using Example-Based Grammatical Error Correction

1 code implementation ACL 2022 Masahiro Kaneko, Sho Takase, Ayana Niwa, Naoaki Okazaki

In this study, we introduce an Example-Based GEC (EB-GEC) that presents examples to language learners as a basis for a correction result.

Grammatical Error Correction

ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization

no code implementations NAACL (ACL) 2022 Mengsay Loem, Sho Takase, Masahiro Kaneko, Naoaki Okazaki

Through experiments, we show that ExtraPhrase improves the performance of abstractive summarization tasks by more than 0. 50 points in ROUGE scores compared to the setting without data augmentation.

Abstractive Text Summarization Data Augmentation +1

Joint Optimization of Tokenization and Downstream Model

2 code implementations Findings (ACL) 2021 Tatsuya Hiraoka, Sho Takase, Kei Uchiumi, Atsushi Keyaki, Naoaki Okazaki

Since traditional tokenizers are isolated from a downstream task and model, they cannot output an appropriate tokenization depending on the task and model, although recent studies imply that the appropriate tokenization improves the performance.

Machine Translation text-classification +2

Multi-Task Learning for Cross-Lingual Abstractive Summarization

no code implementations LREC 2022 Sho Takase, Naoaki Okazaki

Experimental results indicate that Transum improves the performance from the strong baseline, Transformer, in Chinese-English, Arabic-English, and English-Japanese translation datasets.

Abstractive Text Summarization Cross-Lingual Abstractive Summarization +4

Improving Truthfulness of Headline Generation

1 code implementation ACL 2020 Kazuki Matsumaru, Sho Takase, Naoaki Okazaki

Building a binary classifier that predicts an entailment relation between an article and its headline, we filter out untruthful instances from the supervision data.

Abstractive Text Summarization Headline Generation +1

Evaluation Dataset for Zero Pronoun in Japanese to English Translation

no code implementations LREC 2020 Sho Shimazu, Sho Takase, Toshiaki Nakazawa, Naoaki Okazaki

Therefore, we present a hand-crafted dataset to evaluate whether translation models can resolve the zero pronoun problems in Japanese to English translations.

Machine Translation Translation

All Word Embeddings from One Embedding

1 code implementation NeurIPS 2020 Sho Takase, Sosuke Kobayashi

The proposed method, ALONE (all word embeddings from one), constructs the embedding of a word by modifying the shared embedding with a filter vector, which is word-specific but non-trainable.

Machine Translation Sentence Summarization +2

Character n-gram Embeddings to Improve RNN Language Models

no code implementations13 Jun 2019 Sho Takase, Jun Suzuki, Masaaki Nagata

This paper proposes a novel Recurrent Neural Network (RNN) language model that takes advantage of character information.

Headline Generation Language Modelling +3

Direct Output Connection for a High-Rank Language Model

1 code implementation EMNLP 2018 Sho Takase, Jun Suzuki, Masaaki Nagata

This paper proposes a state-of-the-art recurrent neural network (RNN) language model that combines probability distributions computed not only from a final RNN layer but also from middle layers.

Constituency Parsing Headline Generation +4

Input-to-Output Gate to Improve RNN Language Models

1 code implementation IJCNLP 2017 Sho Takase, Jun Suzuki, Masaaki Nagata

This paper proposes a reinforcing method that refines the output layers of existing Recurrent Neural Network (RNN) language models.

Composing Distributed Representations of Relational Patterns

1 code implementation ACL 2016 Sho Takase, Naoaki Okazaki, Kentaro Inui

Learning distributed representations for relation instances is a central technique in downstream NLP applications.

General Classification Relation +1

Cannot find the paper you are looking for? You can Submit a new open access paper.