Search Results for author:
Found 12 papers, 6 papers with code
Word-level Perturbation Considering Word Length and Compositional Subwords
We present two simple modifications for word-level perturbation: Word Replacement considering Length (WR-L) and Compositional Word Replacement (CWR). In conventional word replacement, a word in an input is replaced with a word sampled from the entire vocabulary, regardless of the length and context of the target word. WR-L considers the length of a target word by sampling words from the Poisson distribution. CWR considers the compositional candidates by restricting the source of sampling to related words that appear in subword regularization. Experimental results showed that the combination of WR-L and CWR improved the performance of text classification and machine translation.
Joint Entity and Relation Extraction Based on Table Labeling Using Convolutional Neural Networks
We adopt table representations to model the entities and relations, casting the entity and relation extraction as a table-labeling problem.
An Analysis of BPE Vocabulary Trimming in Neural Machine Translation
We explore threshold vocabulary trimming in Byte-Pair Encoding subword tokenization, a postprocessing step that replaces rare subwords with their component subwords.
Knowledge of Pretrained Language Models on Surface Information of Tokens
Do pretrained language models have knowledge regarding the surface information of tokens?
Tokenization Preference for Human and Machine Learning Model: An Annotation Study
Is preferred tokenization for humans also preferred for machine-learning (ML) models?
Downstream Task-Oriented Neural Tokenizer Optimization with Vocabulary Restriction as Post Processing
This paper proposes an example of the BiLSTM-based tokenizer with vocabulary restriction, which can capture wider contextual information for the tokenization process than non-neural-based tokenization methods used in existing work.
MaxMatch-Dropout: Subword Regularization for WordPiece
We present a subword regularization method for WordPiece, which uses a maximum matching algorithm for tokenization.
Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation
Subword regularizations use multiple subword segmentations during training to improve the robustness of neural machine translation models.
Joint Optimization of Tokenization and Downstream Model
Since traditional tokenizers are isolated from a downstream task and model, they cannot output an appropriate tokenization depending on the task and model, although recent studies imply that the appropriate tokenization improves the performance.
Optimizing Word Segmentation for Downstream Task
In traditional NLP, we tokenize a given sentence as a preprocessing, and thus the tokenization is unrelated to a target downstream task.
Named Entity Recognition and Relation Extraction using Enhanced Table Filling by Contextualized Representations
In this study, a novel method for extracting named entities and relations from unstructured text based on the table representation is presented.
Ranked #3 on
Relation Extraction
on CoNLL04
(NER Micro F1 metric)
Stochastic Tokenization with a Language Model for Neural Text Classification
To make the model robust against infrequent tokens, we sampled segmentation for each sentence stochastically during training, which resulted in improved performance of text classification.
