Search Results for author:
Found 55 papers, 21 papers with code
ProQE: Proficiency-wise Quality Estimation dataset for Grammatical Error Correction
This study investigates how supervised quality estimation (QE) models of grammatical error correction (GEC) are affected by the learners’ proficiency with the data.
Studying The Impact Of Document-level Context On Simultaneous Neural Machine Translation
To this end and in this paper and we propose wait-k simultaneous document-level NMT where we keep the context encoder as it is and replace the source sentence encoder and target language decoder with their wait-k equivalents.
Debiasing Isn’t Enough! – on the Effectiveness of Debiasing MLMs and Their Social Biases in Downstream Tasks
We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for MLMs, and find that there exists only a weak correlation between these two types of evaluation measures.
TMU Japanese-English Multimodal Machine Translation System for WAT 2020
Subsequently, we pretrain a translation model on the augmented noisy data, and then fine-tune it on the clean data.
Translation of New Named Entities from English to Chinese
New things are being created and new words are constantly being added to languages worldwide.
Sense Embeddings are also Biased – Evaluating Social Biases in Static and Contextualised Sense Embeddings
Sense embedding learning methods learn different embeddings for the different senses of an ambiguous word.
IMPARA: Impact-Based Metric for GEC Using Parallel Data
Correlations between IMPARA and human scores indicate that IMPARA is comparable or better than existing evaluation methods.
Sampling-based Pseudo-Likelihood for Membership Inference Attacks
In this study, we propose a Sampling-based Pseudo-Likelihood (\textbf{SPL}) method for MIA (\textbf{SaMIA}) that calculates SPL using only the text generated by an LLM to detect leaks.
A Little Leak Will Sink a Great Ship: Survey of Transparency for Large Language Models from Start to Finish
In this paper, we conduct an experimental survey to elucidate the relationship between the leakage rate and both the output rate and detection rate for personal information, copyrighted texts, and benchmark data.
Likelihood-based Mitigation of Evaluation Bias in Large Language Models
In this paper, we investigate the presence and impact of likelihood bias in LLM-based evaluators.
Eagle: Ethical Dataset Given from Real Interactions
The existing evaluation metrics and methods to address these ethical challenges use datasets intentionally created by instructing humans to create instances including ethical problems.
Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting
In this study, we examine the impact of LLMs' step-by-step predictions on gender bias in unscalable tasks.
The Gaps between Pre-train and Downstream Settings in Bias Evaluation and Debiasing
Moreover, the performance degradation due to debiasing is also lower in the ICL case compared to that in the FT case.
SAIE Framework: Support Alone Isn't Enough -- Advancing LLM Training with Adversarial Remarks
Large Language Models (LLMs) can justify or critique their predictions through discussions with other models or humans, thereby enriching their intrinsic understanding of instances.
How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detection
Furthermore, our analysis indicates that the high instruction-following ability of LLMs fosters the large impact of such constraints on detection performance.
Controlled Generation with Prompt Insertion for Natural Language Explanations in Grammatical Error Correction
Generating explanations for GEC corrections involves aligning input and output tokens, identifying correction points, and presenting corresponding explanations consistently.
Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labels
In Natural Language Inference (NLI), existing bias evaluation methods have focused on the prediction results of a specific label out of three labels, such as neutral.
The Impact of Debiasing on the Performance of Language Models in Downstream Tasks is Underestimated
In this study, we compare the impact of debiasing on performance across multiple downstream tasks using a wide-range of benchmark datasets that containing female, male, and stereotypical words.
In-Contextual Gender Bias Suppression for Large Language Models
We show that, using CrowsPairs dataset, our textual preambles covering counterfactual statements can suppress gender biases in English LLMs such as LLaMA2.
OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples
Experiments in the domain of student essays show that the proposed detector improves the detection performance on the attacker-generated texts by up to +41. 3 points F1-score.
Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methods
Large-scale pre-trained language models such as GPT-3 have shown remarkable performance across various natural language processing tasks.
Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach
Humans work together to solve common problems by having discussions, explaining, and agreeing or disagreeing with each other.
Reducing Sequence Length by Predicting Edit Operations with Large Language Models
Experiments show that the proposed method achieves comparable performance to the baseline in four tasks, paraphrasing, formality style transfer, GEC, and text simplification, despite reducing the length of the target text by as small as 21%.
Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples
Prior works have relied on human annotated examples to compare existing intrinsic bias evaluation measures.
Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks
We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for Masked Language Models (MLMs), and find that there exists only a weak correlation between these two types of evaluation measures.
Are Neighbors Enough? Multi-Head Neural n-gram can be Alternative to Self-attention
Impressive performance of Transformer has been attributed to self-attention, where dependencies between entire input in a sequence are considered at every position.
Gender Bias in Meta-Embeddings
Different methods have been proposed to develop meta-embeddings from a given set of source embeddings.
Gender Bias in Masked Language Models for Multiple Languages
Unfortunately, it was reported that MLMs also learn discriminative biases regarding attributes such as gender and race.
Interpretability for Language Learners Using Example-Based Grammatical Error Correction
In this study, we introduce an Example-Based GEC (EB-GEC) that presents examples to language learners as a basis for a correction result.
Sense Embeddings are also Biased--Evaluating Social Biases in Static and Contextualised Sense Embeddings
Sense embedding learning methods learn different embeddings for the different senses of an ambiguous word.
Proficiency Matters Quality Estimation in Grammatical Error Correction
This study investigates how supervised quality estimation (QE) models of grammatical error correction (GEC) are affected by the learners' proficiency with the data.
ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization
Through experiments, we show that ExtraPhrase improves the performance of abstractive summarization tasks by more than 0. 50 points in ROUGE scores compared to the setting without data augmentation.
Sentence Concatenation Approach to Data Augmentation for Neural Machine Translation
It is assumed that this issue is caused by insufficient number of long sentences in the training data.
Comparison of Grammatical Error Correction Using Back-Translation Models
Therefore, GEC studies have developed various methods to generate pseudo data, which comprise pairs of grammatical and artificially produced ungrammatical sentences.
Unmasking the Mask -- Evaluating Social Biases in Masked Language Models
To overcome the above-mentioned disfluencies, we propose All Unmasked Likelihood (AUL), a bias evaluation measure that predicts all tokens in a test case given the MLM embedding of the unmasked input.
Simultaneous Multi-Pivot Neural Machine Translation
Parallel corpora are indispensable for training neural machine translation (NMT) models, and parallel corpora for most language pairs do not exist or are scarce.
Dictionary-based Debiasing of Pre-trained Word Embeddings
Word embeddings trained on large corpora have shown to encode high levels of unfair discriminatory gender, racial, religious and ethnic biases.
Debiasing Pre-trained Contextualised Embeddings
In comparison to the numerous debiasing methods proposed for the static non-contextualised word embeddings, the discriminative biases in contextualised embeddings have received relatively little attention.
SOME: Reference-less Sub-Metrics Optimized for Manual Evaluations of Grammatical Error Correction
We propose a reference-less metric trained on manual evaluations of system outputs for grammatical error correction (GEC).
Generating Diverse Corrections with Local Beam Search for Grammatical Error Correction
In this study, we propose a beam search method to obtain diverse outputs in a local sequence transduction task where most of the tokens in the source and target sentences overlap, such as in grammatical error correction (GEC).
Cross-lingual Transfer Learning for Grammatical Error Correction
Cross-lingual transfer learning from high-resource languages (the source models) is effective for training models of low-resource languages (the target models) for various tasks.
Autoencoding Improves Pre-trained Word Embeddings
Prior work investigating the geometry of pre-trained word embeddings have shown that word embeddings to be distributed in a narrow cone and by centering and projecting using principal component vectors one can increase the accuracy of a given set of pre-trained word embeddings.
A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction
Existing approaches for grammatical error correction (GEC) largely rely on supervised learning with manually created GEC datasets.
English-to-Japanese Diverse Translation by Combining Forward and Backward Outputs
We introduce our TMU system that is submitted to The 4th Workshop on Neural Generation and Translation (WNGT2020) to English-to-Japanese (En→Ja) track on Simultaneous Translation And Paraphrase for Language Education (STAPLE) shared task.
Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction
The answer to this question is not as straightforward as one might expect because the previous common methods for incorporating a MLM into an EncDec model have potential drawbacks when applied to GEC.
Ranked #2 on
Grammatical Error Correction
on JFLEG
Towards Multimodal Simultaneous Neural Machine Translation
Simultaneous translation involves translating a sentence before the speaker's utterance is completed in order to realize real-time understanding in multiple languages.
Japanese-Russian TMU Neural Machine Translation System using Multilingual Model for WAT 2019
We introduce our system that is submitted to the News Commentary task (Japanese{\textless}-{\textgreater}Russian) of the 6th Workshop on Asian Translation.
Grammatical-Error-Aware Incorrect Example Retrieval System for Learners of Japanese as a Second Language
Existing example retrieval systems do not include grammatically incorrect examples or present only a few examples, if any.
TMU Transformer System Using BERT for Re-ranking at BEA 2019 Grammatical Error Correction on Restricted Track
Thus, it is not straightforward to utilize language representations trained from a large corpus, such as Bidirectional Encoder Representations from Transformers (BERT), in a form suitable for the learner{'}s grammatical errors.
Controlling Grammatical Error Correction Using Word Edit Rate
In this paper, we propose a method for neural grammar error correction (GEC) that can control the degree of correction.
Gender-preserving Debiasing for Pre-trained Word Embeddings
Word embeddings learnt from massive text collections have demonstrated significant levels of discriminative biases such as gender, racial or ethnic biases, which in turn bias the down-stream NLP applications that use those word embeddings.
Multi-Head Multi-Layer Attention to Deep Language Representations for Grammatical Error Detection
In this work, we investigate the effect of utilizing information not only from the final layer but also from intermediate layers of a pre-trained language representation model to detect grammatical errors.
Cross-Corpora Evaluation and Analysis of Grammatical Error Correction Models --- Is Single-Corpus Evaluation Enough?
This study explores the necessity of performing cross-corpora evaluation for grammatical error correction (GEC) models.
TMU System for SLAM-2018
We introduce the TMU systems for the second language acquisition modeling shared task 2018 (Settles et al., 2018).
Grammatical Error Detection Using Error- and Grammaticality-Specific Word Embeddings
In this study, we improve grammatical error detection by learning word embeddings that consider grammaticality and error patterns.
Ranked #6 on
Grammatical Error Detection
on FCE
