Search Results for author:
Found 20 papers, 13 papers with code
BPE-Dropout: Simple and Effective Subword Regularization
Subword segmentation is widely used to address the open vocabulary problem in machine translation.
Ranked #1 on
Machine Translation
on IWSLT2017 French-English
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
Multi-head self-attention is a key component of the Transformer, a state-of-the-art architecture for neural machine translation.
Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation
We find that models trained with more data tend to rely on source information more and to have more sharp token contributions; the training process is non-monotonic with several stages of different nature.
LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models
We present the LM Transparency Tool (LM-TT), an open-source interactive toolkit for analyzing the internal workings of Transformer-based language models.
Information Flow Routes: Automatically Interpreting Language Models at Scale
These routes can be represented as graphs where nodes correspond to token representations and edges to operations inside the network.
HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translation
Hallucinations in machine translation are translations that contain information completely unrelated to the input.
When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion
Though machine translation errors caused by the lack of context beyond one sentence have long been acknowledged, the development of context-aware NMT systems is hampered by several problems.
Context-Aware Monolingual Repair for Neural Machine Translation
For training, the DocRepair model requires only monolingual document-level data in the target language.
Information-Theoretic Probing with Minimum Description Length
Instead, we propose an alternative to the standard probes, information-theoretic probing with minimum description length (MDL).
Embedding Words in Non-Vector Space with Unsupervised Graph Learning
We adopt a recent method learning a representation of data in the form of a differentiable weighted graph and use it to modify the GloVe training algorithm.
A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation
We show that, while gains in BLEU are moderate for those systems, they outperform baselines by a large margin in terms of accuracy on our contrastive test set.
Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation
Although the problem of hallucinations in neural machine translation (NMT) has received some attention, research on this highly pathological phenomenon lacks solid ground.
Sequence Modeling with Unconstrained Generation Order
The dominant approach to sequence generation is to produce a sequence in some predefined order, e. g. left to right.
Context-Aware Neural Machine Translation Learns Anaphora Resolution
Standard machine translation systems process sentences in isolation and hence ignore extra-sentential information, even though extended context can both prevent mistakes in ambiguous cases and improve translation coherence.
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives
In this work, we use canonical correlation analysis and mutual information estimators to study how information flows across Transformer layers and how this process depends on the choice of learning objective.
Unsupervised Discovery of Interpretable Latent Manipulations in Language VAEs
In this work, we make the first step towards unsupervised discovery of interpretable directions in language latent spaces.
Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT
Differently from the traditional statistical MT that decomposes the translation task into distinct separately learned components, neural machine translation uses a single neural network to model the entire translation process.
Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better
We propose to use a method that evaluates the percentage of the source contribution to a generated translation.
Neurons in Large Language Models: Dead, N-gram, Positional
Specifically, we focus on the OPT family of models ranging from 125m to 66b parameters and rely only on whether an FFN neuron is activated or not.
Know When To Stop: A Study of Semantic Drift in Text Generation
Overall, our methods generalize and can be applied to any long-form text generation to produce more reliable information, by balancing trade-offs between factual accuracy, information quantity and computational cost.
