Search Results for author:
Found 17 papers, 12 papers with code
Neurons in Large Language Models: Dead, N-gram, Positional
Specifically, we focus on the OPT family of models ranging from 125m to 66b parameters and rely only on whether an FFN neuron is activated or not.
HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translation
Hallucinations in machine translation are translations that contain information completely unrelated to the input.
Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better
We propose to use a method that evaluates the percentage of the source contribution to a generated translation.
Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation
Although the problem of hallucinations in neural machine translation (NMT) has received some attention, research on this highly pathological phenomenon lacks solid ground.
Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT
Differently from the traditional statistical MT that decomposes the translation task into distinct separately learned components, neural machine translation uses a single neural network to model the entire translation process.
Unsupervised Discovery of Interpretable Latent Manipulations in Language VAEs
In this work, we make the first step towards unsupervised discovery of interpretable directions in language latent spaces.
Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation
We find that models trained with more data tend to rely on source information more and to have more sharp token contributions; the training process is non-monotonic with several stages of different nature.
Embedding Words in Non-Vector Space with Unsupervised Graph Learning
We adopt a recent method learning a representation of data in the form of a differentiable weighted graph and use it to modify the GloVe training algorithm.
Information-Theoretic Probing with Minimum Description Length
Instead, we propose an alternative to the standard probes, information-theoretic probing with minimum description length (MDL).
Sequence Modeling with Unconstrained Generation Order
The dominant approach to sequence generation is to produce a sequence in some predefined order, e. g. left to right.
BPE-Dropout: Simple and Effective Subword Regularization
Subword segmentation is widely used to address the open vocabulary problem in machine translation.
Ranked #1 on
Machine Translation
on IWSLT2017 English-Arabic
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives
In this work, we use canonical correlation analysis and mutual information estimators to study how information flows across Transformer layers and how this process depends on the choice of learning objective.
Context-Aware Monolingual Repair for Neural Machine Translation
For training, the DocRepair model requires only monolingual document-level data in the target language.
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
Multi-head self-attention is a key component of the Transformer, a state-of-the-art architecture for neural machine translation.
When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion
Though machine translation errors caused by the lack of context beyond one sentence have long been acknowledged, the development of context-aware NMT systems is hampered by several problems.
A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation
We show that, while gains in BLEU are moderate for those systems, they outperform baselines by a large margin in terms of accuracy on our contrastive test set.
Context-Aware Neural Machine Translation Learns Anaphora Resolution
Standard machine translation systems process sentences in isolation and hence ignore extra-sentential information, even though extended context can both prevent mistakes in ambiguous cases and improve translation coherence.
