Search Results for author:
Found 52 papers, 23 papers with code
Findings of the WMT 2020 Shared Task on Machine Translation Robustness
We report the findings of the second edition of the shared task on improving robustness in Machine Translation (MT).
BERGAMOT-LATTE Submissions for the WMT20 Quality Estimation Shared Task
We explore (a) a black-box approach to QE based on pre-trained representations; and (b) glass-box approaches that leverage various indicators that can be extracted from the neural MT systems.
Findings of the WMT 2020 Shared Task on Quality Estimation
We report the results of the WMT20 shared task on Quality Estimation, where the challenge is to predict the quality of the output of neural machine translation systems at the word, sentence and document levels.
Findings of the WMT 2020 Shared Task on Parallel Corpus Filtering and Alignment
Following two preceding WMT Shared Task on Parallel Corpus Filtering (Koehn et al., 2018, 2019), we posed again the challenge of assigning sentence-level quality scores for very noisy corpora of sentence pairs crawled from the web, with the goal of sub-selecting the highest-quality data to be used to train ma-chine translation systems.
Findings of the AmericasNLP 2021 Shared Task on Open Machine Translation for Indigenous Languages of the Americas
This paper presents the results of the 2021 Shared Task on Open Machine Translation for Indigenous Languages of the Americas.
Findings of the 2021 Conference on Machine Translation (WMT21)
This paper presents the results of the newstranslation task, the multilingual low-resourcetranslation for Indo-European languages, thetriangular translation task, and the automaticpost-editing task organised as part of the Con-ference on Machine Translation (WMT) 2021. In the news task, participants were asked tobuild machine translation systems for any of10 language pairs, to be evaluated on test setsconsisting mainly of news stories.
Findings of the WMT 2021 Shared Task on Quality Estimation
We report the results of the WMT 2021 shared task on Quality Estimation, where the challenge is to predict the quality of the output of neural machine translation systems at the word and sentence levels.
Findings of the WMT 2021 Shared Task on Large-Scale Multilingual Machine Translation
There were a total of 10 participating teams for the tasks, with a total of 151 intermediate model submissions and 13 final models.
Efficient LLM Training and Serving with Heterogeneous Context Sharding among Attention Heads
Inspired by the sharding concept in database and the fact that attention parallelizes over heads on accelerators, we propose Sparsely-Sharded (S2) Attention, an attention algorithm that allocates heterogeneous context partitions for different attention heads to divide and conquer.
The Hitchhiker's Guide to Human Alignment with *PO
With the growing utilization of large language models (LLMs) across domains, alignment towards human preferences has become one of the most critical aspects of training models.
sPhinX: Sample Efficient Multilingual Instruction Fine-Tuning Through N-shot Guided Prompting
Despite the remarkable success of LLMs in English, there is a significant gap in performance in non-English languages.
Beyond Metrics: Evaluating LLMs' Effectiveness in Culturally Nuanced, Low-Resource Real-World Scenarios
The deployment of Large Language Models (LLMs) in real-world applications presents both opportunities and challenges, particularly in multilingual and code-mixed communication settings.
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.
Ranked #5 on
MMR total
on MRR-Benchmark
(using extra training data)
ODIN: A Single Model for 2D and 3D Segmentation
The gap in performance between methods that consume posed images versus post-processed 3D point clouds has fueled the belief that 2D and 3D perception require distinct model architectures.
Ranked #1 on
3D Instance Segmentation
on ScanNet200
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia
We present a novel method to study grounding abilities using Fakepedia, a novel dataset of counterfactual texts constructed to clash with a model's internal parametric knowledge.
Implicit Chain of Thought Reasoning via Knowledge Distillation
In this work, we explore an alternative reasoning approach: instead of explicitly producing the chain of thought reasoning steps, we use the language model's internal hidden states to perform implicit reasoning.
DUBLIN -- Document Understanding By Language-Image Network
Visual document understanding is a complex task that involves analyzing both the text and the visual elements in document images.
Ranked #1 on
Visual Question Answering (VQA)
on DeepForm
Language Is Not All You Need: Aligning Perception with Language Models
A big convergence of language, multimodal perception, action, and world modeling is a key step toward artificial general intelligence.
Understanding the Effectiveness of Very Large Language Models on Dialog Evaluation
The paper shows that the choice of datasets used for training a model contributes to how well it performs on a task as well as on how the prompt should be structured.
TorchScale: Transformers at Scale
Large Transformers have achieved state-of-the-art performance across many tasks.
Holistic Evaluation of Language Models
We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models.
Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning
In this paper, we elaborate upon recipes for building multilingual representation models that are not only competitive with existing state-of-the-art models but are also more parameter efficient, thereby promoting better adoption in resource-constrained scenarios and practical applications.
Language Model Decoding as Likelihood-Utility Alignment
Specifically, by analyzing the correlation between the likelihood and the utility of predictions across a diverse set of tasks, we provide empirical evidence supporting the proposed taxonomy and a set of principles to structure reasoning when choosing a decoding algorithm.
Foundation Transformers
A big convergence of model architectures across language, vision, speech, and multimodal is emerging.
How Robust is Neural Machine Translation to Language Imbalance in Multilingual Tokenizer Training?
Since a skewed data distribution is considered to be harmful, a sampling strategy is usually used to balance languages in the corpus.
Data Selection Curriculum for Neural Machine Translation
In this work, we introduce a two-stage curriculum training framework for NMT where we fine-tune a base NMT model on subsets of data, selected by both deterministic scoring using pre-trained methods and online scoring that considers prediction scores of the emerging NMT model.
OCR Improves Machine Translation for Low-Resource Languages
We aim to investigate the performance of current OCR systems on low resource languages and low resource scripts.
Few-shot Learning with Multilingual Language Models
Large-scale generative language models such as GPT-3 are competitive few-shot learners.
Alternative Input Signals Ease Transfer in Multilingual Machine Translation
Further, we find that incorporating alternative inputs via self-ensemble can be particularly effective when training set is small, leading to +5 BLEU when only 5% of the total training data is accessible.
Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications
Sentence-level Quality estimation (QE) of machine translation is traditionally formulated as a regression task, and the performance of QE models is typically measured by Pearson correlation with human labels.
LAWDR: Language-Agnostic Weighted Document Representations from Pre-trained Models
Cross-lingual document representations enable language understanding in multilingual contexts and allow transfer learning from high-resource to low-resource languages at the document level.
The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation
One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks.
Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data
The scarcity of parallel data is a major obstacle for training high-quality machine translation systems for low-resource languages.
AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages
Continued pretraining offers improvements, with an average accuracy of 44. 05%.
Quality Estimation without Human-labeled Data
Quality estimation aims to measure the quality of translated content without access to a reference translation.
An Exploratory Study on Multilingual Quality Estimation
Predicting the quality of machine translation has traditionally been addressed with language-specific models, under the assumption that the quality label distribution or linguistic features exhibit traits that are not shared across languages.
Beyond English-Centric Multilingual Machine Translation
Existing work in translation demonstrated the potential of massively multilingual machine translation by training a single model able to translate between any pair of languages.
MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset
We present MLQE-PE, a new dataset for Machine Translation (MT) Quality Estimation (QE) and Automatic Post-Editing (APE).
Self-training Improves Pre-training for Natural Language Understanding
Unsupervised pre-training has led to much recent progress in natural language understanding.
Multilingual Translation with Extensible Multilingual Pretraining and Finetuning
Recent work demonstrates the potential of multilingual pretraining of creating one model that can be used for various tasks in different languages.
Unsupervised Quality Estimation for Neural Machine Translation
Quality Estimation (QE) is an important component in making Machine Translation (MT) useful in real-world applications, as it is aimed to inform the user on the quality of the MT output at test time.
CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairs
We mine sixty-eight snapshots of the Common Crawl corpus and identify web document pairs that are translations of each other.
Unsupervised Cross-lingual Representation Learning at Scale
We also present a detailed empirical analysis of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and low resource languages at scale.
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
Pre-training text representations have led to significant improvements in many areas of natural language processing.
The FLORES Evaluation Datasets for Low-Resource Machine Translation: Nepali--English and Sinhala--English
For machine translation, a vast majority of language pairs in the world are considered low-resource because they have little parallel data available.
Facebook AI's WAT19 Myanmar-English Translation Task Submission
This paper describes Facebook AI's submission to the WAT 2019 Myanmar-English translation task.
Findings of the WMT 2019 Shared Task on Parallel Corpus Filtering for Low-Resource Conditions
Following the WMT 2018 Shared Task on Parallel Corpus Filtering, we posed the challenge of assigning sentence-level quality scores for very noisy corpora of sentence pairs crawled from the web, with the goal of sub-selecting 2{\%} and 10{\%} of the highest-quality data to be used to train machine translation systems.
WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia
We present an approach based on multilingual sentence embeddings to automatically extract parallel sentences from the content of Wikipedia articles in 85 languages, including several dialects or low-resource languages.
The FLoRes Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English
For machine translation, a vast majority of language pairs in the world are considered low-resource because they have little parallel data available.
