Search Results for author:
Found 15 papers, 10 papers with code
PARADISE”:" Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretraining
Despite the success of multilingual sequence-to-sequence pretraining, most existing approaches rely on monolingual corpora and do not make use of the strong cross-lingual signal contained in parallel data.
On the Impact of Data Augmentation on Downstream Performance in Natural Language Processing
The reason for this is that within NLP, the impact of proposed data augmentation methods on performance has not been evaluated in a unified manner, and effective data augmentation methods are unclear.
Low-Resource Machine Translation Using Cross-Lingual Language Model Pretraining
This paper describes UTokyo’s submission to the AmericasNLP 2021 Shared Task on machine translation systems for indigenous languages of the Americas.
Learning to Model Editing Processes
We introduce baseline results and metrics on this task, finding that modeling editing processes improves performance on a variety of axes on both our proposed task and related downstream tasks compared to previous single-step models of edits.
Large Language Models are Zero-Shot Reasoners
Pretrained large language models (LLMs) are widely used in many sub-fields of natural language processing (NLP) and generally known as excellent few-shot learners with task-specific exemplars.
Ranked #1 on
Arithmetic Reasoning
on MultiArith
A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation
We focus on two questions: 1) How can pre-trained models be used for languages not included in the initial pre-training?
Can Wikipedia Help Offline Reinforcement Learning?
In this paper, we look to take advantage of this formulation of reinforcement learning as sequence modeling and investigate the transferability of pre-trained sequence models on other domains (vision, language) when finetuned on offline RL tasks (control, games).
AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages
Reproducible benchmarks are crucial in driving progress of machine translation research.
PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretraining
Despite the success of multilingual sequence-to-sequence pretraining, most existing approaches rely on monolingual corpora, and do not make use of the strong cross-lingual signal contained in parallel data.
LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer
Moreover, compared to previous methods on unsupervised data synthesis, our method results in higher quality parallel style pairs and improves model performance.
Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformers
In light of this, we explore parameter-sharing methods in Transformers with a specific focus on generative models.
Subformer: A Parameter Reduced Transformer
We also perform equally well as Transformer-big with 40% less parameters and outperform the model by 0. 7 BLEU with 12M less parameters.
Ranked #20 on
Machine Translation
on WMT2014 English-German
VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling
In this paper, we tackle the task of definition modeling, where the goal is to learn to generate definitions of words and phrases.
Variational Inference for Learning Representations of Natural Language Edits
Document editing has become a pervasive component of the production of information, with version control systems enabling edits to be efficiently stored and applied.
Combining Pretrained High-Resource Embeddings and Subword Representations for Low-Resource Languages
The contrast between the need for large amounts of data for current Natural Language Processing (NLP) techniques, and the lack thereof, is accentuated in the case of African languages, most of which are considered low-resource.
