Search Results for author: Graham Neubig

Found 258 papers, 145 papers with code

Project MAIA: Multilingual AI Agent Assistant

no code implementations EAMT 2020 André F. T. Martins, Joao Graca, Paulo Dimas, Helena Moniz, Graham Neubig

This paper presents the Multilingual Artificial Intelligence Agent Assistant (MAIA), a project led by Unbabel with the collaboration of CMU, INESC-ID and IT Lisbon.

Translation

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalisation

2 code implementations ICML 2020 Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin Johnson

However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse range of languages and tasks is still missing.

Zero-Shot Cross-Lingual Transfer

Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations

1 code implementation17 Dec 2021 Siddhant Arora, Danish Pruthi, Norman Sadeh, William W. Cohen, Zachary C. Lipton, Graham Neubig

Through our evaluation, we observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.

Deception Detection

VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning

1 code implementation5 Dec 2021 Qibin Chen, Jeremy Lacomis, Edward J. Schwartz, Graham Neubig, Bogdan Vasilescu, Claire Le Goues

Machine learning-based program analysis methods use variable name representations for a wide range of tasks, such as suggesting new variable names and bug detection.

Contrastive Learning Learning Semantic Representations +1

DEEP: DEnoising Entity Pre-training for Neural Machine Translation

no code implementations14 Nov 2021 Junjie Hu, Hiroaki Hayashi, Kyunghyun Cho, Graham Neubig

It has been shown that machine translation models usually generate poor translations for named entities that are infrequent in the training corpus.

Denoising Multi-Task Learning +2

Lexically Aware Semi-Supervised Learning for OCR Post-Correction

1 code implementation4 Nov 2021 Shruti Rijhwani, Daisy Rosenblum, Antonios Anastasopoulos, Graham Neubig

In addition, to enforce consistency in the recognized vocabulary, we introduce a lexically-aware decoding method that augments the neural post-correction model with a count-based language model constructed from the recognized texts, implemented using weighted finite-state automata (WFSA) for efficient and effective decoding.

Language Modelling Optical Character Recognition

Breaking Down Multilingual Machine Translation

no code implementations15 Oct 2021 Ting-Rui Chiang, Yi-Pei Chen, Yi-Ting Yeh, Graham Neubig

While multilingual training is now an essential ingredient in machine translation (MT) systems, recent work has demonstrated that it has different effects in different multilingual settings, such as many-to-one, one-to-many, and many-to-many learning.

Machine Translation Translation

On The Ingredients of an Effective Zero-shot Semantic Parser

no code implementations15 Oct 2021 Pengcheng Yin, John Wieting, Avirup Sil, Graham Neubig

Semantic parsers map natural language utterances into meaning representations (e. g., programs).

Semantic Parsing Zero-Shot Learning

Systematic Inequalities in Language Technology Performance across the World's Languages

1 code implementation13 Oct 2021 Damián Blasi, Antonios Anastasopoulos, Graham Neubig

Natural language processing (NLP) systems have become a central technology in communication, education, medicine, artificial intelligence, and many other domains of research and development.

Dependency Parsing Machine Translation +5

Towards a Unified View of Parameter-Efficient Transfer Learning

1 code implementation8 Oct 2021 Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, Graham Neubig

Furthermore, our unified framework enables the transfer of design elements across different approaches, and as a result we are able to instantiate new parameter-efficient fine-tuning methods that tune less parameters than previous methods while being more effective, achieving comparable results to fine-tuning all parameters on all four tasks.

Machine Translation Text Classification +2

Capturing Structural Locality in Non-parametric Language Models

no code implementations6 Oct 2021 Frank F. Xu, Junxian He, Graham Neubig, Vincent J. Hellendoorn

Structural locality is a ubiquitous feature of real-world datasets, wherein data points are organized into local hierarchies.

Learning to Superoptimize Real-world Programs

no code implementations28 Sep 2021 Alex Shypula, Pengcheng Yin, Jeremy Lacomis, Claire Le Goues, Edward Schwartz, Graham Neubig

We also report that SILO's rate of superoptimization on our test set is over five times that of a standard policy gradient approach and a model pre-trained on compiler optimization demonstration.

Imitation Learning

Dependency Induction Through the Lens of Visual Perception

1 code implementation CoNLL (EMNLP) 2021 Ruisi Su, Shruti Rijhwani, Hao Zhu, Junxian He, Xinyu Wang, Yonatan Bisk, Graham Neubig

Our experiments find that concreteness is a strong indicator for learning dependency grammars, improving the direct attachment score (DAS) by over 50\% as compared to state-of-the-art models trained on pure text.

Constituency Grammar Induction Dependency Parsing

Hierarchical Control of Situated Agents through Natural Language

no code implementations16 Sep 2021 Shuyan Zhou, Pengcheng Yin, Graham Neubig

When humans conceive how to perform a particular task, they do so hierarchically: splitting higher-level tasks into smaller sub-tasks.

Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative

no code implementations15 Sep 2021 Lucio M. Dery, Paul Michel, Ameet Talwalkar, Graham Neubig

First, on three different low-resource NLP tasks from two domains, we demonstrate that multi-tasking the end-task and auxiliary objectives results in significantly better downstream task performance than the widely-used task-agnostic continued pre-training paradigm of Gururangan et al. (2020).

Meta-Learning

When Does Translation Require Context? A Data-driven, Multilingual Exploration

no code implementations15 Sep 2021 Kayo Yin, Patrick Fernandes, André F. T. Martins, Graham Neubig

Although proper handling of discourse phenomena significantly contributes to the quality of machine translation (MT), common translation quality metrics do not adequately capture them.

Machine Translation Translation

When is Wall a Pared and when a Muro? -- Extracting Rules Governing Lexical Selection

1 code implementation13 Sep 2021 Aditi Chaudhary, Kayo Yin, Antonios Anastasopoulos, Graham Neubig

Learning fine-grained distinctions between vocabulary items is a key challenge in learning a new language.

Efficient Nearest Neighbor Language Models

1 code implementation EMNLP 2021 Junxian He, Graham Neubig, Taylor Berg-Kirkpatrick

Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore, which allows them to learn through explicitly memorizing the training datapoints.

Domain Adaptation Language Modelling

Distributionally Robust Multilingual Machine Translation

1 code implementation EMNLP 2021 Chunting Zhou, Daniel Levy, Xian Li, Marjan Ghazvininejad, Graham Neubig

Multilingual neural machine translation (MNMT) learns to translate multiple language pairs with a single model, potentially improving both the accuracy and the memory-efficiency of deployed models.

Machine Translation Translation

Few-shot Language Coordination by Modeling Theory of Mind

no code implementations12 Jul 2021 Hao Zhu, Graham Neubig, Yonatan Bisk

Positive results from our experiments hint at the importance of explicitly modeling communication as a socio-pragmatic progress.

BARTScore: Evaluating Generated Text as Text Generation

1 code implementation NeurIPS 2021 Weizhe Yuan, Graham Neubig, PengFei Liu

In this work, we conceptualize the evaluation of generated text as a text generation problem, modeled using pre-trained sequence-to-sequence models.

Machine Translation Text Generation +2

Examining and Combating Spurious Features under Distribution Shift

1 code implementation14 Jun 2021 Chunting Zhou, Xuezhe Ma, Paul Michel, Graham Neubig

Group distributionally robust optimization (DRO) provides an effective tool to alleviate covariate shift by minimizing the worst-case training loss over a set of pre-defined groups.

CitationIE: Leveraging the Citation Graph for Scientific Information Extraction

1 code implementation ACL 2021 Vijay Viswanathan, Graham Neubig, PengFei Liu

Automatically extracting key information from scientific documents has the potential to help scientists work more efficiently and accelerate the pace of scientific progress.

Data Augmentation for Sign Language Gloss Translation

no code implementations MTSummit 2021 Amit Moryossef, Kayo Yin, Graham Neubig, Yoav Goldberg

Sign language translation (SLT) is often decomposed into video-to-gloss recognition and gloss-to-text translation, where a gloss is a sequence of transcribed spoken-language words in the order in which they are signed.

Data Augmentation Low-Resource Neural Machine Translation +2

Measuring and Increasing Context Usage in Context-Aware Machine Translation

1 code implementation ACL 2021 Patrick Fernandes, Kayo Yin, Graham Neubig, André F. T. Martins

Recent work in neural machine translation has demonstrated both the necessity and feasibility of using inter-sentential context -- context from sentences other than those currently being translated.

Document Level Machine Translation Machine Translation +1

Paraphrastic Representations at Scale

1 code implementation30 Apr 2021 John Wieting, Kevin Gimpel, Graham Neubig, Taylor Berg-Kirkpatrick

We train these models on large amounts of data, achieving significantly improved performance from the original papers proposing the methods on a suite of monolingual semantic similarity, cross-lingual semantic similarity, and bitext mining tasks.

Semantic Similarity Semantic Textual Similarity

MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning

1 code implementation NAACL 2021 Mengzhou Xia, Guoqing Zheng, Subhabrata Mukherjee, Milad Shokouhi, Graham Neubig, Ahmed Hassan Awadallah

Extensive experiments on real-world low-resource languages - without access to large-scale monolingual corpora or large amounts of labeled data - for tasks like cross-lingual sentiment analysis and named entity recognition show the effectiveness of our approach.

Cross-Lingual Transfer Meta-Learning +3

ExplainaBoard: An Explainable Leaderboard for NLP

1 code implementation ACL 2021 PengFei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaicheng Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, Zi-Yi Dou, Graham Neubig

In this paper, we present a new conceptualization and implementation of NLP evaluation: the ExplainaBoard, which in addition to inheriting the functionality of the standard leaderboard, also allows researchers to (i) diagnose strengths and weaknesses of a single system (e. g.~what is the best-performing system bad at?)

Machine Translation

Phoneme Recognition through Fine Tuning of Phonetic Representations: a Case Study on Luhya Language Varieties

no code implementations4 Apr 2021 Kathleen Siminyu, Xinjian Li, Antonios Anastasopoulos, David Mortensen, Michael R. Marlo, Graham Neubig

Models pre-trained on multiple languages have shown significant promise for improving speech recognition, particularly for low-resource languages.

Speech Recognition

Modeling the Second Player in Distributionally Robust Optimization

1 code implementation ICLR 2021 Paul Michel, Tatsunori Hashimoto, Graham Neubig

Distributionally robust optimization (DRO) provides a framework for training machine learning models that are able to perform well on a collection of related data distributions (the "uncertainty set").

Model Selection

Multi-view Subword Regularization

1 code implementation NAACL 2021 Xinyi Wang, Sebastian Ruder, Graham Neubig

Multilingual pretrained representations generally rely on subword segmentation algorithms to create a shared multilingual vocabulary.

Cross-Lingual Transfer

Meta Back-translation

1 code implementation ICLR 2021 Hieu Pham, Xinyi Wang, Yiming Yang, Graham Neubig

Back-translation is an effective strategy to improve the performance of Neural Machine Translation~(NMT) by generating pseudo-parallel data.

Machine Translation Meta-Learning +1

Towards More Fine-grained and Reliable NLP Performance Prediction

1 code implementation EACL 2021 Zihuiwen Ye, PengFei Liu, Jinlan Fu, Graham Neubig

We perform an analysis of four types of NLP tasks, and both demonstrate the feasibility of fine-grained performance prediction and the necessity to perform reliability analysis for performance prediction methods in the future.

Can We Automate Scientific Reviewing?

1 code implementation30 Jan 2021 Weizhe Yuan, PengFei Liu, Graham Neubig

The rapid development of science and technology has been accompanied by an exponential growth in peer-reviewed scientific publications.

Review Generation

Learning Structural Edits via Incremental Tree Transformations

1 code implementation ICLR 2021 Ziyu Yao, Frank F. Xu, Pengcheng Yin, Huan Sun, Graham Neubig

To show the unique benefits of modeling tree edits directly, we further propose a novel edit encoder for learning to represent edits, as well as an imitation learning method that allows the editor to be more robust.

Imitation Learning

In-IDE Code Generation from Natural Language: Promise and Challenges

no code implementations27 Jan 2021 Frank F. Xu, Bogdan Vasilescu, Graham Neubig

A great part of software development involves conceptualizing or communicating the underlying procedures and logic that needs to be expressed in programs.

Code Generation Data Visualization Software Engineering

Word Alignment by Fine-tuning Embeddings on Parallel Corpora

2 code implementations EACL 2021 Zi-Yi Dou, Graham Neubig

In addition, we demonstrate that we are able to train multilingual word aligners that can obtain robust performance on different language pairs.

Cross-Lingual Transfer Translation +2

How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering

1 code implementation2 Dec 2020 Zhengbao Jiang, Jun Araki, Haibo Ding, Graham Neubig

We examine this question from the point of view of calibration, the property of a probabilistic model's predicted probabilities actually being well correlated with the probabilities of correctness.

Common Sense Reasoning Question Answering

Evaluating Explanations: How much do explanations from the teacher aid students?

1 code implementation1 Dec 2020 Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary C. Lipton, Graham Neubig, William W. Cohen

While many methods purport to explain predictions by highlighting salient features, what aims these explanations serve and how they ought to be evaluated often go unstated.

Question Answering Text Classification

Automatic Interlinear Glossing for Under-Resourced Languages Leveraging Translations

no code implementations COLING 2020 Xingyuan Zhao, Satoru Ozaki, Antonios Anastasopoulos, Graham Neubig, Lori Levin

Interlinear Glossed Text (IGT) is a widely used format for encoding linguistic information in language documentation projects and scholarly papers.

Cross-Lingual Transfer TAG

Endangered Languages meet Modern NLP

no code implementations COLING 2020 Antonios Anastasopoulos, Christopher Cox, Graham Neubig, Hilaria Cruz

This tutorial will focus on NLP for endangered languages documentation and revitalization.

Decoding and Diversity in Machine Translation

no code implementations26 Nov 2020 Nicholas Roberts, Davis Liang, Graham Neubig, Zachary C. Lipton

This makes human-level BLEU a misleading benchmark in that modern MT systems cannot approach human-level BLEU while simultaneously maintaining human-level translation diversity.

Machine Translation Translation

WikiAsp: A Dataset for Multi-domain Aspect-based Summarization

1 code implementation16 Nov 2020 Hiroaki Hayashi, Prashant Budania, Peng Wang, Chris Ackerson, Raj Neervannan, Graham Neubig

In this paper, we propose WikiAsp, a large-scale dataset for multi-domain aspect-based summarization that attempts to spur research in the direction of open-domain aspect-based summarization.

Interpretable Multi-dataset Evaluation for Named Entity Recognition

1 code implementation EMNLP 2020 Jinlan Fu, PengFei Liu, Graham Neubig

With the proliferation of models for natural language processing tasks, it is even harder to understand the differences between models and their relative merits.

Named Entity Recognition NER

OCR Post Correction for Endangered Language Texts

1 code implementation EMNLP 2020 Shruti Rijhwani, Antonios Anastasopoulos, Graham Neubig

There is little to no data available to build natural language processing models for most endangered languages.

Optical Character Recognition

Detecting Hallucinated Content in Conditional Neural Sequence Generation

1 code implementation Findings (ACL) 2021 Chunting Zhou, Graham Neubig, Jiatao Gu, Mona Diab, Paco Guzman, Luke Zettlemoyer, Marjan Ghazvininejad

Neural sequence models can generate highly fluent sentences, but recent studies have also shown that they are also prone to hallucinate additional content not supported by the input.

Abstractive Text Summarization Machine Translation

Weakly- and Semi-supervised Evidence Extraction

1 code implementation Findings of the Association for Computational Linguistics 2020 Danish Pruthi, Bhuwan Dhingra, Graham Neubig, Zachary C. Lipton

For many prediction tasks, stakeholders desire not only predictions but also supporting evidence that a human can use to verify its correctness.

Reducing Confusion in Active Learning for Part-Of-Speech Tagging

no code implementations2 Nov 2020 Aditi Chaudhary, Antonios Anastasopoulos, Zaid Sheikh, Graham Neubig

Active learning (AL) uses a data selection algorithm to select useful training samples to minimize annotation cost.

Active Learning Part-Of-Speech Tagging +1

On Learning Text Style Transfer with Direct Rewards

1 code implementation NAACL 2021 Yixin Liu, Graham Neubig, John Wieting

In most cases, the lack of parallel corpora makes it impossible to directly train supervised models for the text style transfer task.

Machine Translation Semantic Similarity +4

Explicit Alignment Objectives for Multilingual Bidirectional Encoders

no code implementations NAACL 2021 Junjie Hu, Melvin Johnson, Orhan Firat, Aditya Siddhant, Graham Neubig

Pre-trained cross-lingual encoders such as mBERT (Devlin et al., 2019) and XLMR (Conneau et al., 2020) have proven to be impressively effective at enabling transfer-learning of NLP systems from high-resource languages to low-resource languages.

Sentence Classification Transfer Learning +1

GSum: A General Framework for Guided Neural Abstractive Summarization

1 code implementation NAACL 2021 Zi-Yi Dou, PengFei Liu, Hiroaki Hayashi, Zhengbao Jiang, Graham Neubig

Neural abstractive summarization models are flexible and can produce coherent summaries, but they are sometimes unfaithful and can be difficult to control.

Abstractive Text Summarization

Re-evaluating Evaluation in Text Summarization

1 code implementation EMNLP 2020 Manik Bhandari, Pranav Gour, Atabak Ashfaq, PengFei Liu, Graham Neubig

Automated evaluation metrics as a stand-in for manual evaluation are an essential part of the development of text-generation tasks such as text summarization.

Text Generation Text Summarization

X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models

1 code implementation EMNLP 2020 Zhengbao Jiang, Antonios Anastasopoulos, Jun Araki, Haibo Ding, Graham Neubig

We further propose a code-switching-based method to improve the ability of multilingual LMs to access knowledge, and verify its effectiveness on several benchmark languages.

Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation

no code implementations Findings of the Association for Computational Linguistics 2020 Luyu Gao, Xinyi Wang, Graham Neubig

To improve the performance of Neural Machine Translation~(NMT) for low-resource languages~(LRL), one effective strategy is to leverage parallel data from a related high-resource language~(HRL).

Machine Translation Translation

Automatic Extraction of Rules Governing Morphological Agreement

1 code implementation EMNLP 2020 Aditi Chaudhary, Antonios Anastasopoulos, Adithya Pratapa, David R. Mortensen, Zaid Sheikh, Yulia Tsvetkov, Graham Neubig

Using cross-lingual transfer, even with no expert annotations in the language of interest, our framework extracts a grammatical specification which is nearly equivalent to those created with large amounts of gold-standard annotated data.

Cross-Lingual Transfer

The Return of Lexical Dependencies: Neural Lexicalized PCFGs

3 code implementations29 Jul 2020 Hao Zhu, Yonatan Bisk, Graham Neubig

In this paper we demonstrate that $\textit{context free grammar (CFG) based methods for grammar induction benefit from modeling lexical dependencies}$.

Transliteration for Cross-Lingual Morphological Inflection

no code implementations WS 2020 Nikitha Murikinati, Antonios Anastasopoulos, Graham Neubig

Cross-lingual transfer between typologically related languages has been proven successful for the task of morphological inflection.

Cross-Lingual Transfer Morphological Inflection +1

Findings of the Fourth Workshop on Neural Generation and Translation

no code implementations WS 2020 Kenneth Heafield, Hiroaki Hayashi, Yusuke Oda, Ioannis Konstas, Andrew Finch, Graham Neubig, Xi-An Li, Alex Birch, ra

We describe the finding of the Fourth Workshop on Neural Generation and Translation, held in concert with the annual conference of the Association for Computational Linguistics (ACL 2020).

Machine Translation Translation

Learning Sparse Prototypes for Text Generation

1 code implementation NeurIPS 2020 Junxian He, Taylor Berg-Kirkpatrick, Graham Neubig

While effective, these methods are inefficient at test time as a result of needing to store and index the entire training corpus.

Language Modelling Prototype Selection +2

TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data

1 code implementation ACL 2020 Pengcheng Yin, Graham Neubig, Wen-tau Yih, Sebastian Riedel

Recent years have witnessed the burgeoning of pretrained language models (LMs) for text-based natural language (NL) understanding tasks.

Semantic Parsing Text-To-Sql

Soft Gazetteers for Low-Resource Named Entity Recognition

1 code implementation ACL 2020 Shruti Rijhwani, Shuyan Zhou, Graham Neubig, Jaime Carbonell

However, designing such features for low-resource languages is challenging, because exhaustive entity gazetteers do not exist in these languages.

Cross-Lingual Entity Linking Entity Linking +2

Predicting Performance for Natural Language Processing Tasks

1 code implementation ACL 2020 Mengzhou Xia, Antonios Anastasopoulos, Ruochen Xu, Yiming Yang, Graham Neubig

Given the complexity of combinations of tasks, languages, and domains in natural language processing (NLP) research, it is computationally prohibitive to exhaustively test newly proposed models on each possible experimental setting.

Politeness Transfer: A Tag and Generate Approach

1 code implementation ACL 2020 Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W. black, Shrimai Prabhumoye

This paper introduces a new task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning.

Style Transfer TAG

Practical Comparable Data Collection for Low-Resource Languages via Images

1 code implementation24 Apr 2020 Aman Madaan, Shruti Rijhwani, Antonios Anastasopoulos, Yiming Yang, Graham Neubig

We propose a method of curating high-quality comparable training data for low-resource languages with monolingual annotators.

Machine Translation Translation

Incorporating External Knowledge through Pre-training for Natural Language to Code Generation

1 code implementation ACL 2020 Frank F. Xu, Zhengbao Jiang, Pengcheng Yin, Bogdan Vasilescu, Graham Neubig

Open-domain code generation aims to generate code in a general-purpose programming language (such as Python) from natural language (NL) intents.

Ranked #2 on Code Generation on CoNaLa (using extra training data)

Code Generation Data Augmentation

AlloVera: A Multilingual Allophone Database

no code implementations LREC 2020 David R. Mortensen, Xinjian Li, Patrick Littell, Alexis Michaud, Shruti Rijhwani, Antonios Anastasopoulos, Alan W. black, Florian Metze, Graham Neubig

While phonemic representations are language specific, phonetic representations (stated in terms of (allo)phones) are much closer to a universal (language-independent) transcription.

Speech Recognition

Weight Poisoning Attacks on Pre-trained Models

1 code implementation14 Apr 2020 Keita Kurita, Paul Michel, Graham Neubig

We show that by applying a regularization method, which we call RIPPLe, and an initialization procedure, which we call Embedding Surgery, such attacks are possible even with limited knowledge of the dataset and fine-tuning procedure.

Sentiment Analysis

Balancing Training for Multilingual Neural Machine Translation

2 code implementations ACL 2020 Xinyi Wang, Yulia Tsvetkov, Graham Neubig

When training multilingual machine translation (MT) models that can translate to/from multiple languages, we are faced with imbalanced training sets: some languages have much more training data than others.

Machine Translation Translation

Dynamic Data Selection and Weighting for Iterative Back-Translation

1 code implementation EMNLP 2020 Zi-Yi Dou, Antonios Anastasopoulos, Graham Neubig

Back-translation has proven to be an effective method to utilize monolingual data in neural machine translation (NMT), and iteratively conducting back-translation can further improve the model performance.

Curriculum Learning Domain Adaptation +2

A Set of Recommendations for Assessing Human-Machine Parity in Language Translation

1 code implementation3 Apr 2020 Samuel Läubli, Sheila Castilho, Graham Neubig, Rico Sennrich, Qinlan Shen, Antonio Toral

The quality of machine translation has increased remarkably over the past years, to the degree that it was found to be indistinguishable from professional human translation in a number of empirical investigations.

Machine Translation Translation

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

2 code implementations24 Mar 2020 Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, Melvin Johnson

However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse range of languages and tasks is still missing.

Cross-Lingual Transfer

Improving Candidate Generation for Low-resource Cross-lingual Entity Linking

1 code implementation TACL 2020 Shuyan Zhou, Shruti Rijhawani, John Wieting, Jaime Carbonell, Graham Neubig

Cross-lingual entity linking (XEL) is the task of finding referents in a target-language knowledge base (KB) for mentions extracted from source-language texts.

Cross-Lingual Entity Linking Entity Linking +1

Differentiable Reasoning over a Virtual Knowledge Base

1 code implementation ICLR 2020 Bhuwan Dhingra, Manzil Zaheer, Vidhisha Balachandran, Graham Neubig, Ruslan Salakhutdinov, William W. Cohen

In particular, we describe a neural module, DrKIT, that traverses textual data like a KB, softly following paths of relations between mentions of entities in the corpus.

Re-Ranking

A Probabilistic Formulation of Unsupervised Text Style Transfer

4 code implementations ICLR 2020 Junxian He, Xinyi Wang, Graham Neubig, Taylor Berg-Kirkpatrick

Across all style transfer tasks, our approach yields substantial gains over state-of-the-art non-generative baselines, including the state-of-the-art unsupervised machine translation techniques that our approach generalizes.

Decipherment Language Modelling +6

Merging Weak and Active Supervision for Semantic Parsing

1 code implementation29 Nov 2019 Ansong Ni, Pengcheng Yin, Graham Neubig

Experiments on WikiTableQuestions with human annotators show that our method can improve the performance with only 100 active queries, especially for weakly-supervised parsers learnt from a cold start.

Active Learning Semantic Parsing

How Can We Know What Language Models Know?

1 code implementation TACL 2020 Zhengbao Jiang, Frank F. Xu, Jun Araki, Graham Neubig

Recent work has presented intriguing results examining the knowledge contained in language models (LM) by having the LM fill in the blanks of prompts such as "Obama is a _ by profession".

LAMA

Optimizing Data Usage via Differentiable Rewards

1 code implementation ICML 2020 Xinyi Wang, Hieu Pham, Paul Michel, Antonios Anastasopoulos, Jaime Carbonell, Graham Neubig

To acquire a new skill, humans learn better and faster if a tutor, based on their current knowledge level, informs them of how much attention they should pay to particular content or practice problems.

Image Classification Machine Translation

A Bilingual Generative Transformer for Semantic Sentence Embedding

2 code implementations EMNLP 2020 John Wieting, Graham Neubig, Taylor Berg-Kirkpatrick

Semantic sentence embedding models encode natural language sentences into vectors, such that closeness in embedding space indicates closeness in the semantics between the sentences.

Semantic Similarity Semantic Textual Similarity +1

Generalizing Natural Language Analysis through Span-relation Representations

2 code implementations ACL 2020 Zhengbao Jiang, Wei Xu, Jun Araki, Graham Neubig

Natural language processing covers a wide variety of tasks predicting syntax, semantics, and information content, and usually each type of output is generated with specially designed architectures.

Aspect-Based Sentiment Analysis Constituency Parsing +6

Understanding Knowledge Distillation in Non-autoregressive Machine Translation

no code implementations ICLR 2020 Chunting Zhou, Graham Neubig, Jiatao Gu

We find that knowledge distillation can reduce the complexity of data sets and help NAT to model the variations in the output data.

Knowledge Distillation Machine Translation +1

Comparing Top-Down and Bottom-Up Neural Generative Dependency Models

no code implementations CONLL 2019 Austin Matthews, Graham Neubig, Chris Dyer

Recurrent neural network grammars generate sentences using phrase-structure syntax and perform very well on both parsing and language modeling.

Language Modelling

Findings of the Third Workshop on Neural Generation and Translation

no code implementations WS 2019 Hiroaki Hayashi, Yusuke Oda, Alexandra Birch, Ioannis Konstas, Andrew Finch, Minh-Thang Luong, Graham Neubig, Katsuhito Sudoh

This document describes the findings of the Third Workshop on Neural Generation and Translation, held in concert with the annual conference of the Empirical Methods in Natural Language Processing (EMNLP 2019).

Machine Translation Translation

Simple and Effective Paraphrastic Similarity from Parallel Translations

4 code implementations ACL 2019 John Wieting, Kevin Gimpel, Graham Neubig, Taylor Berg-Kirkpatrick

We present a model and methodology for learning paraphrastic sentence embeddings directly from bitext, removing the time-consuming intermediate step of creating paraphrase corpora.

Sentence Embeddings

Towards Zero-resource Cross-lingual Entity Linking

1 code implementation WS 2019 Shuyan Zhou, Shruti Rijhwani, Graham Neubig

Cross-lingual entity linking (XEL) grounds named entities in a source language to an English Knowledge Base (KB), such as Wikipedia.

Cross-Lingual Entity Linking Entity Linking

Regularizing Trajectories to Mitigate Catastrophic Forgetting

no code implementations25 Sep 2019 Paul Michel, Elisabeth Salesky, Graham Neubig

Regularization-based continual learning approaches generally prevent catastrophic forgetting by augmenting the training loss with an auxiliary objective.

Continual Learning

Learning to Deceive with Attention-Based Explanations

3 code implementations ACL 2020 Danish Pruthi, Mansi Gupta, Bhuwan Dhingra, Graham Neubig, Zachary C. Lipton

Attention mechanisms are ubiquitous components in neural architectures applied to natural language processing.

Fairness

Beyond BLEU: Training Neural Machine Translation with Semantic Similarity

1 code implementation14 Sep 2019 John Wieting, Taylor Berg-Kirkpatrick, Kevin Gimpel, Graham Neubig

While most neural machine translation (NMT) systems are still trained using maximum likelihood estimation, recent work has demonstrated that optimizing systems to directly improve evaluation metrics such as BLEU can substantially improve final translation accuracy.

Machine Translation Semantic Similarity +2

What Makes A Good Story? Designing Composite Rewards for Visual Storytelling

1 code implementation11 Sep 2019 Junjie Hu, Yu Cheng, Zhe Gan, Jingjing Liu, Jianfeng Gao, Graham Neubig

Previous storytelling approaches mostly focused on optimizing traditional metrics such as BLEU, ROUGE and CIDEr.

Visual Storytelling

Contextualized Representations for Low-resource Utterance Tagging

no code implementations WS 2019 Bhargavi Paranjape, Graham Neubig

Utterance-level analysis of the speaker{'}s intentions and emotions is a core task in conversational understanding.

Emotion Classification

Handling Syntactic Divergence in Low-resource Machine Translation

1 code implementation IJCNLP 2019 Chunting Zhou, Xuezhe Ma, Junjie Hu, Graham Neubig

Despite impressive empirical successes of neural machine translation (NMT) on standard benchmarks, limited parallel data impedes the application of NMT models to many language pairs.

Data Augmentation Machine Translation +1

A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers

1 code implementation IJCNLP 2019 Aditi Chaudhary, Jiateng Xie, Zaid Sheikh, Graham Neubig, Jaime G. Carbonell

Most state-of-the-art models for named entity recognition (NER) rely on the availability of large amounts of labeled data, making them challenging to extend to new, lower-resourced languages.

Active Learning Cross-Lingual Transfer +3

Latent Relation Language Models

no code implementations21 Aug 2019 Hiroaki Hayashi, Zecong Hu, Chenyan Xiong, Graham Neubig

In this paper, we propose Latent Relation Language Models (LRLMs), a class of language models that parameterizes the joint distribution over the words in a document and the entities that occur therein via knowledge graph relations.

Language Modelling

Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces

1 code implementation ACL 2019 Barun Patra, Joel Ruben Antony Moniz, Sarthak Garg, Matthew R. Gormley, Graham Neubig

We then propose Bilingual Lexicon Induction with Semi-Supervision (BLISS) --- a semi-supervised approach that relaxes the isometric assumption while leveraging both limited aligned bilingual lexicons and a larger set of unaligned word embeddings, as well as a novel hubness filtering technique.

Bilingual Lexicon Induction Word Embeddings

Mitigating Noisy Inputs for Question Answering

no code implementations8 Aug 2019 Denis Peskov, Joe Barrow, Pedro Rodriguez, Graham Neubig, Jordan Boyd-Graber

We investigate and mitigate the effects of noise from Automatic Speech Recognition systems on two factoid Question Answering (QA) tasks.

Machine Translation Optical Character Recognition +3

Improving Robustness of Neural Machine Translation with Multi-task Learning

1 code implementation WS 2019 Shuyan Zhou, Xiangkai Zeng, Yingqi Zhou, Antonios Anastasopoulos, Graham Neubig

While neural machine translation (NMT) achieves remarkable performance on clean, in-domain text, performance is known to degrade drastically when facing text which is full of typos, grammatical errors and other varieties of noise.

Machine Translation Multi-Task Learning +1

Reranking for Neural Semantic Parsing

no code implementations ACL 2019 Pengcheng Yin, Graham Neubig

Semantic parsing considers the task of transducing natural language (NL) utterances into machine executable meaning representations (MRs).

Code Generation Semantic Parsing

Beyond BLEU:Training Neural Machine Translation with Semantic Similarity

no code implementations ACL 2019 John Wieting, Taylor Berg-Kirkpatrick, Kevin Gimpel, Graham Neubig

While most neural machine translation (NMT)systems are still trained using maximum likelihood estimation, recent work has demonstrated that optimizing systems to directly improve evaluation metrics such as BLEU can significantly improve final translation accuracy.

Machine Translation Semantic Similarity +2

Generalized Data Augmentation for Low-Resource Translation

no code implementations ACL 2019 Mengzhou Xia, Xiang Kong, Antonios Anastasopoulos, Graham Neubig

Translation to or from low-resource languages LRLs poses challenges for machine translation in terms of both adequacy and fluency.

Data Augmentation Translation +1

Self-Attentional Models for Lattice Inputs

no code implementations ACL 2019 Matthias Sperber, Graham Neubig, Ngoc-Quan Pham, Alex Waibel

Lattices are an efficient and effective method to encode ambiguity of upstream systems in natural language processing tasks, for example to compactly capture multiple speech recognition hypotheses, or to represent multiple linguistic analyses.

Speech Recognition Translation

Learning to Describe Unknown Phrases with Local and Global Contexts

no code implementations NAACL 2019 Shonosuke Ishiwatari, Hiroaki Hayashi, Naoki Yoshinaga, Graham Neubig, Shoetsu Sato, Masashi Toyoda, Masaru Kitsuregawa

When reading a text, it is common to become stuck on unfamiliar words and phrases, such as polysemous words with novel senses, rarely used idioms, internet slang, or emerging entities.

Improving Open Information Extraction via Iterative Rank-Aware Learning

1 code implementation ACL 2019 Zhengbao Jiang, Pengcheng Yin, Graham Neubig

We found that the extraction likelihood, a confidence measure used by current supervised open IE systems, is not well calibrated when comparing the quality of assertions extracted from different sentences.

General Classification Open Information Extraction

Choosing Transfer Languages for Cross-Lingual Learning

1 code implementation ACL 2019 Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, Graham Neubig

Cross-lingual transfer, where a high-resource transfer language is used to improve the accuracy of a low-resource task language, is now an invaluable tool for improving performance of natural language processing (NLP) on low-resource languages.

Cross-Lingual Transfer

Are Sixteen Heads Really Better than One?

3 code implementations NeurIPS 2019 Paul Michel, Omer Levy, Graham Neubig

Attention is a powerful and ubiquitous mechanism for allowing neural models to focus on particular salient pieces of information by taking their weighted average when making predictions.

Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation

no code implementations ACL 2019 Xinyi Wang, Graham Neubig

To improve low-resource Neural Machine Translation (NMT) with multilingual corpora, training on the most related high-resource language only is often more effective than using all data available (Neubig and Hu, 2018).

Low-Resource Neural Machine Translation Translation

On Meaning-Preserving Adversarial Perturbations for Sequence-to-Sequence Models

no code implementations ICLR 2019 Paul Michel, Graham Neubig, Xi-An Li, Juan Miguel Pino

Adversarial examples have been shown to be an effective way of assessing the robustness of neural sequence-to-sequence (seq2seq) models, by applying perturbations to the input of a model leading to large degradation in performance.

Adversarial Robustness Machine Translation +1

Attention-Passing Models for Robust and Data-Efficient End-to-End Speech Translation

no code implementations TACL 2019 Matthias Sperber, Graham Neubig, Jan Niehues, Alex Waibel

Speech translation has traditionally been approached through cascaded models consisting of a speech recognizer trained on a corpus of transcribed speech, and a machine translation system trained on parallel texts.

Machine Translation Speech Recognition +1

Density Matching for Bilingual Word Embedding

1 code implementation NAACL 2019 Chunting Zhou, Xuezhe Ma, Di Wang, Graham Neubig

Recent approaches to cross-lingual word embedding have generally been based on linear transformations between the sets of embedding vectors in the two languages.

Bilingual Lexicon Induction Word Embeddings +1

Lost in Interpretation: Predicting Untranslated Terminology in Simultaneous Interpretation

1 code implementation NAACL 2019 Nikolai Vogler, Craig Stewart, Graham Neubig

Simultaneous interpretation, the translation of speech from one language to another in real-time, is an inherently difficult and strenuous task.

Translation

Competence-based Curriculum Learning for Neural Machine Translation

2 code implementations NAACL 2019 Emmanouil Antonios Platanios, Otilia Stretcu, Graham Neubig, Barnabas Poczos, Tom M. Mitchell

In this paper, we propose a curriculum learning framework for NMT that reduces training time, reduces the need for specialized heuristics or large batch sizes, and results in overall better performance.

Curriculum Learning Machine Translation +1

compare-mt: A Tool for Holistic Comparison of Language Generation Systems

2 code implementations NAACL 2019 Graham Neubig, Zi-Yi Dou, Junjie Hu, Paul Michel, Danish Pruthi, Xinyi Wang, John Wieting

In this paper, we describe compare-mt, a tool for holistic analysis and comparison of the results of systems for language generation tasks such as machine translation.

Machine Translation Text Generation +1

On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models

1 code implementation NAACL 2019 Paul Michel, Xi-An Li, Graham Neubig, Juan Miguel Pino

Adversarial examples --- perturbations to the input of a model that elicit large changes in the output --- have been shown to be an effective way of assessing the robustness of sequence-to-sequence (seq2seq) models.

Adversarial Robustness Machine Translation

The ARIEL-CMU Systems for LoReHLT18

no code implementations24 Feb 2019 Aditi Chaudhary, Siddharth Dalmia, Junjie Hu, Xinjian Li, Austin Matthews, Aldrian Obaja Muis, Naoki Otani, Shruti Rijhwani, Zaid Sheikh, Nidhi Vyas, Xinyi Wang, Jiateng Xie, Ruochen Xu, Chunting Zhou, Peter J. Jansen, Yiming Yang, Lori Levin, Florian Metze, Teruko Mitamura, David R. Mortensen, Graham Neubig, Eduard Hovy, Alan W. black, Jaime Carbonell, Graham V. Horwood, Shabnam Tafreshi, Mona Diab, Efsun S. Kayi, Noura Farra, Kathleen McKeown

This paper describes the ARIEL-CMU submissions to the Low Resource Human Language Technologies (LoReHLT) 2018 evaluations for the tasks Machine Translation (MT), Entity Discovery and Linking (EDL), and detection of Situation Frames in Text and Speech (SF Text and Speech).

Machine Translation Translation

Multilingual Neural Machine Translation With Soft Decoupled Encoding

1 code implementation ICLR 2019 Xinyi Wang, Hieu Pham, Philip Arthur, Graham Neubig

Multilingual training of neural machine translation (NMT) systems has led to impressive accuracy improvements on low-resource languages.

Machine Translation Translation

An Adversarial Approach to High-Quality, Sentiment-Controlled Neural Dialogue Generation

no code implementations22 Jan 2019 Xiang Kong, Bohan Li, Graham Neubig, Eduard Hovy, Yiming Yang

In this work, we propose a method for neural dialogue response generation that allows not only generating semantically reasonable responses according to the dialogue history, but also explicitly controlling the sentiment of the response via sentiment labels.

Dialogue Generation

Lagging Inference Networks and Posterior Collapse in Variational Autoencoders

2 code implementations ICLR 2019 Junxian He, Daniel Spokoyny, Graham Neubig, Taylor Berg-Kirkpatrick

The variational autoencoder (VAE) is a popular combination of deep latent variable model and accompanying variational learning technique.

Text Generation

Towards a General-Purpose Linguistic Annotation Backend

no code implementations13 Dec 2018 Graham Neubig, Patrick Littell, Chian-Yu Chen, Jean Lee, Zirui Li, Yu-Hsiang Lin, Yuyan Zhang

In this extended abstract, we describe the beginnings of a new project that will attempt to ease this language documentation process through the use of natural language processing (NLP) technology.

Zero-shot Neural Transfer for Cross-lingual Entity Linking

1 code implementation9 Nov 2018 Shruti Rijhwani, Jiateng Xie, Graham Neubig, Jaime Carbonell

To address this problem, we investigate zero-shot cross-lingual entity linking, in which we assume no bilingual lexical resources are available in the source low-resource language.

Cross-Lingual Entity Linking Entity Linking

Learning to Describe Phrases with Local and Global Contexts

1 code implementation1 Nov 2018 Shonosuke Ishiwatari, Hiroaki Hayashi, Naoki Yoshinaga, Graham Neubig, Shoetsu Sato, Masashi Toyoda, Masaru Kitsuregawa

When reading a text, it is common to become stuck on unfamiliar words and phrases, such as polysemous words with novel senses, rarely used idioms, internet slang, or emerging entities.

Reading Comprehension

Optimizing Segmentation Granularity for Neural Machine Translation

no code implementations19 Oct 2018 Elizabeth Salesky, Andrew Runge, Alex Coda, Jan Niehues, Graham Neubig

However, the granularity of these subword units is a hyperparameter to be tuned for each language and task, using methods such as grid search.

Machine Translation Translation

TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation

4 code implementations EMNLP 2018 Pengcheng Yin, Graham Neubig

We present TRANX, a transition-based neural semantic parser that maps natural language (NL) utterances into formal meaning representations (MRs).