Search Results for author:
Found 356 papers, 216 papers with code
Segmentation for Efficient Supervised Language Annotation with an Explicit Cost-Utility Tradeoff
In this paper, we study the problem of manually correcting automatic annotations of natural language in as efficient a manner as possible.
Language Resource Addition: Dictionary or Corpus?
The experimental results showed that the annotated sentence addition to the training corpus is better than the entries addition to the dictionary.
Collection of a Simultaneous Translation Corpus for Comparative Analysis
This makes it possible to compare translation data with simultaneous interpretation data.
Towards Multilingual Conversations in the Medical Domain: Development of Multilingual Medical Data and A Network-based ASR System
This paper outlines the recent development on multilingual medical data and multilingual speech recognition system for network-based speech-to-speech translation in the medical domain.
Semantic Parsing of Ambiguous Input through Paraphrasing and Verification
We propose a new method for semantic parsing of ambiguous and ungrammatical input, such as search queries.
Neural Reranking Improves Subjective Quality of Machine Translation: NAIST at WAT2015
We further perform a detailed analysis of reasons for this increase, finding that the main contributions of the neural models lie in improvement of the grammatical correctness of the output, as opposed to improvements in lexical choice of content words.
Morphological Inflection Generation Using Character Sequence to Sequence Learning
Morphological inflection generation is the task of generating the inflected form of a given lemma corresponding to a particular linguistic transformation.
Optimizing Computer-Assisted Transcription Quality with Iterative User Interfaces
Our goal is to improve the human transcription quality via appropriate user interface design.
Generalizing and Hybridizing Count-based and Neural Language Models
Language models (LMs) are statistical models that calculate probabilities over sequences of words or other discrete symbols.
Incorporating Discrete Translation Lexicons into Neural Machine Translation
Neural machine translation (NMT) often makes mistakes in translating low-frequency content words that are essential to understanding the meaning of the sentence.
Controlling Output Length in Neural Encoder-Decoders
Neural encoder-decoder models have shown great success in many sequence generation tasks.
Learning to Translate in Real-time with Neural Machine Translation
Translating in real-time, a. k. a.
Lexicons and Minimum Risk Training for Neural Machine Translation: NAIST-CMU at WAT2016
This year, the Nara Institute of Science and Technology (NAIST)/Carnegie Mellon University (CMU) submission to the Japanese-English translation track of the 2016 Workshop on Asian Translation was based on attentional neural machine translation (NMT) models.
Eve: A Gradient Based Optimization Method with Locally and Globally Adaptive Learning Rates
Adaptive gradient methods for stochastic optimization adjust the learning rate for each parameter locally.
What Do Recurrent Neural Network Grammars Learn About Syntax?
We investigate what information they learn, from a linguistic perspective, through various ablations to the model and the data, and by augmenting the model with an attention mechanism (GA-RNNG) to enable closer inspection.
Ranked #20 on
Constituency Parsing
on Penn Treebank
Overview of the 3rd Workshop on Asian Translation
For the WAT2016, 15 institutions participated in the shared tasks.
Lightly Supervised Quality Estimation
Evaluating the quality of output from language processing systems such as machine translation or speech recognition is an essential step in ensuring that they are sufficient for practical use.
DyNet: The Dynamic Neural Network Toolkit
In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then examples are fed into an engine that executes this computation and computes its derivatives.
Neural Machine Translation and Sequence-to-sequence Models: A Tutorial
This tutorial introduces a new and powerful set of techniques variously called "neural machine translation" or "neural sequence-to-sequence models".
Cross-Lingual Word Embeddings for Low-Resource Language Modeling
We investigate the use of such lexicons to improve language models when textual training data is limited to as few as a thousand sentences.
Neural Lattice-to-Sequence Models for Uncertain Inputs
In this work, we extend the TreeLSTM (Tai et al., 2015) into a LatticeLSTM that is able to consume word lattices, and can be used as encoder in an attentional encoder-decoder model.
A Syntactic Neural Model for General-Purpose Code Generation
We consider the problem of parsing natural language descriptions into source code written in a general-purpose programming language like Python.
Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction
Labeled sequence transduction is a task of transforming one sequence into another sequence that satisfies desiderata specified by a set of labels.
Learning Character-level Compositionality with Visual Features
Previous work has modeled the compositionality of words by creating character-level models of meaning, reducing problems of sparsity for rare words.
Neural Machine Translation via Binary Code Prediction
In this paper, we propose a new method for calculating the output layer in neural machine translation systems.
Softmax Q-Distribution Estimation for Structured Prediction: A Theoretical Interpretation for RAML
Reward augmented maximum likelihood (RAML), a simple and effective learning framework to directly optimize towards the reward function in structured prediction tasks, has led to a number of impressive empirical successes.
On-the-fly Operation Batching in Dynamic Computation Graphs
Dynamic neural network toolkits such as PyTorch, DyNet, and Chainer offer more flexibility for implementing models that cope with data of varying dimensions and structure, relative to toolkits that operate on statically declared computations (e. g., TensorFlow, CNTK, and Theano).
Controllable Invariance through Adversarial Feature Learning
Learning meaningful representations that maintain the content necessary for a particular task while filtering away detrimental variations is a problem of great interest in machine learning.
An Empirical Study of Mini-Batch Creation Strategies for Neural Machine Translation
Training of neural machine translation (NMT) models usually uses mini-batches for efficiency purposes.
Stronger Baselines for Trustable Results in Neural Machine Translation
As a result, it is often difficult to determine whether improvements from research will carry over to systems deployed for real-world use.
CharManteau: Character Embedding Models For Portmanteau Creation
Portmanteaus are a word formation phenomenon where two words are combined to form a new word.
Learning Language Representations for Typology Prediction
One central mystery of neural NLP is what neural models "know" about their subject matter.
How Would You Say It? Eliciting Lexically Diverse Dialogue for Supervised Semantic Parsing
Wang et al. (2015) proposed a method to build semantic parsing datasets by generating canonical utterances using a grammar and having crowdworkers paraphrase them into natural wording.
A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models
In experiments, we show that optimizing this new training objective yields substantially better results on two sequence tasks (Named Entity Recognition and CCG Supertagging) when compared with both cross entropy trained greedy decoding and cross entropy trained beam decoding baselines.
Ranked #3 on
Motion Segmentation
on Hopkins155
Handling Homographs in Neural Machine Translation
Homographs, words with different meanings but the same surface form, have long caused difficulty for machine translation systems, as it is difficult to select the correct translation based on the context.
Transcribing Against Time
We investigate the problem of manually correcting errors from an automatic speech transcript in a cost-sensitive fashion.
Improving Neural Machine Translation through Phrase-based Forced Decoding
Compared to traditional statistical machine translation (SMT), neural machine translation (NMT) often sacrifices adequacy for the sake of fluency.
Overview of the 4th Workshop on Asian Translation
For the WAT2017, 12 institutions participated in the shared tasks.
Convolutional Neural Networks for Medical Diagnosis from Admission Notes
The hope is that the tool can be used to reduce mis-diagnosis.
Cavs: A Vertex-centric Programming Interface for Dynamic Neural Networks
Recent deep learning (DL) models have moved beyond static network architectures to dynamic ones, handling data where the network structure changes every example, such as sequences of variable lengths, trees, and graphs.
Linguistic unit discovery from multi-modal inputs in unwritten languages: Summary of the "Speaking Rosetta" JSALT 2017 Workshop
We summarize the accomplishments of a multi-disciplinary workshop exploring the computational and scientific issues surrounding the discovery of linguistic units (subwords and words) in a language without orthography.
XNMT: The eXtensible Neural Machine Translation Toolkit
In this paper we describe the design of XNMT and its experiment configuration system, and demonstrate its utility on the tasks of machine translation, speech recognition, and multi-tasked machine translation/parsing.
Neural Lattice Language Models
In this work, we propose a new language modeling paradigm that has the ability to perform both prediction and moderation of information flow at multiple granularities: neural lattice language models.
Self-Attentional Acoustic Models
Self-attention is a method of encoding sequences of vectors by relating these vectors to each-other based on pairwise similarities.
Attentive Interaction Model: Modeling Changes in View in Argumentation
We present a neural architecture for modeling argumentative dialogue that explicitly models the interplay between an Opinion Holder's (OH's) reasoning and a challenger's argument, with the goal of predicting if the argument successfully changes the OH's view.
Guiding Neural Machine Translation with Retrieved Translation Pieces
Specifically, for an input sentence, we use a search engine to retrieve sentence pairs whose source sides are similar with the input sentence, and then collect $n$-grams that are both in the retrieved target sentences and aligned with words that match in the source sentences, which we call "translation pieces".
When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?
The performance of Neural Machine Translation (NMT) systems often suffers in low-resource scenarios where sufficiently large-scale parallel corpora cannot be obtained.
Stack-Pointer Networks for Dependency Parsing
Combining pointer networks~\citep{vinyals2015pointer} with an internal stack, the proposed model first reads and encodes the whole sentence, then builds the dependency tree top-down (from root-to-leaf) in a depth-first fashion.
Ranked #14 on
Dependency Parsing
on Penn Treebank
Extreme Adaptation for Personalized Neural Machine Translation
Every person speaks or writes their own flavor of their native language, influenced by a number of factors: the content they tend to talk about, their gender, their social status, or their geographical origin.
Automatic Estimation of Simultaneous Interpreter Performance
Simultaneous interpretation, translation of the spoken word in real-time, is both highly challenging and physically demanding.
Neural Factor Graph Models for Cross-lingual Morphological Tagging
Morphological analysis involves predicting the syntactic traits of a word (e. g. {POS: Noun, Case: Acc, Gender: Fem}).
Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow
For tasks like code synthesis from natural language, code retrieval, and code summarization, data-driven models have shown great promise.
Modelling Natural Language, Programs, and their Intersection
As a result, in the past several years there has been an increasing research interest in methods that focus on the intersection of programming and natural language, allowing users to use natural language to interact with computers in the complex ways that programs allow us to do.
Using Morphological Knowledge in Open-Vocabulary Neural Language Models
Languages with productive morphology pose problems for language models that generate words from a fixed vocabulary.
Stress Test Evaluation for Natural Language Inference
Natural language inference (NLI) is the task of determining if a natural language hypothesis can be inferred from a given premise in a justifiable manner.
Natural Language Inference
Natural Language Understanding
+1
Multi-Source Neural Machine Translation with Missing Data
This study focuses on the use of incomplete multilingual corpora in multi-encoder NMT and mixture of NMT experts and examines a very simple implementation where missing source translations are replaced by a special symbol <NULL>.
Findings of the Second Workshop on Neural Machine Translation and Generation
This document describes the findings of the Second Workshop on Neural Machine Translation and Generation, held in concert with the annual conference of the Association for Computational Linguistics (ACL 2018).
StructVAE: Tree-structured Latent Variable Models for Semi-supervised Semantic Parsing
Semantic parsing is the task of transducing natural language (NL) utterances into formal meaning representations (MRs), commonly represented as tree structures.
Learning to Generate Move-by-Move Commentary for Chess Games from Large-Scale Social Forum Data
This paper examines the problem of generating natural language descriptions of chess games.
Rapid Adaptation of Neural Machine Translation to New Languages
This paper examines the problem of adapting neural machine translation systems to new, low-resourced languages (LRLs) as effectively and rapidly as possible.
SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation
In this work, we examine methods for data augmentation for text-based tasks such as neural machine translation (NMT).
Contextual Parameter Generation for Universal Neural Machine Translation
We propose a simple modification to existing neural machine translation (NMT) models that enables using a single universal model to translate between multiple languages while allowing for language specific parameterization, and that can also be used for domain adaptation.
Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations
Much work in Natural Language Processing (NLP) has been for resource-rich languages, making generalization to new, less-resourced languages challenging.
Unsupervised Learning of Syntactic Structure with Invertible Neural Projections
In this work, we propose a novel generative model that jointly learns discrete syntactic structure and continuous word representations in an unsupervised fashion by cascading an invertible neural network with a structured generative prior.
A Tree-based Decoder for Neural Machine Translation
Recent advances in Neural Machine Translation (NMT) show that adding syntactic information to NMT systems can improve the quality of their translations.
Retrieval-Based Neural Code Generation
In models to generate program source code from natural language, representing this code in a tree structure has been a common approach.
Neural Cross-Lingual Named Entity Recognition with Minimal Resources
To improve robustness to word order differences, we propose to use self-attention, which allows for a degree of flexibility with respect to word order.
Parameter Sharing Methods for Multilingual Self-Attentional Translation Models
In multilingual neural machine translation, it has been shown that sharing a single translation model between multiple languages can achieve competitive performance, sometimes even leading to performance gains over bilingually trained models.
Contextual Encoding for Translation Quality Estimation
In this paper, propose a method to effectively encode the local and global contextual information for each target word using a three-part neural network approach.
MTNT: A Testbed for Machine Translation of Noisy Text
In this paper, we propose a benchmark dataset for Machine Translation of Noisy Text (MTNT), consisting of noisy comments on Reddit (www. reddit. com) and professionally sourced translations.
BLISS in Non-Isometric Embedding Spaces
We then propose Bilingual Lexicon Induction with Semi-Supervision (BLISS) --- a novel semi-supervised approach that relaxes the isometric assumption while leveraging both limited aligned bilingual lexicons and a larger set of unaligned word embeddings, as well as a novel hubness filtering technique.
Measuring Density and Similarity of Task Relevant Information in Neural Representations
Neural models achieve state-of-the-art performance due to their ability to extract salient features useful to downstream tasks.
TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation
We present TRANX, a transition-based neural semantic parser that maps natural language (NL) utterances into formal meaning representations (MRs).
Ranked #2 on
Semantic Parsing
on ATIS
Multi-Source Neural Machine Translation with Data Augmentation
By using information from these multiple sources, these systems achieve large gains in accuracy.
Optimizing Segmentation Granularity for Neural Machine Translation
However, the granularity of these subword units is a hyperparameter to be tuned for each language and task, using methods such as grid search.
Learning to Represent Edits
We introduce the problem of learning distributed representations of edits.
Learning to Describe Phrases with Local and Global Contexts
When reading a text, it is common to become stuck on unfamiliar words and phrases, such as polysemous words with novel senses, rarely used idioms, internet slang, or emerging entities.
Zero-shot Neural Transfer for Cross-lingual Entity Linking
To address this problem, we investigate zero-shot cross-lingual entity linking, in which we assume no bilingual lexical resources are available in the source low-resource language.
Towards a General-Purpose Linguistic Annotation Backend
In this extended abstract, we describe the beginnings of a new project that will attempt to ease this language documentation process through the use of natural language processing (NLP) technology.
Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
The variational autoencoder (VAE) is a popular combination of deep latent variable model and accompanying variational learning technique.
Ranked #1 on
Text Generation
on Yahoo Questions
An Adversarial Approach to High-Quality, Sentiment-Controlled Neural Dialogue Generation
In this work, we propose a method for neural dialogue response generation that allows not only generating semantically reasonable responses according to the dialogue history, but also explicitly controlling the sentiment of the response via sentiment labels.
Multilingual Neural Machine Translation With Soft Decoupled Encoding
Multilingual training of neural machine translation (NMT) systems has led to impressive accuracy improvements on low-resource languages.
The ARIEL-CMU Systems for LoReHLT18
This paper describes the ARIEL-CMU submissions to the Low Resource Human Language Technologies (LoReHLT) 2018 evaluations for the tasks Machine Translation (MT), Entity Discovery and Linking (EDL), and detection of Situation Frames in Text and Speech (SF Text and Speech).
Improving Robustness of Machine Translation with Synthetic Noise
Modern Machine Translation (MT) systems perform consistently well on clean, in-domain text.
On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models
Adversarial examples --- perturbations to the input of a model that elicit large changes in the output --- have been shown to be an effective way of assessing the robustness of sequence-to-sequence (seq2seq) models.
compare-mt: A Tool for Holistic Comparison of Language Generation Systems
In this paper, we describe compare-mt, a tool for holistic analysis and comparison of the results of systems for language generation tasks such as machine translation.
Competence-based Curriculum Learning for Neural Machine Translation
In this paper, we propose a curriculum learning framework for NMT that reduces training time, reduces the need for specialized heuristics or large batch sizes, and results in overall better performance.
Lost in Interpretation: Predicting Untranslated Terminology in Simultaneous Interpretation
Simultaneous interpretation, the translation of speech from one language to another in real-time, is an inherently difficult and strenuous task.
Density Matching for Bilingual Word Embedding
Recent approaches to cross-lingual word embedding have generally been based on linear transformations between the sets of embedding vectors in the two languages.
Attention-Passing Models for Robust and Data-Efficient End-to-End Speech Translation
Speech translation has traditionally been approached through cascaded models consisting of a speech recognizer trained on a corpus of transcribed speech, and a machine translation system trained on parallel texts.
On Meaning-Preserving Adversarial Perturbations for Sequence-to-Sequence Models
Adversarial examples have been shown to be an effective way of assessing the robustness of neural sequence-to-sequence (seq2seq) models, by applying perturbations to the input of a model leading to large degradation in performance.
Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation
To improve low-resource Neural Machine Translation (NMT) with multilingual corpora, training on the most related high-resource language only is often more effective than using all data available (Neubig and Hu, 2018).
Are Sixteen Heads Really Better than One?
Attention is a powerful and ubiquitous mechanism for allowing neural models to focus on particular salient pieces of information by taking their weighted average when making predictions.
Choosing Transfer Languages for Cross-Lingual Learning
Cross-lingual transfer, where a high-resource transfer language is used to improve the accuracy of a low-resource task language, is now an invaluable tool for improving performance of natural language processing (NLP) on low-resource languages.
Improving Open Information Extraction via Iterative Rank-Aware Learning
We found that the extraction likelihood, a confidence measure used by current supervised open IE systems, is not well calibrated when comparing the quality of assertions extracted from different sentences.
Learning to Describe Unknown Phrases with Local and Global Contexts
When reading a text, it is common to become stuck on unfamiliar words and phrases, such as polysemous words with novel senses, rarely used idioms, internet slang, or emerging entities.
Domain Adaptation of Neural Machine Translation by Lexicon Induction
It has been previously noted that neural machine translation (NMT) is very sensitive to domain shift.
Self-Attentional Models for Lattice Inputs
Lattices are an efficient and effective method to encode ambiguity of upstream systems in natural language processing tasks, for example to compactly capture multiple speech recognition hypotheses, or to represent multiple linguistic analyses.
Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections
The parameters of source model and target model are softly shared through a regularized log likelihood objective.
Generalized Data Augmentation for Low-Resource Translation
Translation to or from low-resource languages LRLs poses challenges for machine translation in terms of both adequacy and fluency.
Findings of the First Shared Task on Machine Translation Robustness
We share the findings of the first shared task on improving robustness of Machine Translation (MT).
Reranking for Neural Semantic Parsing
Semantic parsing considers the task of transducing natural language (NL) utterances into machine executable meaning representations (MRs).
Ranked #4 on
Code Generation
on Django
Beyond BLEU:Training Neural Machine Translation with Semantic Similarity
While most neural machine translation (NMT)systems are still trained using maximum likelihood estimation, recent work has demonstrated that optimizing systems to directly improve evaluation metrics such as BLEU can significantly improve final translation accuracy.
Improving Robustness of Neural Machine Translation with Multi-task Learning
While neural machine translation (NMT) achieves remarkable performance on clean, in-domain text, performance is known to degrade drastically when facing text which is full of typos, grammatical errors and other varieties of noise.
Mitigating Noisy Inputs for Question Answering
We investigate and mitigate the effects of noise from Automatic Speech Recognition systems on two factoid Question Answering (QA) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Pushing the Limits of Low-Resource Morphological Inflection
Recent years have seen exceptional strides in the task of automatic morphological inflection generation.
Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces
We then propose Bilingual Lexicon Induction with Semi-Supervision (BLISS) --- a semi-supervised approach that relaxes the isometric assumption while leveraging both limited aligned bilingual lexicons and a larger set of unaligned word embeddings, as well as a novel hubness filtering technique.
Latent Relation Language Models
In this paper, we propose Latent Relation Language Models (LRLMs), a class of language models that parameterizes the joint distribution over the words in a document and the entities that occur therein via knowledge graph relations.
A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers
Most state-of-the-art models for named entity recognition (NER) rely on the availability of large amounts of labeled data, making them challenging to extend to new, lower-resourced languages.
Unsupervised Domain Adaptation for Neural Machine Translation with Domain-Aware Feature Embeddings
The recent success of neural machine translation models relies on the availability of high quality, in-domain data.
Handling Syntactic Divergence in Low-resource Machine Translation
Despite impressive empirical successes of neural machine translation (NMT) on standard benchmarks, limited parallel data impedes the application of NMT models to many language pairs.
Contextualized Representations for Low-resource Utterance Tagging
Utterance-level analysis of the speaker{'}s intentions and emotions is a core task in conversational understanding.
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text
In this paper, we investigate a simple fix for posterior collapse which yields surprisingly effective results.
FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flow
Most sequence-to-sequence (seq2seq) models are autoregressive; they generate each token by conditioning on previously generated tokens.
Ranked #3 on
Machine Translation
on WMT2016 English-Romanian
What Makes A Good Story? Designing Composite Rewards for Visual Storytelling
Previous storytelling approaches mostly focused on optimizing traditional metrics such as BLEU, ROUGE and CIDEr.
Ranked #10 on
Visual Storytelling
on VIST
Beyond BLEU: Training Neural Machine Translation with Semantic Similarity
While most neural machine translation (NMT) systems are still trained using maximum likelihood estimation, recent work has demonstrated that optimizing systems to directly improve evaluation metrics such as BLEU can substantially improve final translation accuracy.
Learning to Deceive with Attention-Based Explanations
Attention mechanisms are ubiquitous components in neural architectures applied to natural language processing.
Regularizing Trajectories to Mitigate Catastrophic Forgetting
Regularization-based continual learning approaches generally prevent catastrophic forgetting by augmenting the training loss with an auxiliary objective.
Towards Zero-resource Cross-lingual Entity Linking
Cross-lingual entity linking (XEL) grounds named entities in a source language to an English Knowledge Base (KB), such as Wikipedia.
Simple and Effective Paraphrastic Similarity from Parallel Translations
We present a model and methodology for learning paraphrastic sentence embeddings directly from bitext, removing the time-consuming intermediate step of creating paraphrase corpora.
Domain Differential Adaptation for Neural Machine Translation
We then use these learned domain differentials to adapt models for the target task accordingly.
Cross-lingual Alignment vs Joint Training: A Comparative Study and A Simple Unified Framework
Learning multilingual representations of text has proven a successful method for many cross-lingual transfer learning tasks.
Findings of the Third Workshop on Neural Generation and Translation
This document describes the findings of the Third Workshop on Neural Generation and Translation, held in concert with the annual conference of the Empirical Methods in Natural Language Processing (EMNLP 2019).
Comparing Top-Down and Bottom-Up Neural Generative Dependency Models
Recurrent neural network grammars generate sentences using phrase-structure syntax and perform very well on both parsing and language modeling.
Understanding Knowledge Distillation in Non-autoregressive Machine Translation
We find that knowledge distillation can reduce the complexity of data sets and help NAT to model the variations in the output data.
Should All Cross-Lingual Embeddings Speak English?
Most of recent work in cross-lingual word embeddings is severely Anglocentric.
A Bilingual Generative Transformer for Semantic Sentence Embedding
Semantic sentence embedding models encode natural language sentences into vectors, such that closeness in embedding space indicates closeness in the semantics between the sentences.
Generalizing Natural Language Analysis through Span-relation Representations
Natural language processing covers a wide variety of tasks predicting syntax, semantics, and information content, and usually each type of output is generated with specially designed architectures.
Ranked #1 on
Relation Extraction
on WLPC
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+8
Optimizing Data Usage via Differentiable Rewards
To acquire a new skill, humans learn better and faster if a tutor, based on their current knowledge level, informs them of how much attention they should pay to particular content or practice problems.
How Can We Know What Language Models Know?
Recent work has presented intriguing results examining the knowledge contained in language models (LM) by having the LM fill in the blanks of prompts such as "Obama is a _ by profession".
Merging Weak and Active Supervision for Semantic Parsing
Experiments on WikiTableQuestions with human annotators show that our method can improve the performance with only 100 active queries, especially for weakly-supervised parsers learnt from a cold start.
A Probabilistic Formulation of Unsupervised Text Style Transfer
Across all style transfer tasks, our approach yields substantial gains over state-of-the-art non-generative baselines, including the state-of-the-art unsupervised machine translation techniques that our approach generalizes.
Learning Relation Entailment with Structured and Textual Information
We propose several methods that incorporate both structured and textual information to represent relations for this task.
Differentiable Reasoning over a Virtual Knowledge Base
In particular, we describe a neural module, DrKIT, that traverses textual data like a KB, softly following paths of relations between mentions of entities in the corpus.
Universal Phone Recognition with a Multilingual Allophone System
Multilingual models can improve language processing, particularly for low resource situations, by sharing parameters across languages.
Improving Candidate Generation for Low-resource Cross-lingual Entity Linking
Cross-lingual entity linking (XEL) is the task of finding referents in a target-language knowledge base (KB) for mentions extracted from source-language texts.
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
However, these broad-coverage benchmarks have been mostly limited to English, and despite an increasing interest in multilingual models, a benchmark that enables the comprehensive evaluation of such methods on a diverse range of languages and tasks is still missing.
A Set of Recommendations for Assessing Human-Machine Parity in Language Translation
The quality of machine translation has increased remarkably over the past years, to the degree that it was found to be indistinguishable from professional human translation in a number of empirical investigations.
Dynamic Data Selection and Weighting for Iterative Back-Translation
Back-translation has proven to be an effective method to utilize monolingual data in neural machine translation (NMT), and iteratively conducting back-translation can further improve the model performance.
Weight Poisoning Attacks on Pre-trained Models
We show that by applying a regularization method, which we call RIPPLe, and an initialization procedure, which we call Embedding Surgery, such attacks are possible even with limited knowledge of the dataset and fine-tuning procedure.
Balancing Training for Multilingual Neural Machine Translation
When training multilingual machine translation (MT) models that can translate to/from multiple languages, we are faced with imbalanced training sets: some languages have much more training data than others.
AlloVera: A Multilingual Allophone Database
While phonemic representations are language specific, phonetic representations (stated in terms of (allo)phones) are much closer to a universal (language-independent) transcription.
Incorporating External Knowledge through Pre-training for Natural Language to Code Generation
Open-domain code generation aims to generate code in a general-purpose programming language (such as Python) from natural language (NL) intents.
Ranked #3 on
Code Generation
on CoNaLa-Ext
Practical Comparable Data Collection for Low-Resource Languages via Images
We propose a method of curating high-quality comparable training data for low-resource languages with monolingual annotators.
A Summary of the First Workshop on Language Technology for Language Documentation and Revitalization
Despite recent advances in natural language processing and other language technology, the application of such technology to language documentation and conservation has been limited.
Politeness Transfer: A Tag and Generate Approach
This paper introduces a new task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning.
A Benchmark for Structured Procedural Knowledge Extraction from Cooking Videos
Watching instructional videos are often used to learn about procedures.
Predicting Performance for Natural Language Processing Tasks
Given the complexity of combinations of tasks, languages, and domains in natural language processing (NLP) research, it is computationally prohibitive to exhaustively test newly proposed models on each possible experimental setting.
Soft Gazetteers for Low-Resource Named Entity Recognition
However, designing such features for low-resource languages is challenging, because exhaustive entity gazetteers do not exist in these languages.
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
Recent years have witnessed the burgeoning of pretrained language models (LMs) for text-based natural language (NL) understanding tasks.
Ranked #10 on
Text-To-SQL
on spider
(Exact Match Accuracy (Dev) metric)
Learning Sparse Prototypes for Text Generation
While effective, these methods are inefficient at test time as a result of needing to store and index the entire training corpus.
Findings of the Fourth Workshop on Neural Generation and Translation
We describe the finding of the Fourth Workshop on Neural Generation and Translation, held in concert with the annual conference of the Association for Computational Linguistics (ACL 2020).
Weight Poisoning Attacks on Pretrained Models
Recently, NLP has seen a surge in the usage of large pre-trained models.
Transliteration for Cross-Lingual Morphological Inflection
Cross-lingual transfer between typologically related languages has been proven successful for the task of morphological inflection.
TICO-19: the Translation Initiative for Covid-19
Further, the team is converting the test and development data into translation memories (TMXs) that can be used by localizers from and to any of the languages.
The Return of Lexical Dependencies: Neural Lexicalized PCFGs
In this paper we demonstrate that $\textit{context free grammar (CFG) based methods for grammar induction benefit from modeling lexical dependencies}$.
Automatic Extraction of Rules Governing Morphological Agreement
Using cross-lingual transfer, even with no expert annotations in the language of interest, our framework extracts a grammatical specification which is nearly equivalent to those created with large amounts of gold-standard annotated data.
Improving Target-side Lexical Transfer in Multilingual Neural Machine Translation
To improve the performance of Neural Machine Translation~(NMT) for low-resource languages~(LRL), one effective strategy is to leverage parallel data from a related high-resource language~(HRL).
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models
We further propose a code-switching-based method to improve the ability of multilingual LMs to access knowledge, and verify its effectiveness on several benchmark languages.
Re-evaluating Evaluation in Text Summarization
Automated evaluation metrics as a stand-in for manual evaluation are an essential part of the development of text-generation tasks such as text summarization.
GSum: A General Framework for Guided Neural Abstractive Summarization
Neural abstractive summarization models are flexible and can produce coherent summaries, but they are sometimes unfaithful and can be difficult to control.
Explicit Alignment Objectives for Multilingual Bidirectional Encoders
Pre-trained cross-lingual encoders such as mBERT (Devlin et al., 2019) and XLMR (Conneau et al., 2020) have proven to be impressively effective at enabling transfer-learning of NLP systems from high-resource languages to low-resource languages.
NeuSpell: A Neural Spelling Correction Toolkit
We introduce NeuSpell, an open-source toolkit for spelling correction in English.
On Learning Text Style Transfer with Direct Rewards
In most cases, the lack of parallel corpora makes it impossible to directly train supervised models for the text style transfer task.
Reducing Confusion in Active Learning for Part-Of-Speech Tagging
Active learning (AL) uses a data selection algorithm to select useful training samples to minimize annotation cost.
Weakly- and Semi-supervised Evidence Extraction
For many prediction tasks, stakeholders desire not only predictions but also supporting evidence that a human can use to verify its correctness.
Detecting Hallucinated Content in Conditional Neural Sequence Generation
Neural sequence models can generate highly fluent sentences, but recent studies have also shown that they are also prone to hallucinate additional content not supported by the input.
OCR Post Correction for Endangered Language Texts
There is little to no data available to build natural language processing models for most endangered languages.
