Search Results for author:
Found 84 papers, 31 papers with code
Mega: Moving Average Equipped Gated Attention
The design choices in the Transformer attention mechanism, including weak inductive bias and quadratic computational complexity, have limited its application for modeling long sequences.
Ranked #1 on
Long-range modeling
on LRA
Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnostics
Recent works that revealed the vulnerability of dialogue state tracking (DST) models to distributional shifts have made holistic comparisons on robustness and qualitative analyses increasingly important for understanding their relative performance.
Ranked #4 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.1
(using extra training data)
Dialogue State Tracking
Multi-domain Dialogue State Tracking
+1
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
The quadratic complexity and weak length extrapolation of Transformers limits their ability to scale to long sequences, and while sub-quadratic solutions like linear attention and state space models exist, they empirically underperform Transformers in pretraining efficiency and downstream task accuracy.
Transfer Learning for Low-Resource Neural Machine Translation
Ensembling and unknown word replacement add another 2 Bleu which brings the NMT performance on low-resource machine translation close to a strong syntax based machine translation (SBMT) system, exceeding its performance on one language pair.
Luna: Linear Unified Nested Attention
Specifically, with the first attention function, Luna packs the input sequence into a sequence of fixed length.
WARP: Word-level Adversarial ReProgramming
Transfer learning from pretrained language models recently became the dominant approach for solving many NLP tasks.
Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation
We study the power of cross-attention in the Transformer architecture within the context of transfer learning for machine translation, and extend the findings of studies into cross-attention when training from scratch.
CaSiNo: A Corpus of Campsite Negotiation Dialogues for Automatic Negotiation Systems
Automated systems that negotiate with humans have broad applications in pedagogy and conversational AI.
NewsEdits: A News Article Revision Dataset and a Document-Level Reasoning Challenge
News article revision histories provide clues to narrative and factual evolution in news articles.
NewsEdits: A News Article Revision Dataset and a Novel Document-Level Reasoning Challenge
News article revision histories provide clues to narrative and factual evolution in news articles.
X-METRA-ADA: Cross-lingual Meta-Transfer Learning Adaptation to Natural Language Understanding and Question Answering
We extensively evaluate our framework on two challenging cross-lingual NLU tasks: multilingual task-oriented dialog and typologically diverse question answering.
RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation
Endowing chatbots with a consistent persona is essential to an engaging conversation, yet it remains an unresolved challenge.
Grounding Conversations with Improvised Dialogues
Effective dialogue involves grounding, the process of establishing mutual knowledge that is essential for communication between people.
Many-to-English Machine Translation Tools, Data, and Pretrained Models
While there are more than 7000 languages in the world, most translation research efforts have targeted a few high-resource languages.
A Grounded Unsupervised Universal Part-of-Speech Tagger for Low-Resource Languages
We also show extrinsically that incorporating our POS tagger into a name tagger leads to state-of-the-art tagging performance in Sinhalese and Kinyarwanda, two languages with nearly no labeled POS data available.
WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models
We present WinoQueer: a benchmark specifically designed to measure whether large language models (LLMs) encode biases that are harmful to the LGBTQ+ community.
Finding the Optimal Vocabulary Size for Neural Machine Translation
We cast neural machine translation (NMT) as a classification task in an autoregressive setting and analyze the limitations of both classification and autoregression components.
Opponent Modeling in Negotiation Dialogues by Related Data Adaptation
A practical model for this task needs to infer these priorities of the opponent on the fly based on partial dialogues as input, without needing additional annotations for training.
Cross-lingual Lifelong Learning
The longstanding goal of multi-lingual learning has been to develop a universal cross-lingual model that can withstand the changes in multi-lingual data distributions.
Continual Dialogue State Tracking via Example-Guided Question Answering
Dialogue systems are frequently updated to accommodate new services, but naively updating them by continually training with data for new services in diminishing performance on previously learnt services.
Comprehensible Context-driven Text Game Playing
As such, an LSTM-based DQN can take tens of days to finish the training process.
Learn How to Cook a New Recipe in a New House: Using Map Familiarization, Curriculum Learning, and Bandit Feedback to Learn Families of Text-Based Adventure Games
We consider the task of learning to play families of text-based computer adventure games, i. e., fully textual environments with a common theme (e. g. cooking) and goal (e. g. prepare a meal from a recipe) but with different specifics; new instances of such games are relatively straightforward for humans to master after a brief exposure to the genre but have been curiously difficult for computer agents to learn.
Challenges in Context-Aware Neural Machine Translation
Context-aware neural machine translation involves leveraging information beyond sentence-level context to resolve inter-sentential discourse dependencies and improve document-level translation quality, and has given rise to a number of recent techniques.
Experience Grounds Language
Language understanding research is held back by a failure to relate language to the physical world it describes and to the social interactions it facilitates.
Macro-Average: Rare Types Are Important Too
While traditional corpus-level evaluation metrics for machine translation (MT) correlate well with fluency, they struggle to reflect adequacy.
"Don't quote me on that": Finding Mixtures of Sources in News Articles
Journalists publish statements provided by people, or \textit{sources} to contextualize current events, help voters make informed decisions, and hold powerful individuals accountable.
Identifying Informational Sources in News Articles
News articles are driven by the informational sources journalists use in reporting.
Learning to Generalize for Sequential Decision Making
However, the large amount of computation necessary to adequately train and explore the search space of sequential decision making, under a reinforcement learning paradigm, precludes the inclusion of large contextualized language models, which might otherwise enable the desired generalization ability.
Machine Translation Robustness to Natural Asemantic Variation
We find that existing MT models fail when presented with NAV data, but we demonstrate strategies to improve performance on NAV by fine-tuning them with human-generated variations.
Authorship Style Transfer with Policy Optimization
Authorship style transfer aims to rewrite a given text into a specified target while preserving the original meaning in the source.
Recurrent Neural Networks as Weighted Language Recognizers
We investigate the computational complexity of various problems for simple recurrent neural networks (RNNs) as formal models for recognizing weighted languages.
Building a Fine-Grained Entity Typing System Overnight for a New X (X = Language, Domain, Genre)
Recent research has shown great progress on fine-grained entity typing.
Using Syntax-Based Machine Translation to Parse English into Abstract Meaning Representation
To make this work, we transform the AMR structure into a form suitable for the mechanics of SBMT and useful for modeling.
Augmenting Statistical Machine Translation with Subword Translation of Out-of-Vocabulary Words
Most statistical machine translation systems cannot translate words that are unseen in the training data.
Out-of-the-box Universal Romanization Tool uroman
We present uroman, a tool for converting text in myriads of languages and scripts such as Chinese, Arabic and Cyrillic into a common Latin-script representation.
Translating a Language You Don't Know In the Chinese Room
In a corruption of John Searle{'}s famous AI thought experiment, the Chinese Room (Searle, 1980), we twist its original intent by enabling humans to translate text, e. g. from Uyghur to English, even if they don{'}t have any prior knowledge of the source language.
Cross-lingual Name Tagging and Linking for 282 Languages
The ambitious goal of this work is to develop a cross-lingual name tagging and linking framework for 282 languages that exist in Wikipedia.
ELISA-EDL: A Cross-lingual Entity Extraction, Linking and Localization System
We demonstrate ELISA-EDL, a state-of-the-art re-trainable system to extract entity mentions from low-resource languages, link them to external English knowledge bases, and visualize locations related to disaster topics on a world heatmap.
SemEval-2017 Task 9: Abstract Meaning Representation Parsing and Generation
In the generation subtask, participants were asked to generate English sentences given AMR graphs in the news/forum domain.
Towards Controllable Story Generation
We present a general framework of analyzing existing story corpora to generate controllable and creative new stories.
An Analysis (and an Annotated Corpus) of User Responses to Machine Translation Output
We present an annotated resource consisting of open-domain translation requests, automatic translations and user-provided corrections collected from casual users of the translation portal http://reverso. net.
Cross-lingual Multi-Level Adversarial Transfer to Enhance Low-Resource Name Tagging
We focus on improving name tagging for low-resource languages using annotations from related languages.
Extracting Structured Scholarly Information from the Machine Translation Literature
Understanding the experimental results of a scientific paper is crucial to understanding its contribution and to comparing it with related work.
Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation
Given a rough, word-by-word gloss of a source language sentence, target language natives can uncover the latent, fully-fluent rendering of the translation.
SARAL: A Low-Resource Cross-Lingual Domain-Focused Information Retrieval System for Effective Rapid Document Triage
In this paper we present an end-to-end cross-lingual information retrieval (CLIR) and summarization system for low-resource languages that 1) enables English speakers to search foreign language repositories of text and audio using English queries, 2) summarizes the retrieved documents in English with respect to a particular information need, and 3) provides complete transcriptions and translations as needed.
What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis
While recent work has shown promising results on cross-lingual transfer from high-resource languages to low-resource languages, it is unclear what knowledge is transferred.
A Universal Parent Model for Low-Resource Neural Machine Translation Transfer
In this work, we present a `universal' pre-trained neural parent model with constant vocabulary that can be used as a starting point for training practically any new low-resource language to a fixed target language.
Cross-lingual Joint Entity and Word Embedding to Improve Entity Linking and Parallel Sentence Mining
Because this multilingual common space directly relates the semantics of contextual words in the source language to that of entities in the target language, we leverage it for unsupervised cross-lingual entity linking.
Contextualized Cross-Lingual Event Trigger Extraction with Minimal Resources
Our work is the first to experiment with two event architecture variants in a cross-lingual setting, to show the effectiveness of contextualized embeddings obtained using BERT, and to explore and analyze its performance on Arabic.
Cross-lingual Structure Transfer for Relation and Event Extraction
The identification of complex semantic structures such as events and entity relations, already a challenging Information Extraction task, is doubly difficult from sources written in under-resourced and under-annotated languages.
Do Nuclear Submarines Have Nuclear Captains? A Challenge Dataset for Commonsense Reasoning over Adjectives and Objects
This provides evidence that the amount of commonsense knowledge encoded in these language models does not extend far beyond that already baked into the word embeddings.
Exploring Early Prediction of Buyer-Seller Negotiation Outcomes
Agents that negotiate with humans find broad applications in pedagogy and conversational AI.
Zero-Shot Learning of Text Adventure Games with Sentence-Level Semantics
Reinforcement learning algorithms such as Q-learning have shown great promise in training models to learn the optimal action to take for a given system state; a goal in applications with an exploratory or adversarial nature such as task-oriented dialogues or games.
Cross-lingual Structure Transfer for Zero-resource Event Extraction
Most of the current cross-lingual transfer learning methods for Information Extraction (IE) have been only applied to name tagging.
Summary-Oriented Question Generation for Informational Queries
Our model shows SOTA performance of SQ generation on the NQ dataset (20. 1 BLEU-4).
Connecting the Dots: Event Graph Schema Induction with Path Language Modeling
Event schemas can guide our understanding and ability to make predictions with respect to what might happen next.
Can Sequence-to-Sequence Models Crack Substitution Ciphers?
Decipherment of historical ciphers is a challenging problem.
Multitask Learning for Class-Imbalanced Discourse Classification
Small class-imbalanced datasets, common in many high-level semantic tasks like discourse analysis, present a particular challenge to current deep-learning architectures.
NewsEdits: A Dataset of Revision Histories for News Articles (Technical Report: Data Processing)
In this work, we present, to our knowledge, the first publicly available dataset of news article revision histories, or NewsEdits.
Modeling "Newsworthiness" for Lead-Generation Across Corpora
Journalists obtain "leads", or story ideas, by reading large corpora of government records: court cases, proposed bills, etc.
StateCensusLaws.org: A Web Application for Consuming and Annotating Legal Discourse Learning
In this work, we create a web application to highlight the output of NLP models trained to parse and label discourse segments in law text.
Viola: A Topic Agnostic Generate-and-Rank Dialogue System
We present Viola, an open-domain dialogue system for spoken conversation that uses a topic-agnostic dialogue manager based on a simple generate-and-rank approach.
Salience-Aware Event Chain Modeling for Narrative Understanding
Storytelling, whether via fables, news reports, documentaries, or memoirs, can be thought of as the communication of interesting and related events that, taken together, form a concrete process.
Multitask Semi-Supervised Learning for Class-Imbalanced Discourse Classification
As labeling schemas evolve over time, small differences can render datasets following older schemas unusable.
Ranked #1 on
Text Classification
on NewsDiscourse
Explaining Face Presentation Attack Detection Using Natural Language
Due to the limited amount of annotated data in our study, we apply a light-weight LSTM network as our natural language generation model.
Segmenting Numerical Substitution Ciphers
In this work, we propose the first automatic methods to segment those ciphers using Byte Pair Encoding (BPE) and unigram language models.
Know Where You're Going: Meta-Learning for Parameter-Efficient Fine-Tuning
A recent family of techniques, dubbed lightweight fine-tuning methods, facilitates parameter-efficient transfer learning by updating only a small set of additional parameters while keeping the parameters of the pretrained language model frozen.
Investigating the Benefits of Free-Form Rationales
After controlling for instances where rationales leak the correct answer while not providing additional background knowledge, we find that incorporating only 5% of rationales during training can boost model performance by 47. 22% for CoS-E and 57. 14% for ECQA during inference.
Towards WinoQueer: Developing a Benchmark for Anti-Queer Bias in Large Language Models
This paper presents exploratory work on whether and to what extent biases against queer and trans people are encoded in large language models (LLMs) such as BERT.
Augmenting Training Data for Massive Semantic Matching Models in Low-Traffic E-commerce Stores
We use the Aggregated Label eXtreme Multi-label Classification (AL-XMC) system (Shen et al., 2020) as an example semantic matching model and show via crowd-sourced human judgments that, when the training data is augmented through query reformulations, the quality of AL-XMC improves over a baseline that does not use query reformulation.
Checks and Strategies for Enabling Code-Switched Machine Translation
Code-switching is a common phenomenon among multilingual speakers, where alternation between two or more languages occurs within the context of a single conversation.
Anger Breeds Controversy: Analyzing Controversy and Emotions on Reddit
We collect discussions from a wide variety of topical forums and use emotion detection to recognize a range of emotions from text, including anger, fear, joy, admiration, etc.
CPL-NoViD: Context-Aware Prompt-based Learning for Norm Violation Detection in Online Communities
Detecting norm violations in online communities is critical to maintaining healthy and safe spaces for online discussions.
Analyzing Norm Violations in Live-Stream Chat
Previous approaches to detecting toxic language and norm violations have been primarily concerned with conversations from online forums and social media, such as Reddit and Twitter.
Multilingual Sentence-Level Semantic Search using Meta-Distillation Learning
Our approach leverages meta-distillation learning based on MAML, an optimization-based Model-Agnostic Meta-Learner.
Tracking the Newsworthiness of Public Documents
Journalists must find stories in huge amounts of textual data (e. g. leaks, bills, press releases) as part of their jobs: determining when and why text becomes news can help us understand coverage patterns and help us build assistive tools.
Can Language Model Moderators Improve the Health of Online Discourse?
Human moderation of online conversation is essential to maintaining civility and focus in a dialogue, but is challenging to scale and harmful to moderators.
