Search Results for author:
Found 45 papers, 24 papers with code
Iterative Feature Mining for Constraint-Based Data Collection to Increase Data Diversity and Model Robustness
Diverse data is crucial for training robust models, but crowdsourced text often lacks diversity as workers tend to write simple variations from prompts.
SQLucid: Grounding Natural Language Database Queries with Interactive Explanations
Though recent advances in machine learning have led to significant improvements in natural language interfaces for databases, the accuracy and reliability of these systems remain limited, especially in high-stakes domains.
Do Text-to-Vis Benchmarks Test Real Use of Visualisations?
Large language models are able to generate code for visualisations in response to user requests.
More Victories, Less Cooperation: Assessing Cicero's Diplomacy Play
The boardgame Diplomacy is a challenging setting for communicative and cooperative artificial intelligence.
What Can Natural Language Processing Do for Peer Review?
We hope that our work will help set the agenda for research in machine-assisted scientific quality control in the age of AI, within the NLP community and beyond.
Supporting Sensemaking of Large Language Model Outputs at Scale
Large language models (LLMs) are capable of generating multiple responses to a single prompt, yet little effort has been expended to help end-users or system designers make use of this capability.
A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity
While alignment algorithms are now commonly used to tune pre-trained language models towards a user's preferences, we lack explanations for the underlying mechanisms in which models become ``aligned'', thus making it difficult to explain phenomena like jailbreaks.
Interactive Text-to-SQL Generation via Editable Step-by-Step Explanations
Many techniques have been proposed to automatically generate SQL from natural language, but they suffer from two issues: (1) they still make many mistakes, particularly for complex queries, and (2) they do not provide a flexible way for non-expert users to validate and refine incorrect queries.
Augmenting Task-Oriented Dialogue Systems with Relation Extraction
The standard task-oriented dialogue pipeline uses intent classification and slot-filling to interpret user utterances.
Using Paraphrases to Study Properties of Contextual Embeddings
We use paraphrases as a unique source of data to analyze contextualized embeddings, with a particular focus on BERT.
Learning to Learn End-to-End Goal-Oriented Dialog From Related Dialog Tasks
For each goal-oriented dialog task of interest, large amounts of data need to be collected for end-to-end learning of a neural dialog system.
Micromodels for Efficient, Explainable, and Reusable Systems: A Case Study on Mental Health
Many statistical models have high accuracy on test benchmarks, but are not explainable, struggle in low-resource scenarios, cannot be reused for multiple tasks, and cannot easily integrate domain expertise.
Exploring Self-Identified Counseling Expertise in Online Support Forums
A growing number of people engage in online health forums, making it important to understand the quality of the advice they receive.
Quantifying and Avoiding Unfair Qualification Labour in Crowdsourcing
Extensive work has argued in favour of paying crowd workers a wage that is at least equivalent to the U. S. federal minimum wage.
Chord Embeddings: Analyzing What They Capture and Their Role for Next Chord Prediction and Artist Attribute Prediction
In the first case study, we demonstrate that using chord embeddings in a next chord prediction task yields predictions that more closely match those by experienced musicians.
Inconsistencies in Crowdsourced Slot-Filling Annotations: A Typology and Identification Methods
Using three new noisy crowd-annotated datasets, we show that a wide range of inconsistencies occur and can impact system performance if not addressed.
Exploring the Value of Personalized Word Embeddings
Our results show that a subset of words belonging to specific psycholinguistic categories tend to vary more in their representations across users and that combining generic and personalized word embeddings yields the best performance, with a 4. 7% relative reduction in perplexity.
A Novel Workflow for Accurately and Efficiently Crowdsourcing Predicate Senses and Argument Labels
Resources for Semantic Role Labeling (SRL) are typically annotated by experts at great expense.
Compositional Demographic Word Embeddings
Word embeddings are usually derived from corpora containing text from many individuals, thus leading to general purpose representations rather than individually personalized representations.
Improving Low Compute Language Modeling with In-Domain Embedding Initialisation
In the process, we show that the standard convention of tying input and output embeddings does not improve perplexity when initializing with embeddings trained on in-domain data.
Analyzing the Surprising Variability in Word Embedding Stability Across Languages
Word embeddings are powerful representations that form the foundation of many natural language processing architectures, both in English and in other languages.
No-Press Diplomacy: Modeling Multi-Agent Gameplay
Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal.
The Eighth Dialog System Technology Challenge
This paper introduces the Eighth Dialog System Technology Challenge.
No Press Diplomacy: Modeling Multi-Agent Gameplay
Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal.
An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction
We find that while the classifiers perform well on in-scope intent classification, they struggle to identify out-of-scope queries.
DSTC7 Task 1: Noetic End-to-End Response Selection
Goal-oriented dialogue in complex domains is an extremely challenging problem and there are relatively few datasets.
Conversational Response Selection
Goal-Oriented Dialogue Systems
SLATE: A Super-Lightweight Annotation Tool for Experts
Many annotation tools have been developed, covering a wide variety of tasks and providing features like user management, pre-processing, and automatic labeling.
Look Who's Talking: Inferring Speaker Attributes from Personal Longitudinal Dialog
We examine a large dialog corpus obtained from the conversation history of a single individual with 104 conversation partners.
Outlier Detection for Improved Data Quality and Diversity in Dialog Systems
We also present a novel data collection pipeline built atop our detection technique to automatically and iteratively mine unique data samples while discarding erroneous samples.
Dialog System Technology Challenge 7
This paper introduces the Seventh Dialog System Technology Challenges (DSTC), which use shared datasets to explore the problem of building dialog systems.
A Large-Scale Corpus for Conversation Disentanglement
Disentangling conversations mixed together in a single stream of messages is a difficult task, made harder by the lack of large manually annotated datasets.
Improving Text-to-SQL Evaluation Methodology
Second, we show that the current division of data into training and test sets measures robustness to variations in the way questions are asked, but only partially tests how well systems generalize to new queries; therefore, we propose a complementary dataset split for evaluation of future work.
Ranked #1 on
SQL Parsing
on IMDb
Effective Crowdsourcing for a New Type of Summarization Task
Most summarization research focuses on summarizing the entire given text, but in practice readers are often interested in only one aspect of the document or conversation.
Data Collection for Dialogue System: A Startup Perspective
In this paper, we present a study of crowdsourcing methods for a user intent classification task in our deployed dialogue system.
Factors Influencing the Surprising Instability of Word Embeddings
Despite the recent popularity of word embedding methods, there is only a small body of work exploring the limitations of these representations.
Identifying Products in Online Cybercrime Marketplaces: A Dataset for Fine-grained Domain Adaptation
One weakness of machine-learned NLP models is that they typically perform poorly on out-of-domain data.
Parsing with Traces: An $O(n^4)$ Algorithm and a Structural Representation
General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons.
Ranked #2 on
Missing Elements
on Penn Treebank
Understanding Task Design Trade-offs in Crowdsourced Paraphrase Collection
Linguistically diverse datasets are critical for training and evaluating robust machine learning systems, but data collection is a costly process that often requires experts.
Parsing with Traces: An O(n4) Algorithm and a Structural Representation
General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons.
