Search Results for author:
Found 32 papers, 18 papers with code
Retrieval Data Augmentation Informed by Downstream Question Answering Performance
Training retrieval models to fetch contexts for Question Answering (QA) over large corpora requires labeling relevant passages in those corpora.
The Art of Saying No: Contextual Noncompliance in Language Models
Chat-based language models are designed to be helpful, yet they should not comply with every user request.
OLMo: Accelerating the Science of Language Models
Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs.
Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
Since the release of T\"ULU [Wang et al., 2023b], open resources for instruction tuning have developed quickly, from better base models to new finetuning techniques.
Evaluating In-Context Learning of Libraries for Code Generation
Contemporary Large Language Models (LLMs) exhibit a high degree of code generation and comprehension capability.
TRAM: Bridging Trust Regions and Sharpness Aware Minimization
We propose Trust Region Aware Minimization (TRAM), a SAM algorithm fine-tuning for low parameter sharpness and smooth, informative representations preserving pre-trained structure.
How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources
Our evaluations show that the best model in any given evaluation reaches on average 87% of ChatGPT performance, and 73% of GPT-4 performance, suggesting that further investment in building better base models and instruction-tuning data is required to close the gap.
ReFIT: Relevance Feedback from a Reranker during Inference
Retrieve-and-rerank is a prevalent framework in neural information retrieval, wherein a bi-encoder network initially retrieves a pre-defined number of candidates (e. g., K=100), which are then reranked by a more powerful cross-encoder model.
LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
Motivated by our survey, we present LongEval, a set of guidelines for human evaluation of faithfulness in long-form summaries that addresses the following challenges: (1) How can we achieve high inter-annotator agreement on faithfulness scores?
AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization
We propose AGRO -- Adversarial Group discovery for Distributionally Robust Optimization -- an end-to-end approach that jointly identifies error-prone groups and improves accuracy on them.
Data-Efficient Finetuning Using Cross-Task Nearest Neighbors
Obtaining labeled data to train a model for a task of interest is often expensive.
Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets
We propose to tackle this problem by generating a debiased version of a dataset, which can then be used to train a debiased, off-the-shelf model, by simply replacing its training data.
Ranked #1 on
Natural Language Inference
on HANS
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
Readers of academic research papers often read with the goal of answering specific questions.
Ranked #1 on
Evidence Selection
on QASPER
Learning with Instance Bundles for Reading Comprehension
When training most modern reading comprehension models, all the questions associated with a context are treated as being independent from each other.
Mitigating False-Negative Contexts in Multi-document Question Answering with Retrieval Marginalization
We also show that retrieval marginalization results in 4. 1 QA F1 improvement over a non-marginalized baseline on HotpotQA in the fullwiki setting.
IIRC: A Dataset of Incomplete Information Reading Comprehension Questions
However, most existing reading comprehension (RC) tasks only focus on questions for which the contexts provide all the information required to answer them, thus not evaluating a system's performance at identifying a potential lack of sufficient information and locating sources for that information.
Evaluating NLP Models via Contrast Sets
Unfortunately, when a dataset has systematic gaps (e. g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities.
Easy, Reproducible and Quality-Controlled Data Collection with CROWDAQ
High-quality and large-scale data are key to success for AI systems.
Evaluating Models' Local Decision Boundaries via Contrast Sets
Unfortunately, when a dataset has systematic gaps (e. g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities.
Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning
Machine comprehension of texts longer than a single sentence often requires coreference resolution.
Iterative Search for Weakly Supervised Semantic Parsing
Training semantic parsers from question-answer pairs typically involves searching over an exponentially large space of logical forms, and an unguided search can easily be misled by spurious logical forms that coincidentally evaluate to the correct answer.
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
We introduce a new English reading comprehension benchmark, DROP, which requires Discrete Reasoning Over the content of Paragraphs.
Ranked #14 on
Question Answering
on DROP Test
Neural Semantic Parsing
Semantic parsing, the study of translating natural language utterances into machine-executable programs, is a well-established research area and has applications in question answering, instruction following, voice assistants, and code generation.
AllenNLP: A Deep Semantic Natural Language Processing Platform
This paper describes AllenNLP, a platform for research on deep learning methods in natural language understanding.
Neural Semantic Parsing with Type Constraints for Semi-Structured Tables
We present a new semantic parsing model for answering compositional questions on semi-structured Wikipedia tables.
Ontology-Aware Token Embeddings for Prepositional Phrase Attachment
Type-level word embeddings use the same set of parameters to represent all instances of a word regardless of its context, ignoring the inherent lexical ambiguity in language.
Experiment Segmentation in Scientific Discourse as Clause-level Structured Prediction using Recurrent Neural Networks
We propose a deep learning model for identifying structure within experiment narratives in scientific literature.
Tharwa: A Large Scale Dialectal Arabic - Standard Arabic - English Lexicon
Multiple levels of quality checks are performed on the output of each step in the creation process.
