Search Results for author:
Found 32 papers, 24 papers with code
Third-Party Language Model Performance Prediction from Instruction
Language model-based instruction-following systems have lately shown increasing performance on many benchmark tasks, demonstrating the capability of adapting to a broad variety of instructions.
Tur[k]ingBench: A Challenge Benchmark for Web Agents
We evaluate the performance of state-of-the-art models, including language-only, vision-only, and layout-only models, and their combinations, on this benchmark.
Set the Clock: Temporal Alignment of Pretrained Language Models
We then develop several methods, from prompting to finetuning, to align LMs to use their most recent knowledge when answering questions, and investigate various factors in this alignment.
Can Language Models Act as Knowledge Bases at Scale?
Large language models (LLMs) have demonstrated remarkable proficiency in understanding and generating responses to complex queries through large-scale pre-training.
OLMo: Accelerating the Science of Language Models
Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs.
Tuning Language Models by Proxy
Despite the general capabilities of large pretrained language models, they consistently benefit from further adaptation to better achieve desired behaviors.
Fine-grained Hallucination Detection and Editing for Language Models
On our benchmark, our automatic and human evaluations show that FAVA significantly outperforms ChatGPT and GPT-4 on fine-grained hallucination detection, and edits suggested by FAVA improve the factuality of LM-generated text.
Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
Since the release of T\"ULU [Wang et al., 2023b], open resources for instruction tuning have developed quickly, from better base models to new finetuning techniques.
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
Our framework trains a single arbitrary LM that adaptively retrieves passages on-demand, and generates and reflects on retrieved passages and its own generations using special tokens, called reflection tokens.
Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging
In this work, we study Reinforcement Learning from Personalized Human Feedback (RLPHF) problem, wherein LLMs are aligned to multiple (sometimes conflicting) preferences by modeling alignment as a Multi-Objective Reinforcement Learning (MORL) problem.
BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
Retrieval augmentation addresses many critical problems in large language models such as hallucination, staleness, and privacy leaks.
How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources
Our evaluations show that the best model in any given evaluation reaches on average 87% of ChatGPT performance, and 73% of GPT-4 performance, suggesting that further investment in building better base models and instruction-tuning data is required to close the gap.
TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering
We introduce TIFA (Text-to-Image Faithfulness evaluation with question Answering), an automatic evaluation metric that measures the faithfulness of a generated image to its text input via visual question answering (VQA).
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Applying our method to the vanilla GPT3, we demonstrate a 33% absolute improvement over the original model on Super-NaturalInstructions, on par with the performance of InstructGPT-001, which was trained with private user data and human annotations.
HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation
By converting instructions into modules, HINT models can effectively disregard the length of instructions and few-shot example inputs in terms of compute usage.
One Embedder, Any Task: Instruction-Finetuned Text Embeddings
Our analysis suggests that INSTRUCTOR is robust to changes in instructions, and that instruction finetuning mitigates the challenge of training a single model on diverse datasets.
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
This large and diverse collection of tasks enables rigorous benchmarking of cross-task generalization under instructions -- training models to follow instructions on a subset of tasks and evaluating them on the remaining unseen ones.
Probing Across Time: What Does RoBERTa Know and When?
Models of language trained on very large corpora have been demonstrated useful for NLP.
MultiModalQA: Complex Question Answering over Text, Tables and Images
When answering complex questions, people can seamlessly combine information from visual, textual and tabular sources.
Automated Lay Language Summarization of Biomedical Scientific Reviews
Health literacy has emerged as a crucial factor in making appropriate health decisions and ensuring treatment outcomes.
LiveQA: A Question Answering Dataset over Sports Live
In this paper, we introduce LiveQA, a new question answering dataset constructed from play-by-play live broadcast.
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics
Experiments across four datasets show that these model-dependent measures reveal three distinct regions in the data map, each with pronounced characteristics.
Do NLP Models Know Numbers? Probing Numeracy in Embeddings
The ability to understand and work with numbers (numeracy) is critical for many complex reasoning tasks.
Machine Reading Comprehension: a Literature Review
Machine reading comprehension aims to teach machines to understand a text like a human and is a new challenging direction in Artificial Intelligence.
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
We introduce a new English reading comprehension benchmark, DROP, which requires Discrete Reasoning Over the content of Paragraphs.
Ranked #14 on
Question Answering
on DROP Test
Toward Fast and Accurate Neural Discourse Segmentation
Discourse segmentation, which segments texts into Elementary Discourse Units, is a fundamental step in discourse analysis.
Bag-of-Words as Target for Neural Machine Translation
However, most of the existing neural machine translation models only use one of the correct translations as the targets, and the other correct sentences are punished as the incorrect sentences in the training stage.
Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification
Machine reading comprehension (MRC) on real web data usually requires the machine to answer a question by analyzing multiple passages retrieved by search engine.
Ranked #3 on
Question Answering
on MS MARCO
Tag-Enhanced Tree-Structured Neural Networks for Implicit Discourse Relation Classification
Identifying implicit discourse relations between text spans is a challenging task because it requires understanding the meaning of the text.
General Classification
Implicit Discourse Relation Classification
+3
DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
Experiments show that human performance is well above current state-of-the-art baseline systems, leaving plenty of room for the community to make improvements.
A Two-Stage Parsing Method for Text-Level Discourse Analysis
Previous work introduced transition-based algorithms to form a unified architecture of parsing rhetorical structures (including span, nuclearity and relation), but did not achieve satisfactory performance.
Ranked #5 on
Discourse Parsing
on RST-DT
Towards Non-projective High-Order Dependency Parser
This paper presents a novel high-order dependency parsing framework that targets non-projective treebanks.
