Search Results for author:
Found 34 papers, 24 papers with code
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models
Language models have been shown to exhibit positive scaling, where performance improves as models are scaled up in terms of size, compute, or data.
Med-EASi: Finely Annotated Dataset and Models for Controllable Simplification of Medical Texts
Automatic medical text simplification can assist providers with patient-friendly communication and make medical texts more accessible, thereby improving health literacy.
Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
Spurred by advancements in scale, large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot -- i. e., without adaptation on downstream data.
REPLUG: Retrieval-Augmented Black-Box Language Models
We introduce REPLUG, a retrieval-augmented language modeling framework that treats the language model (LM) as a black box and augments it with a tuneable retrieval model.
Ranked #5 on
Question Answering
on TriviaQA
Zero-shot causal learning
There are a large number of methods to predict the effect of an existing intervention based on historical data from individuals who received it.
Retrieval-Augmented Multimodal Language Modeling
To integrate knowledge in a more scalable and modular way, we propose a retrieval-augmented multimodal model, which enables a base multimodal model (generator) to refer to relevant knowledge fetched by a retriever from external memory (e. g., multimodal documents on the web).
Ranked #1 on
Text-to-Image Generation
on COCO
Holistic Evaluation of Language Models
We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models.
Deep Bidirectional Language-Knowledge Graph Pretraining
Pretraining a language model (LM) on text has been shown to help various downstream NLP tasks.
Ranked #1 on
Riddle Sense
on Riddle Sense
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
VQA-GNN: Reasoning with Multimodal Semantic Graph for Visual Question Answering
In this work, we propose a novel visual question answering method, VQA-GNN, which unifies the image-level information and conceptual knowledge to perform joint reasoning of the scene.
LinkBERT: Pretraining Language Models with Document Links
Language model (LM) pretraining can learn various knowledge from text corpora, helping downstream tasks.
Ranked #1 on
Document Classification
on HOC
GreaseLM: Graph REASoning Enhanced Language Models for Question Answering
Answering complex questions about textual narratives requires reasoning over both stated context and the world knowledge that underlies it.
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases.
Ranked #1 on
Task-Oriented Dialogue Systems
on KVRET
Extending the WILDS Benchmark for Unsupervised Adaptation
Unlabeled data can be a powerful point of leverage for mitigating these distribution shifts, as it is frequently much more available than labeled data and can often be obtained from distributions beyond the source distribution as well.
GreaseLM: Graph REASoning Enhanced Language Models
Answering complex questions about textual narratives requires reasoning over both stated context and the world knowledge that underlies it.
LM-Critic: Language Models for Unsupervised Grammatical Error Correction
Training a model for grammatical error correction (GEC) requires a set of labeled ungrammatical / grammatical sentence pairs, but manually annotating such pairs can be expensive.
Ranked #2 on
Grammatical Error Correction
on Unrestricted
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Break-It-Fix-It: Unsupervised Learning for Program Repair
To bridge this gap, we propose a new training approach, Break-It-Fix-It (BIFI), which has two key ideas: (i) we use the critic to check a fixer's output on real bad inputs and add good (fixed) outputs to the training data, and (ii) we train a breaker to generate realistic bad code from good code.
Ranked #1 on
Program Repair
on DeepFix
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
The problem of answering questions using knowledge from pre-trained language models (LMs) and knowledge graphs (KGs) presents two challenges: given a QA context (question and answer choice), methods need to (i) identify relevant knowledge from large KGs, and (ii) perform joint reasoning over the QA context and KG.
Ranked #2 on
Riddle Sense
on Riddle Sense
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Distribution shifts -- where the training distribution differs from the test distribution -- can substantially degrade the accuracy of machine learning (ML) systems deployed in the wild.
Graph-based, Self-Supervised Program Repair from Diagnostic Feedback
Second, we present a self-supervised learning paradigm for program repair that leverages unlabeled programs available online to create a large amount of extra program repair examples, which we use to pre-train our models.
Ranked #1 on
Program Synthesis
on SPoC TestW
A Neural Topic-Attention Model for Medical Term Abbreviation Disambiguation
Specifically, a neural topic-attention model is applied to learn improved contextualized sentence representations for medical term abbreviation disambiguation.
CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases
We present CoSQL, a corpus for building cross-domain, general-purpose database (DB) querying dialogue systems.
Ranked #8 on
Dialogue State Tracking
on CoSQL
ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks
Scientific article summarization is challenging: large, annotated corpora are not available, and the summary should ideally include the article's impacts on research community.
Ranked #1 on
Scientific Document Summarization
on CL-SciSumm
The CL-SciSumm Shared Task 2018: Results and Key Insights
This overview describes the official results of the CL-SciSumm Shared Task 2018 -- the first medium-scale shared task on scientific document summarization in the computational linguistics (CL) domain.
Overview and Results: CL-SciSumm Shared Task 2019
All papers are from the open access research papers in the CL domain.
SParC: Cross-Domain Semantic Parsing in Context
The best model obtains an exact match accuracy of 20. 2% over all questions and less than10% over all interaction sequences, indicating that the cross-domain setting and the con-textual phenomena of the dataset present significant challenges for future research.
TopicEq: A Joint Topic and Mathematical Equation Model for Scientific Texts
Scientific documents rely on both mathematics and text to communicate ideas.
Ranked #1 on
Topic Models
on arXiv
(Topic Coherence@50 metric)
SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Task
In this paper we propose SyntaxSQLNet, a syntax tree network to address the complex and cross-domain text-to-SQL generation task.
Ranked #7 on
Text-To-SQL
on SParC
SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-Domain Text-to-SQL Task
In this paper we propose SyntaxSQLNet, a syntax tree network to address the complex and cross-domain text-to-SQL generation task.
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task
We define a new complex and cross-domain semantic parsing and text-to-SQL task where different complex SQL queries and databases appear in train and test sets.
Ranked #10 on
Semantic Parsing
on spider
Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering
Coreference resolution aims to identify in a text all mentions that refer to the same real-world entity.
Robust Multilingual Part-of-Speech Tagging via Adversarial Training
Adversarial training (AT) is a powerful regularization method for neural networks, aiming to achieve robustness to input perturbations.
Ranked #2 on
Part-Of-Speech Tagging
on UD
Graph-based Neural Multi-Document Summarization
We propose a neural multi-document summarization (MDS) system that incorporates sentence relation graphs.
Ranked #1 on
Multi-Document Summarization
on DUC 2004
