Search Results for author:
Found 7 papers, 4 papers with code
Large Language Models Can Be Easily Distracted by Irrelevant Context
We use this benchmark to measure the distractibility of cutting-edge prompting techniques for large language models, and find that the model performance is dramatically decreased when irrelevant information is included.
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
BIG-Bench (Srivastava et al., 2022) is a diverse evaluation suite that focuses on tasks believed to be beyond the capabilities of current language models.
Compositional Semantic Parsing with Large Language Models
Humans can reason compositionally when presented with new tasks.
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Although chain-of-thought prompting has shown impressive results on many natural language reasoning tasks, it often performs poorly on tasks which need to solve problems harder than the demonstration examples.
Ranked #14 on
Arithmetic Reasoning
on GSM8K
*-CFQ: Analyzing the Scalability of Machine Learning on a Compositional Task
We present *-CFQ ("star-CFQ"): a suite of large-scale datasets of varying scope based on the CFQ semantic parsing benchmark, designed for principled investigation of the scalability of machine learning systems in a realistic compositional task setting.
Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures
While mainstream machine learning methods are known to have limited ability to compositionally generalize, new architectures and techniques continue to be proposed to address this limitation.
Measuring Compositional Generalization: A Comprehensive Method on Realistic Data
We present a large and realistic natural language question answering dataset that is constructed according to this method, and we use it to analyze the compositional generalization ability of three machine learning architectures.
