Search Results for author:
Found 26 papers, 18 papers with code
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
Chain-of-thought (CoT) via prompting is the de facto method for eliciting reasoning capabilities from large language models (LLMs).
CodeUpdateArena: Benchmarking Knowledge Editing on API Updates
An instance in our benchmark consists of a synthetic API function update paired with a program synthesis example that uses the updated functionality; our goal is to update an LLM to be able to solve this program synthesis example without providing documentation of the update at inference time.
LoFiT: Localized Fine-tuning on LLM Representations
For truthfulness and reasoning tasks, we find that LoFiT's intervention vectors are more effective for LLM adaptation than vectors from representation intervention methods such as Inference-time Intervention.
Unveiling the Impact of Coding Data Instruction Fine-Tuning on Large Language Models Reasoning
First, coding data tuning enhances the overall reasoning capabilities of LLMs across different model families and scales.
CityLLaVA: Efficient Fine-Tuning for VLMs in City Scenario
In the vast and dynamic landscape of urban settings, Traffic Safety Description and Analysis plays a pivotal role in applications ranging from insurance inspection to accident prevention.
AmbigDocs: Reasoning across Documents on Different Entities under the Same Name
and a set of documents discussing different people named Michael Jordan, can LMs distinguish entity mentions to generate a cohesive answer to the question?
STDiff: Spatio-temporal Diffusion for Continuous Stochastic Video Prediction
Predicting future frames of a video is challenging because it is difficult to learn the uncertainty of the underlying factors influencing their contents.
Crafting In-context Examples according to LMs' Parametric Knowledge
We perform analysis on three multi-answer question answering datasets, which allows us to further study answer set ordering strategies based on the LM's knowledge of each answer.
Effective Large Language Model Adaptation for Improved Grounding and Citation Generation
Our framework tunes LLMs to selfground the claims in their responses and provide accurate citations to retrieved documents.
MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning
We evaluate a range of LLMs and prompting techniques on this dataset and characterize the gaps that remain for techniques like chain-of-thought to perform robust reasoning.
EEL: Efficiently Encoding Lattices for Reranking
Standard decoding approaches for conditional text generation tasks typically search for an output hypothesis with high model probability, but this may not yield the best hypothesis according to human judgments of quality.
SatLM: Satisfiability-Aided Language Models Using Declarative Prompting
In this paper, we propose a new satisfiability-aided language modeling (SatLM) approach for improving the reasoning capabilities of LLMs.
Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting
We first generate sets of candidate explanations for each example in the prompt using a leave-one-out scheme, then find an effective combination of these explanations with a two-stage framework.
Video Prediction by Efficient Transformers
Video prediction is a challenging computer vision task that has a wide range of applications.
Complementary Explanations for Effective In-Context Learning
Large language models (LLMs) have exhibited remarkable capabilities in learning from explanations in prompts, but there has been limited understanding of exactly how these explanations function or why they are effective.
Assessing Out-of-Domain Language Model Performance from Few Examples
We address the task of predicting out-of-domain (OOD) performance in a few-shot fashion: given a few target-domain examples and a set of models with similar training performance, can we understand how these models will perform on OOD test data?
A unified model for continuous conditional video prediction
Different conditional video prediction tasks, like video future frame prediction and video frame interpolation, are normally solved by task-related models even though they share many common underlying characteristics.
Diagnosing Ensemble Few-Shot Classifiers
The base learners and labeled samples (shots) in an ensemble few-shot classifier greatly affect the model performance.
The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning
Does prompting a large language model (LLM) like GPT-3 with explanations improve in-context learning?
VPTR: Efficient Transformers for Video Prediction
Based on this new Transformer block, a fully autoregressive video future frames prediction Transformer is proposed.
Can Explanations Be Useful for Calibrating Black Box Models?
Our approach first extracts a set of features combining human intuition about the task with model attributions generated by black box interpretation techniques, then uses a simple calibrator, in the form of a classifier, to predict whether the base model was correct or not.
RnG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering
We present RnG-KBQA, a Rank-and-Generate approach for KBQA, which remedies the coverage issue with a generation model while preserving a strong generalization capability.
Connecting Attributions and QA Model Behavior on Realistic Counterfactuals
When a model attribution technique highlights a particular part of the input, a user might understand this highlight as making a statement about counterfactuals (Miller, 2019): if that part of the input were to change, the model's prediction might change as well.
Optimal Neural Program Synthesis from Multimodal Specifications
Multimodal program synthesis, which leverages different types of user input to synthesize a desired program, is an attractive way to scale program synthesis to challenging settings; however, it requires integrating noisy signals from the user, like natural language, with hard constraints on the program's behavior.
Benchmarking Multimodal Regex Synthesis with Complex Structures
Existing datasets for regular expression (regex) generation from natural language are limited in complexity; compared to regex tasks that users post on StackOverflow, the regexes in these datasets are simple, and the language used to describe them is not diverse.
Sketch-Driven Regular Expression Generation from Natural Language and Examples
Our system achieves state-of-the-art performance on the prior datasets and solves 57% of the real-world dataset, which existing neural systems completely fail on.
