Search Results for author:
Found 23 papers, 16 papers with code
Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training
Highlighting particularly relevant regions of an image can improve the performance of vision-language models (VLMs) on various vision-language (VL) tasks by guiding the model to attend more closely to these regions of interest.
Language-guided Skill Learning with Temporal Variational Inference
We present an algorithm for skill discovery from expert demonstrations.
Soft Self-Consistency Improves Language Model Agents
Current "sample and select" methods such as self-consistency (SC) rely on majority voting to score answers.
GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
As Large Language Models (LLMs) are integrated into critical real-world applications, their strategic and logical reasoning abilities are increasingly crucial.
MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models
Experiments on seven widely-used commonsense and math reasoning benchmarks show that MAGDi improves the reasoning capabilities of smaller models, outperforming several methods that distill from a single teacher and multiple teachers.
ReGAL: Refactoring Programs to Discover Generalizable Abstractions
While large language models (LLMs) are increasingly being used for program synthesis, they lack the global view needed to develop useful abstractions; they generally predict programs one at a time, often repeating the same functionality.
Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models
An increasing number of vision-language tasks can be handled with little to no training, i. e., in a zero and few-shot manner, by marrying large language models (LLMs) to vision encoders, resulting in large vision-language models (LVLMs).
Zero and Few-shot Semantic Parsing with Ambiguous Inputs
We attempt to address this shortcoming by introducing AmP, a framework, dataset, and challenge for translating ambiguous natural language to formal representations like logic and code.
Did You Mean...? Confidence-based Trade-offs in Semantic Parsing
We then examine how confidence scores can help optimize the trade-off between usability and safety.
Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning
Visual Question Answering (VQA) models often perform poorly on out-of-distribution data and struggle on domain generalization.
Calibrated Interpretation: Confidence Estimation in Semantic Parsing
Sequence generation models are increasingly being used to translate natural language into programs, i. e. to perform executable semantic parsing.
Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA
Natural language is ambiguous.
When More Data Hurts: A Troubling Quirk in Developing Broad-Coverage Natural Language Understanding Systems
Rejecting class imbalance as the sole culprit, we reveal that the trend is closely associated with an effect we call source signal dilution, where strong lexical cues for the new symbol become diluted as the training dataset grows.
The Curious Case of Control
Given the advanced fluency of large generative language models, we ask whether model outputs are consistent with these heuristics, and to what degree different models are consistent with each other.
Visual Commonsense in Pretrained Unimodal and Multimodal Models
Our commonsense knowledge about objects includes their typical visual attributes; we know that bananas are typically yellow or green, and not purple.
Ranked #1 on
Visual Commonsense Tests
on ViComTe-color
Guiding Multi-Step Rearrangement Tasks with Natural Language Instructions
Our model completes block manipulation tasks with synthetic commands 530 more often than a UNet-based baseline, and learns to localize actions correctly while creating a mapping of symbols to perceptual input that supports compositional reasoning.
Calibrating Concepts and Operations: Towards Symbolic Reasoning on Real Images
Our experiments show CCO substantially boosts the performance of neural symbolic methods on real images.
Joint Universal Syntactic and Semantic Parsing
While numerous attempts have been made to jointly parse syntax and semantics, high performance in one domain typically comes at the price of performance in the other.
Iterative Paraphrastic Augmentation with Discriminative Span Alignment
We introduce a novel paraphrastic augmentation strategy based on sentence-level lexically constrained paraphrasing and discriminative span alignment.
Universal Decompositional Semantic Parsing
We introduce a transductive model for parsing into Universal Decompositional Semantics (UDS) representations, which jointly learns to map natural language utterances into UDS graph structures and annotate the graph with decompositional semantic attribute scores.
The Universal Decompositional Semantics Dataset and Decomp Toolkit
We present the Universal Decompositional Semantics (UDS) dataset (v1. 0), which is bundled with the Decomp toolkit (v0. 1).
A Discriminative Neural Model for Cross-Lingual Word Alignment
We introduce a novel discriminative word alignment model, which we integrate into a Transformer-based machine translation model.
