Search Results for author:
Found 37 papers, 9 papers with code
CaT-BENCH: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans
A fundamental aspect of plans is the temporal order in which their steps needs to be executed, which reflects the underlying causal dependencies between them.
Multimodal Contextualized Semantic Parsing from Speech
We introduce Semantic Parsing in Contextual Environments (SPICE), a task designed to enhance artificial agents' contextual awareness by integrating multimodal inputs with prior contexts.
A Survey of Robotic Language Grounding: Tradeoffs between Symbols and Embeddings
With large language models, robots can understand language more flexibly and more capable than ever before.
Natural Language Can Help Bridge the Sim2Real Gap
We demonstrate that training the image encoder to predict the language description or the distance between descriptions of a sim or real image serves as a useful, data-efficient pretraining step that helps learn a domain-invariant image representation.
Distilling Algorithmic Reasoning from LLMs via Explaining Solution Programs
More specifically, we employ an LLM to generate explanations for a set of <problem, solution-program> pairs, then use <problem, explanation> pairs to fine-tune a smaller language model, which we refer to as the Reasoner, to learn algorithmic reasoning that can generate "how-to-solve" hints for unseen problems.
Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community
The ability to interact with machines using natural human language is becoming not just commonplace, but expected.
When is Tree Search Useful for LLM Planning? It Depends on the Discriminator
In this paper, we examine how large language models (LLMs) solve multi-step problems under a language agent framework with three components: a generator, a discriminator, and a planning method.
Sparse Meets Dense: A Hybrid Approach to Enhance Scientific Document Retrieval
Traditional information retrieval is based on sparse bag-of-words vector representations of documents and queries.
What is the Best Automated Metric for Text to Motion Generation?
Our findings indicate that none of the metrics currently used for this task show even a moderate correlation with human judgments on a sample level.
Explaining Competitive-Level Programming Solutions using LLMs
In this paper, we approach competitive-level programming problem-solving as a composite task of reasoning and code generation.
Text-to-SQL Error Correction with Language Models of Code
Thus, we propose a novel representation for SQL queries and their edits that adheres more closely to the pre-training corpora of language models of code.
TellMeWhy: A Dataset for Answering Why-Questions in Narratives
They are especially worse on questions whose answers are external to the narrative, thus providing a challenge for future QA and narrative understanding research.
Stacking with Auxiliary Features for Visual Question Answering
We propose four categories of auxiliary features for ensembling for VQA.
Improving Black-box Speech Recognition using Semantic Parsing
Speech is a natural channel for human-computer interaction in robotics and consumer applications.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Guiding Interaction Behaviors for Multi-modal Grounded Language Learning
Multi-modal grounded language learning connects language predicates to physical properties of objects in the world.
Captioning Images with Diverse Objects
We propose minimizing a joint objective which can learn from these diverse data sources and leverage distributional semantic embeddings, enabling the model to generalize and describe novel objects outside of image-caption datasets.
Improving LSTM-based Video Description with Linguistic Knowledge Mined from Text
This paper investigates how linguistic knowledge mined from large text corpora can aid the generation of natural language descriptions of videos.
Sequence to Sequence - Video to Text
Our LSTM model is trained on video-sentence pairs and learns to associate a sequence of video frames to a sequence of words in order to generate a description of the event in the video clip.
Deep Compositional Captioning: Describing Novel Object Categories without Paired Training Data
Current deep caption models can only describe objects contained in paired image-sentence corpora, despite the fact that they are pre-trained with large object recognition datasets, namely ImageNet.
Sequence to Sequence -- Video to Text
Our LSTM model is trained on video-sentence pairs and learns to associate a sequence of video frames to a sequence of words in order to generate a description of the event in the video clip.
Translating Videos to Natural Language Using Deep Recurrent Neural Networks
Solving the visual symbol grounding problem has long been a goal of artificial intelligence.
