Search Results for author:
Found 17 papers, 9 papers with code
Feedback Networks
Currently, the most successful learning models in computer vision are based on learning successive representations followed by a decision layer.
Demo2Vec: Reasoning Object Affordances From Online Videos
Watching expert demonstrations is an important way for humans and robots to reason about affordances of unseen objects.
Ranked #2 on
Video-to-image Affordance Grounding
on OPRA (28x28)
Learning from Noisy Demonstration Sets via Meta-Learned Suitability Assessor
A noisy and diverse demonstration set may hinder the performances of an agent aiming to acquire certain skills via imitation learning.
Program Guided Agent
Developing agents that can learn to follow natural language instructions has been an emerging research direction.
MELINDA: A Multimodal Dataset for Biomedical Experiment Method Classification
We introduce a new dataset, MELINDA, for Multimodal biomEdicaL experImeNt methoD clAssification.
LAMPRET: Layout-Aware Multimodal PreTraining for Document Understanding
text, table, image) and propose a novel layout-aware multimodal hierarchical framework, LAMPreT, to model the blocks and the whole document.
COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentences
To this end, we introduce a new commonsense reasoning benchmark dataset comprising natural language true/false statements, with each sample paired with its complementary counterpart, resulting in 4k sentence pairs.
HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning
Taxonomies are valuable resources for many applications, but the limited coverage due to the expensive manual curation process hinders their general applicability.
Understanding Multimodal Procedural Knowledge by Sequencing Multimodal Instructional Manuals
The ability to sequence unordered events is an essential skill to comprehend and reason about real world task procedures, which often requires thorough understanding of temporal common sense and multimodal information, as these procedures are often communicated through a combination of texts and images.
Learning Action Conditions from Instructional Manuals for Instruction Understanding
The ability to infer pre- and postconditions of an action is vital for comprehending complex instructions, and is essential for applications such as autonomous instruction-guided agents and assistive AI that supports humans to perform physical tasks.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Character-Centric Story Visualization via Visual Planning and Token Alignment
This task requires machines to 1) understand long text inputs and 2) produce a globally consistent image sequence that illustrates the contents of the story.
InSpaceType: Reconsider Space Type in Indoor Monocular Depth Estimation
To facilitate our investigation for robustness and address limitations of previous works, we collect InSpaceType, a high-quality and high-resolution RGBD dataset for general indoor environments.
Indoor Monocular Depth Estimation
Monocular Depth Estimation
Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge
The ability to actively ground task instructions from an egocentric view is crucial for AI agents to accomplish tasks or assist humans virtually.
ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videos
Multimodal counterfactual reasoning is a vital yet challenging ability for AI systems.
Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty
In this paper, we aim to tackle this problem with a unified framework consisting of an end-to-end trainable method and a planning algorithm.
DACO: Towards Application-Driven and Comprehensive Data Analysis via Code Generation
We construct the DACO dataset, containing (1) 440 databases (of tabular data) collected from real-world scenarios, (2) ~2k query-answer pairs that can serve as weak supervision for model training, and (3) a concentrated but high-quality test set with human refined annotations that serves as our main evaluation benchmark.
