Search Results for author:
Found 13 papers, 5 papers with code
OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge
In this paper, we address the task of knowledge-based visual question answering and provide a benchmark, called OK-VQA, where the image content is not sufficient to answer the questions, encouraging methods that rely on external knowledge resources.
Same Object, Different Grasps: Data and Semantic Knowledge for Task-Oriented Grasping
This is largely due to the scale of the datasets both in terms of the number of objects and tasks studied.
A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge
In contrast to the existing knowledge-based VQA datasets, the questions generally cannot be answered by simply querying a knowledge base, and instead require some form of commonsense reasoning about the scene depicted in the image.
Ask Your Humans: Using Human Instructions to Improve Generalization in Reinforcement Learning
We also find that incorporating natural language allows the model to generalize to unseen tasks in a zero-shot setting and to learn quickly from a few demonstrations.
The Pose Knows: Video Forecasting by Generating Pose Futures
First we explicitly model the high level structure of active objects in the scene---humans---and use a VAE to model the possible future movements of humans in the pose space.
Ranked #2 on
Human Pose Forecasting
on Human3.6M
(CMD metric)
The More You Know: Using Knowledge Graphs for Image Classification
One characteristic that sets humans apart from modern learning-based computer vision algorithms is the ability to acquire knowledge about the world and use that knowledge to reason about the visual world.
Hierarchical RL Using an Ensemble of Proprioceptive Periodic Policies
The high-level policy is trained using a sparse, task-dependent reward, and operates by choosing which of the low-level policies to run at any given time.
Empirically Verifying Hypotheses Using Reinforcement Learning
This paper formulates hypothesis verification as an RL problem.
KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA
One of the most challenging question types in VQA is when answering the question requires outside knowledge not present in the image.
Ranked #6 on
Visual Question Answering (VQA)
on A-OKVQA
Agent as Scientist: Learning to Verify Hypotheses
In order to train the agents, we exploit the underlying structure in the majority of hypotheses -- they can be formulated as triplets (pre-condition, action sequence, post-condition).
Learning to Navigate Wikipedia by Taking Random Walks
A fundamental ability of an intelligent web-based agent is seeking out and acquiring new information.
Distilling Internet-Scale Vision-Language Models into Embodied Agents
Instruction-following agents must ground language into their observation and action spaces.
Collaborating with language models for embodied reasoning
On the other hand, Large Scale Language Models (LSLMs) have exhibited strong reasoning ability and the ability to to adapt to new tasks through in-context learning.
