Search Results for author:
Found 18 papers, 7 papers with code
Ferret-UI 2: Mastering Universal User Interface Understanding Across Platforms
Building a generalist model for user interface (UI) understanding is challenging due to various foundational issues, such as platform diversity, resolution variation, and data limitation.
Grounding Multimodal Large Language Models in Actions
For discrete actions, we demonstrate that semantically aligning these actions with the native output token space of the MLLM leads to the strongest performance.
Large Language Models as Generalizable Policies for Embodied Tasks
We show that large language models (LLMs) can be adapted to be generalizable policies for embodied visual tasks.
Housekeep: Tidying Virtual Households using Commonsense Reasoning
Instead, the agent must learn from and is evaluated against human preferences of which objects belong where in a tidy house.
Simple and Effective Synthesis of Indoor 3D Scenes
We study the problem of synthesizing immersive 3D indoor scenes from one or more images.
SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigation
Natural language instructions for visual navigation often use scene descriptions (e. g., "bedroom") and object references (e. g., "green chairs") to provide a breadcrumb trail to a goal location.
The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation
It is fundamental for personal robots to reliably navigate to a specified goal.
Contrast and Classify: Training Robust VQA Models
Recent Visual Question Answering (VQA) models have shown impressive performance on the VQA benchmark but remain sensitive to small linguistic variations in input questions.
Spatially Aware Multimodal Transformers for TextVQA
Further, each head in our multi-head self-attention layer focuses on a different subset of relations.
Sequential Latent Spaces for Modeling the Intention During Diverse Image Captioning
We encourage this temporal latent space to capture the 'intention' about how to complete the sentence by mimicking a representation which summarizes the future.
EvalAI: Towards Better Evaluation Systems for AI Agents
We introduce EvalAI, an open source platform for evaluating and comparing machine learning (ML) and artificial intelligence algorithms (AI) at scale.
nocaps: novel object captioning at scale
To encourage the development of image captioning models that can learn visual concepts from alternative data sources, such as object detection datasets, we present the first large-scale benchmark for this task.
Fabrik: An Online Collaborative Neural Network Editor
We present Fabrik, an online neural network editor that provides tools to visualize, edit, and share neural networks from within a browser.
Sort Story: Sorting Jumbled Images and Captions into Stories
Temporal common sense has applications in AI tasks such as QA, multi-document summarization, and human-AI communication.
Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?
We conduct large-scale studies on `human attention' in Visual Question Answering (VQA) to understand where humans choose to look to answer questions about images.
Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?
We conduct large-scale studies on `human attention' in Visual Question Answering (VQA) to understand where humans choose to look to answer questions about images.
CloudCV: Large Scale Distributed Computer Vision as a Cloud Service
We are witnessing a proliferation of massive visual data.
Object-Proposal Evaluation Protocol is 'Gameable'
Finally, we plan to release an easy-to-use toolbox which combines various publicly available implementations of object proposal algorithms which standardizes the proposal generation and evaluation so that new methods can be added and evaluated on different datasets.
