Search Results for author:
Found 27 papers, 11 papers with code
Retouchdown: Releasing Touchdown on StreetLearn as a Public Resource for Language Grounding Tasks in Street View
These have been added to the StreetLearn dataset and can be obtained via the same process as used previously for StreetLearn.
Multi-Level Gazetteer-Free Geocoding
We present a multi-level geocoding model (MLG) that learns to associate texts to geographic coordinates.
Factored Agents: Decoupling In-Context Learning and Memorization for Robust Tool Use
In this paper, we propose a novel factored agent architecture designed to overcome the limitations of traditional single-agent systems in agentic AI.
Project MPG: towards a generalized performance benchmark for LLM capabilities
There exists an extremely wide array of LLM benchmarking tasks, whereas oftentimes a single number is the most actionable for decision-making, especially by non-experts.
Improving Multi-Agent Debate with Sparse Communication Topology
Multi-agent debate has proven effective in improving large language models quality for reasoning and factuality tasks.
Pedestrian Crossing Action Recognition and Trajectory Prediction with 3D Human Keypoints
Accurate understanding and prediction of human behaviors are critical prerequisites for autonomous vehicles, especially in highly dynamic and interactive scenarios such as intersections in dense urban areas.
RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystems
The development of recommender systems that optimize multi-turn interaction with users, and model the interactions of different agents (e. g., users, content providers, vendors) in the recommender ecosystem have drawn increasing attention in recent years.
On the Evaluation of Vision-and-Language Navigation Instructions
Vision-and-Language Navigation wayfinding agents can be enhanced by exploiting automatically generated navigation instructions.
A Hierarchical Multi-Modal Encoder for Moment Localization in Video Corpus
Identifying a short segment in a long video that semantically matches a text query is a challenging task that has important application potentials in language-based video search, browsing, and navigation.
AQuaMuSe: Automatically Generating Datasets for Query-Based Multi-Document Summarization
Summarization is the task of compressing source document(s) into coherent and succinct passages.
Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding
We introduce Room-Across-Room (RxR), a new Vision-and-Language Navigation (VLN) dataset.
Ranked #5 on
Vision and Language Navigation
on RxR
Learning to Represent Image and Text with Denotation Graph
Recent progresses have leveraged the ideas of pre-training (from language modeling) and attention layers in Transformers to learn representation from datasets containing images aligned with linguistic expressions that describe the images.
Spatial Language Representation with Multi-Level Geocoding
We present a multi-level geocoding model (MLG) that learns to associate texts to geographic locations.
Mean-Field Approximation to Gaussian-Softmax Integral with Application to Uncertainty Estimation
Many methods have been proposed to quantify the predictive uncertainty associated with the outputs of deep neural networks.
BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps
To this end, we propose BabyWalk, a new VLN agent that is learned to navigate by decomposing long instructions into shorter ones (BabySteps) and completing them sequentially.
Environment-agnostic Multitask Learning for Natural Language Grounded Navigation
Recent research efforts enable study for natural language grounded navigation in photo-realistic environments, e. g., following natural language instructions or dialog.
Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View
These have been added to the StreetLearn dataset and can be obtained via the same process as used previously for StreetLearn.
Ranked #8 on
Vision and Language Navigation
on Touchdown Dataset
VALAN: Vision and Language Agent Navigation
VALAN is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture.
Generalized Natural Language Grounded Navigation via Environment-agnostic Multitask Learning
Recent research efforts enable study for natural language grounded navigation in photo-realistic environments, e. g., following natural language instructions or dialog.
Learning Dense Representations for Entity Retrieval
We show that it is feasible to perform entity linking by training a dual encoder (two-tower) model that encodes mentions and entities in the same dense vector space, where candidate entities are retrieved by approximate nearest neighbor search.
RecSim: A Configurable Simulation Platform for Recommender Systems
We propose RecSim, a configurable platform for authoring simulation environments for recommender systems (RSs) that naturally supports sequential interaction with users.
Transferable Representation Learning in Vision-and-Language Navigation
Vision-and-Language Navigation (VLN) tasks such as Room-to-Room (R2R) require machine agents to interpret natural language instructions and learn to act in visually realistic environments to achieve navigation goals.
Ranked #115 on
Vision and Language Navigation
on VLN Challenge
General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warping
We address fundamental flaws in previously used metrics and show how Dynamic Time Warping (DTW), a long known method of measuring similarity between two time series, can be used for evaluation of navigation agents.
Multi-modal Discriminative Model for Vision-and-Language Navigation
Vision-and-Language Navigation (VLN) is a natural language grounding task where agents have to interpret natural language instructions in the context of visual scenes in a dynamic environment to achieve prescribed navigation goals.
Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation
We also show that the existing paths in the dataset are not ideal for evaluating instruction following because they are direct-to-goal shortest paths.
Reinforcement Learning for Slate-based Recommender Systems: A Tractable Decomposition and Practical Methodology
(i) We develop SLATEQ, a decomposition of value-based temporal-difference and Q-learning that renders RL tractable with slates.
Using Web Co-occurrence Statistics for Improving Image Categorization
Albeit the simplicity of the resulting optimization problem, it is effective in improving both recognition and localization accuracy.
