Search Results for author:
Found 37 papers, 10 papers with code
Building Goal-oriented Document-grounded Dialogue Systems
In this paper, we describe our systems for solving the two Doc2Dial shared task: knowledge identification and response generation.
Diverse and Admissible Trajectory Prediction through Multimodal Context Understanding
Multi-agent trajectory forecasting in autonomous driving requires an agent to accurately anticipate the behaviors of the surrounding vehicles and pedestrians, for safe and reliable decision-making.
Your Learned Constraint is Secretly a Backward Reachable Tube
In the language of safe control, this means we recover a backwards reachable tube (BRT) rather than a failure set.
STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learning
Many robot learning methods treat such datasets as multi-task expert data and learn a multi-task, generalist policy by training broadly across them.
GraphEQA: Using 3D Semantic Scene Graphs for Real-time Embodied Question Answering
In Embodied Question Answering (EQA), agents must explore and develop a semantic understanding of an unseen environment in order to answer a situated question with confidence.
Object and Contact Point Tracking in Demonstrations Using 3D Gaussian Splatting
This paper introduces a method to enhance Interactive Imitation Learning (IIL) by extracting touch interaction points and tracking object movement from video demonstrations.
Grounding Robot Policies with Visuomotor Language Guidance
Recent advances in the fields of natural language processing and computer vision have shown great potential in understanding the underlying dynamics of the world from large-scale internet data.
SEAL: Towards Safe Autonomous Driving via Skill-Enabled Adversary Learning for Closed-Loop Scenario Generation
Verification and validation of autonomous driving (AD) systems and components is of increasing importance, as such technology increases in real-world prevalence.
Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making
Resolving the dichotomy between the human-like yet constrained reasoning processes of Cognitive Architectures and the broad but often noisy inference behavior of Large Language Models (LLMs) remains a challenging but exciting pursuit, for enabling reliable machine reasoning capabilities in production systems.
Towards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challenge
In order to develop robots that can effectively serve as versatile and capable home assistants, it is crucial for them to reliably perceive and interact with a wide variety of objects across diverse environments.
Enhancing Vision-Language Models with Scene Graphs for Traffic Accident Understanding
This representation of a traffic scene is referred to as a scene graph, and can be used as input for an accident classifier.
Attribution Regularization for Multimodal Paradigms
This research project aims to address these challenges by proposing a novel regularization term that encourages multimodal models to effectively utilize information from all modalities when making decisions.
Semantic Augmentation in Images using Language
Deep Learning models are incredibly data-hungry and require very large labeled datasets for supervised learning.
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
Motivated by the impressive open-set performance and content generation capabilities of web-scale, large-capacity pre-trained models (i. e., foundation models) in research fields such as Natural Language Processing (NLP) and Computer Vision (CV), we devote this survey to exploring (i) how these existing foundation models from NLP and CV can be applied to the field of general-purpose robotics, and also exploring (ii) what a robotics-specific foundation model would look like.
A Study of Situational Reasoning for Traffic Understanding
Intelligent Traffic Monitoring (ITMo) technologies hold the potential for improving road safety/security and for enabling smart city infrastructure.
Knowledge-enhanced Agents for Interactive Text Games
We experiment with four models on the 10 tasks in the ScienceWorld text-based game environment, to illustrate the impact of knowledge injection on various model configurations and challenging task settings.
Regularizing Self-training for Unsupervised Domain Adaptation via Structural Constraints
Self-training based on pseudo-labels has emerged as a dominant approach for addressing conditional distribution shifts in unsupervised domain adaptation (UDA) for semantic segmentation problems.
Core Challenges in Embodied Vision-Language Planning
Recent advances in the areas of Multimodal Machine Learning and Artificial Intelligence (AI) have led to the development of challenging tasks at the intersection of Computer Vision, Natural Language Processing, and Robotics.
Knowledge-driven Scene Priors for Semantic Audio-Visual Embodied Navigation
We also define a new audio-visual navigation sub-task, where agents are evaluated on novel sounding objects, as opposed to unheard clips of known objects.
Distribution-aware Goal Prediction and Conformant Model-based Planning for Safe Autonomous Driving
The feasibility of collecting a large amount of expert demonstrations has inspired growing research interests in learning-to-drive settings, where models learn by imitating the driving behaviour from experts.
Utilizing Background Knowledge for Robust Reasoning over Traffic Situations
Understanding novel situations in the traffic domain requires an intricate combination of domain-specific and causal commonsense knowledge.
Coalescing Global and Local Information for Procedural Text Understanding
In this paper, we propose Coalescing Global and Local Information (CGLI), a new model that builds entity- and timestep-aware input representations (local input) considering the whole context (global input), and we jointly model the entity states with a structured prediction objective (global output).
An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs
In this paper, we study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models.
Learn-to-Race Challenge 2022: Benchmarking Safe Learning and Cross-domain Generalisation in Autonomous Racing
In the first stage of the challenge, we evaluate an autonomous agent's ability to drive as fast as possible, while adhering to safety constraints.
Generalizable Neuro-symbolic Systems for Commonsense Question Answering
This chapter illustrates how suitable neuro-symbolic models for language understanding can enable domain generalizability and robustness in downstream tasks.
Safe Autonomous Racing via Approximate Reachability on Ego-vision
Given the nature of the task, autonomous agents need to be able to 1) identify and avoid unsafe scenarios under the complex vehicle dynamics, and 2) make sub-second decision in a fast-changing environment.
Unsupervised Domain Adaptation Via Pseudo-labels And Objectness Constraints
Crucially, the objectness constraint is agnostic to the ground-truth semantic segmentation labels and, therefore, remains appropriate for unsupervised adaptation settings.
Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models
In this paper, we investigate what models learn from commonsense reasoning datasets.
Core Challenges in Embodied Vision-Language Planning
Recent advances in the areas of multimodal machine learning and artificial intelligence (AI) have led to the development of challenging tasks at the intersection of Computer Vision, Natural Language Processing, and Embodied AI.
Learn-to-Race: A Multimodal Control Environment for Autonomous Racing
Existing research on autonomous driving primarily focuses on urban driving, which is insufficient for characterising the complex driving behaviour underlying high-speed racing.
Lexically-constrained Text Generation through Commonsense Knowledge Extraction and Injection
Conditional text generation has been a challenging task that is yet to see human-level performance from state-of-the-art models.
Trajformer: Trajectory Prediction with Local Self-Attentive Contexts for Autonomous Driving
Effective feature-extraction is critical to models' contextual understanding, particularly for applications to robotics and autonomous driving, such as multimodal trajectory prediction.
Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering
Guided by a set of hypotheses, the framework studies how to transform various pre-existing knowledge resources into a form that is most effective for pre-training models.
Neuro-symbolic Architectures for Context Understanding
Computational context understanding refers to an agent's ability to fuse disparate sources of information for decision-making and is, therefore, generally regarded as a prerequisite for sophisticated machine reasoning capabilities, such as in artificial intelligence (AI).
Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding
Multi-agent trajectory forecasting in autonomous driving requires an agent to accurately anticipate the behaviors of the surrounding vehicles and pedestrians, for safe and reliable decision-making.
Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering
Non-extractive commonsense QA remains a challenging AI task, as it requires systems to reason about, synthesize, and gather disparate pieces of information, in order to generate responses to queries.
Ranked #16 on
Common Sense Reasoning
on CommonsenseQA
How Would You Say It? Eliciting Lexically Diverse Dialogue for Supervised Semantic Parsing
Wang et al. (2015) proposed a method to build semantic parsing datasets by generating canonical utterances using a grammar and having crowdworkers paraphrase them into natural wording.
