Search Results for author:
Found 53 papers, 21 papers with code
Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation
In this work, we introduce Sigma, a Siamese Mamba network for multi-modal semantic segmentation, utilizing the Selective Structured State Space Model, Mamba.
ShapeGrasp: Zero-Shot Task-Oriented Grasping with Large Language Models through Geometric Decomposition
Task-oriented grasping of unfamiliar objects is a necessary skill for robots in dynamic in-home environments.
CBGT-Net: A Neuromimetic Architecture for Robust Classification of Streaming Data
This paper describes CBGT-Net, a neural network model inspired by the cortico-basal ganglia-thalamic (CBGT) circuits found in mammalian brains.
Negative Yields Positive: Unified Dual-Path Adapter for Vision-Language Models
Recently, large-scale pre-trained Vision-Language Models (VLMs) have demonstrated great potential in learning open-world visual representations, and exhibit remarkable performance across a wide range of downstream tasks through efficient fine-tuning.
HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation
Being able to understand visual scenes is a precursor for many downstream tasks, including autonomous driving, robotics, and other vision-based approaches.
Optimal Task Assignment and Path Planning using Conflict-Based Search with Precedence and Temporal Constraints
To incorporate task assignment, path planning, and a user-defined objective into a coherent framework, this paper examines the Task Assignment and Path Finding with Precedence and Temporal Constraints (TAPF-PTC) problem.
WIT-UAS: A Wildland-fire Infrared Thermal Dataset to Detect Crew Assets From Aerial Views
We present the Wildland-fire Infrared Thermal (WIT-UAS) dataset for long-wave infrared sensing of crew and vehicle assets amidst prescribed wildland fire environments.
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
Motivated by the impressive open-set performance and content generation capabilities of web-scale, large-capacity pre-trained models (i. e., foundation models) in research fields such as Natural Language Processing (NLP) and Computer Vision (CV), we devote this survey to exploring (i) how these existing foundation models from NLP and CV can be applied to the field of robotics, and also exploring (ii) what a robotics-specific foundation model would look like.
Personalized Decision Supports based on Theory of Mind Modeling and Explainable Reinforcement Learning
Our proposed approach is agnostic to task environment and RL model structure, therefore has the potential to be generalized to a wide range of applications.
Benchmarking and Enhancing Disentanglement in Concept-Residual Models
Concept bottleneck models (CBMs) are interpretable models that first predict a set of semantically meaningful features, i. e., concepts, from observations that are subsequently used to condition a downstream task.
Understanding Your Agent: Leveraging Large Language Models for Behavior Explanation
Intelligent agents such as robots are increasingly deployed in real-world, safety-critical settings.
Long-Horizon Dialogue Understanding for Role Identification in the Game of Avalon with Large Language Models
We discuss the capabilities of LLMs to utilize deceptive long-horizon conversations between six human players to determine each player's goal and motivation.
Theory of Mind for Multi-Agent Collaboration via Large Language Models
While Large Language Models (LLMs) have demonstrated impressive accomplishments in both reasoning and planning, their abilities in multi-agent collaborations remains largely unexplored.
Explaining Agent Behavior with Large Language Models
Intelligent agents such as robots are increasingly deployed in real-world, safety-critical settings.
Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videos
This work focuses on anticipating long-term human actions, particularly using short video segments, which can speed up editing workflows through improved suggestions while fostering creativity by suggesting narratives.
Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learning
The ability to model the mental states of others is crucial to human social intelligence, and can offer similar benefits to artificial agents with respect to the social dynamics induced in multi-agent settings.
Solving Multi-Agent Target Assignment and Path Finding with a Single Constraint Tree
As a leading approach to address TAPF, Conflict-Based Search with Target Assignment (CBS-TA) leverages both K-best target assignments to create multiple search trees and Conflict-Based Search (CBS) to resolve collisions in each search tree.
Introspective Action Advising for Interpretable Transfer Learning
Transfer learning can be applied in deep reinforcement learning to accelerate the training of a policy in a target task by transferring knowledge from a policy learned in a related source task.
Sample-Efficient Learning of Novel Visual Concepts
Despite the advances made in visual object recognition, state-of-the-art deep learning models struggle to effectively recognize novel objects in a few-shot setting where only a limited number of examples are provided.
On the Role of Emergent Communication for Social Learning in Multi-Agent Reinforcement Learning
Explicit communication among humans is key to coordinating and learning.
Concept Learning for Interpretable Multi-Agent Reinforcement Learning
Multi-agent robotic systems are increasingly operating in real-world environments in close proximity to humans, yet are largely controlled by policy models with inscrutable deep neural network representations.
Predicting Out-of-Distribution Error with Confidence Optimal Transport
Out-of-distribution (OOD) data poses serious challenges in deployed machine learning models as even subtle changes could incur significant performance drops.
Towards True Lossless Sparse Communication in Multi-Agent Systems
We evaluate the learned communication `language' through direct causal analysis of messages in non-sparse runs to determine the range of lossless sparse budgets, which allow zero-shot sparsity, and the range of sparse budgets that will inquire a reward loss, which is minimized by our learned gating function with few-shot sparsity.
Explainable Action Advising for Multi-Agent Reinforcement Learning
This allows the student to self-reflect on what it has learned, enabling advice generalization and leading to improved sample efficiency and learning performance - even in environments where the teacher is sub-optimal.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
ARC -- Actor Residual Critic for Adversarial Imitation Learning
In AIL, an artificial adversary's misclassification is used as a reward signal that is optimized by any standard Reinforcement Learning (RL) algorithm.
Probe-Based Interventions for Modifying Agent Behavior
Neural nets are powerful function approximators, but the behavior of a given neural net, once trained, cannot be easily modified.
Interpretable Learned Emergent Communication for Human-Agent Teams
In human-agent teams tested in benchmark environments, where agents have been modeled using the Enforcers, we find that a prototype-based method produces meaningful discrete tokens that enable human partners to learn agent communication faster and better than a one-hot baseline.
Emergent Discrete Communication in Semantic Spaces
Neural agents trained in reinforcement learning settings can learn to communicate among themselves via discrete tokens, accomplishing as a team what agents would be unable to do alone.
Learning Embeddings that Capture Spatial Semantics for Indoor Navigation
We know that humans can search for an object like a book, or a plate in an unseen house, based on the spatial semantics of bigger objects detected.
Deep Interpretable Models of Theory of Mind
When developing AI systems that interact with humans, it is essential to design both a system that can understand humans, and a system that humans can understand.
Adaptive Agent Architecture for Real-time Human-Agent Teaming
Previous human-human team research have shown complementary policies in TSF game and diversity in human players' skill, which encourages us to relax the assumptions on human policy.
Predicting Human Strategies in Simulated Search and Rescue Task
In our efforts to model the rescuer's mind, we begin with a simple simulated search and rescue task in Minecraft with human participants.
Addressing reward bias in Adversarial Imitation Learning with neutral reward functions
Generative Adversarial Imitation Learning suffers from the fundamental problem of reward bias stemming from the choice of reward functions used in the algorithm.
Explanation of Reinforcement Learning Model in Dynamic Multi-Agent System
The results show that verbal explanation generated by both models improve subjective satisfaction of users towards the interpretability of DRL systems.
Natural Emergence of Heterogeneous Strategies in Artificially Intelligent Competitive Teams
We corroborate that modeling agents with Graph Neural Networks and training them with Reinforcement Learning leads to the evolution of increasingly complex strategies for each team.
MA 3 : Model Agnostic Adversarial Augmentation for Few Shot learning
Despite the recent developments in vision-related problems using deep neural networks, there still remains a wide scope in the improvement of generalizing these models to unseen examples.
Augmenting GAIL with BC for sample efficient imitation learning
Behavior cloning and GAIL are two widely used methods for performing imitation learning.
Simultaneous Identification of Tweet Purpose and Position
Tweet classification has attracted considerable attention recently.
An Unsupervised Domain-Independent Framework for Automated Detection of Persuasion Tactics in Text
With the increasing growth of social media, people have started relying heavily on the information shared therein to form opinions and make decisions.
Event Outcome Prediction using Sentiment Analysis and Crowd Wisdom in Microblog Feeds
In this work, we investigate whether crowd wisdom is useful in predicting such outcomes and whether their opinions are influenced by the experts in the field.
MAME : Model-Agnostic Meta-Exploration
Meta-Reinforcement learning approaches aim to develop learning procedures that can adapt quickly to a distribution of tasks with the help of a few examples.
Learning Transferable Cooperative Behavior in Multi-Agent Teams
While multi-agent interactions can be naturally modeled as a graph, the environment has traditionally been considered as a black box.
Active Learning with Gaussian Processes for High Throughput Phenotyping
A looming question that must be solved before robotic plant phenotyping capabilities can have significant impact to crop improvement programs is scalability.
Better Safe than Sorry: Evidence Accumulation Allows for Safe Reinforcement Learning
The agent makes no decision by default, and the burden of proof to make a decision falls on the policy to accrue evidence strongly in favor of a single decision.
Object-sensitive Deep Reinforcement Learning
In this paper, we propose a novel method that can incorporate object recognition processing to deep reinforcement learning models.
Transparency and Explanation in Deep Reinforcement Learning Neural Networks
For those systems to be accepted and trusted, the users should be able to understand the reasoning process of the system, i. e. the system should be transparent.
Towards Better Interpretability in Deep Q-Networks
Deep reinforcement learning techniques have demonstrated superior performance in a wide variety of environments.
Challenges of Context and Time in Reinforcement Learning: Introducing Space Fortress as a Benchmark
Research in deep reinforcement learning (RL) has coalesced around improving performance on benchmarks like the Arcade Learning Environment.
Ranked #1 on
Space Fortress
on Autoturn
Community Regularization of Visually-Grounded Dialog
The task of conducting visually grounded dialog involves learning goal-oriented cooperative dialog between autonomous agents who exchange information about a scene through several rounds of questions and answers in natural language.
Combining Model-Free Q-Ensembles and Model-Based Approaches for Informed Exploration
Q-Ensembles are a model-free approach where input images are fed into different Q-networks and exploration is driven by the assumption that uncertainty is proportional to the variance of the output Q-values obtained.
Learning Time-Sensitive Strategies in Space Fortress
Although there has been remarkable progress and impressive performance on reinforcement learning (RL) on Atari games, there are many problems with challenging characteristics that have not yet been explored in Deep Learning for RL.
Joint Embedding of Hierarchical Categories and Entities for Concept Categorization and Dataless Classification
Due to the lack of structured knowledge applied in learning distributed representation of cate- gories, existing work cannot incorporate category hierarchies into entity information.
Joint Embeddings of Hierarchical Categories and Entities
Due to the lack of structured knowledge applied in learning distributed representation of categories, existing work cannot incorporate category hierarchies into entity information.~We propose a framework that embeds entities and categories into a semantic space by integrating structured knowledge and taxonomy hierarchy from large knowledge bases.
