Search Results for author: Simon Stepputtis

Found 20 papers, 6 papers with code

Negative Yields Positive: Unified Dual-Path Adapter for Vision-Language Models

1 code implementation19 Mar 2024 Ce Zhang, Simon Stepputtis, Katia Sycara, Yaqi Xie

Recently, large-scale pre-trained Vision-Language Models (VLMs) have demonstrated great potential in learning open-world visual representations, and exhibit remarkable performance across a wide range of downstream tasks through efficient fine-tuning.

Computational Efficiency Domain Generalization +1

HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation

1 code implementation18 Mar 2024 Ce Zhang, Simon Stepputtis, Joseph Campbell, Katia Sycara, Yaqi Xie

Being able to understand visual scenes is a precursor for many downstream tasks, including autonomous driving, robotics, and other vision-based approaches.

Scene Graph Generation

Benchmarking and Enhancing Disentanglement in Concept-Residual Models

no code implementations30 Nov 2023 Renos Zabounidis, Ini Oguntola, Konghao Zhao, Joseph Campbell, Simon Stepputtis, Katia Sycara

Concept bottleneck models (CBMs) are interpretable models that first predict a set of semantically meaningful features, i. e., concepts, from observations that are subsequently used to condition a downstream task.

Benchmarking Disentanglement

Theory of Mind for Multi-Agent Collaboration via Large Language Models

no code implementations16 Oct 2023 Huao Li, Yu Quan Chong, Simon Stepputtis, Joseph Campbell, Dana Hughes, Michael Lewis, Katia Sycara

While Large Language Models (LLMs) have demonstrated impressive accomplishments in both reasoning and planning, their abilities in multi-agent collaborations remains largely unexplored.

Hallucination Multi-agent Reinforcement Learning

Explaining Agent Behavior with Large Language Models

no code implementations19 Sep 2023 Xijia Zhang, Yue Guo, Simon Stepputtis, Katia Sycara, Joseph Campbell

Intelligent agents such as robots are increasingly deployed in real-world, safety-critical settings.

counterfactual Hallucination +2

Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videos

no code implementations12 Sep 2023 Sarthak Bhagat, Simon Stepputtis, Joseph Campbell, Katia Sycara

This work focuses on anticipating long-term human actions, particularly using short video segments, which can speed up editing workflows through improved suggestions while fostering creativity by suggesting narratives.

Action Anticipation Long Term Action Anticipation

Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learning

no code implementations3 Jul 2023 Ini Oguntola, Joseph Campbell, Simon Stepputtis, Katia Sycara

The ability to model the mental states of others is crucial to human social intelligence, and can offer similar benefits to artificial agents with respect to the social dynamics induced in multi-agent settings.

Multi-agent Reinforcement Learning reinforcement-learning

Introspective Action Advising for Interpretable Transfer Learning

no code implementations21 Jun 2023 Joseph Campbell, Yue Guo, Fiona Xie, Simon Stepputtis, Katia Sycara

Transfer learning can be applied in deep reinforcement learning to accelerate the training of a policy in a target task by transferring knowledge from a policy learned in a related source task.

Transfer Learning

Sample-Efficient Learning of Novel Visual Concepts

1 code implementation15 Jun 2023 Sarthak Bhagat, Simon Stepputtis, Joseph Campbell, Katia Sycara

Despite the advances made in visual object recognition, state-of-the-art deep learning models struggle to effectively recognize novel objects in a few-shot setting where only a limited number of examples are provided.

Multi-Label Classification Object Recognition

Concept Learning for Interpretable Multi-Agent Reinforcement Learning

no code implementations23 Feb 2023 Renos Zabounidis, Joseph Campbell, Simon Stepputtis, Dana Hughes, Katia Sycara

Multi-agent robotic systems are increasingly operating in real-world environments in close proximity to humans, yet are largely controlled by policy models with inscrutable deep neural network representations.

Decision Making Multi-agent Reinforcement Learning +2

Explainable Action Advising for Multi-Agent Reinforcement Learning

1 code implementation15 Nov 2022 Yue Guo, Joseph Campbell, Simon Stepputtis, Ruiyu Li, Dana Hughes, Fei Fang, Katia Sycara

This allows the student to self-reflect on what it has learned, enabling advice generalization and leading to improved sample efficiency and learning performance - even in environments where the teacher is sub-optimal.

Multi-agent Reinforcement Learning reinforcement-learning +2

Imitation Learning of Robot Policies by Combining Language, Vision and Demonstration

no code implementations26 Nov 2019 Simon Stepputtis, Joseph Campbell, Mariano Phielipp, Chitta Baral, Heni Ben Amor

In this work we propose a novel end-to-end imitation learning approach which combines natural language, vision, and motion information to produce an abstract representation of a task, which in turn is used to synthesize specific motion controllers at run-time.

Imitation Learning

Improved Exploration through Latent Trajectory Optimization in Deep Deterministic Policy Gradient

no code implementations15 Nov 2019 Kevin Sebastian Luck, Mel Vecerik, Simon Stepputtis, Heni Ben Amor, Jonathan Scholz

This work evaluates the use of model-based trajectory optimization methods used for exploration in Deep Deterministic Policy Gradient when trained on a latent image embedding.

Continuous Control reinforcement-learning +1

Imitation Learning of Robot Policies using Language, Vision and Motion

no code implementations25 Sep 2019 Simon Stepputtis, Joseph Campbell, Mariano Phielipp, Chitta Baral, Heni Ben Amor

In this work we propose a novel end-to-end imitation learning approach which combines natural language, vision, and motion information to produce an abstract representation of a task, which in turn can be used to synthesize specific motion controllers at run-time.

Imitation Learning

Learning Interactive Behaviors for Musculoskeletal Robots Using Bayesian Interaction Primitives

no code implementations15 Aug 2019 Joseph Campbell, Arne Hitzmann, Simon Stepputtis, Shuhei Ikemoto, Koh Hosoda, Heni Ben Amor

Musculoskeletal robots that are based on pneumatic actuation have a variety of properties, such as compliance and back-drivability, that render them particularly appealing for human-robot collaboration.

Response Generation

Probabilistic Multimodal Modeling for Human-Robot Interaction Tasks

2 code implementations14 Aug 2019 Joseph Campbell, Simon Stepputtis, Heni Ben Amor

Human-robot interaction benefits greatly from multimodal sensor inputs as they enable increased robustness and generalization accuracy.

Cannot find the paper you are looking for? You can Submit a new open access paper.