HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World

no code implementations • ICCV 2023 • Xin Wang, Taein Kwon, Mahdi Rad, Bowen Pan, Ishani Chakraborty, Sean Andrist, Dan Bohus, Ashley Feniello, Bugra Tekin, Felipe Vieira Frujeri, Neel Joshi, Marc Pollefeys

Building an interactive AI assistant that can perceive, reason, and collaborate with humans in the real world has been a long-standing pursuit in the AI community.

Mistake Detection Mixed Reality +1

Platform for Situated Intelligence

1 code implementation • 29 Mar 2021 • Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Mihai Jalobeanu, Patrick Sweeney, Anne Loomis Thompson, Eric Horvitz

We introduce Platform for Situated Intelligence, an open-source framework created to support the rapid development and study of multimodal, integrative-AI systems.

Accelerating the Development of Multimodal, Integrative-AI Systems with Platform for Situated Intelligence

no code implementations • 12 Oct 2020 • Sean Andrist, Dan Bohus

We describe Platform for Situated Intelligence, an open-source framework for multimodal, integrative-AI systems.

REFORM: Recognizing F-formations for Social Robots

1 code implementation • 17 Aug 2020 • Hooman Hedayati, Annika Muehlbradt, Daniel J. Szafir, Sean Andrist

Recognizing and understanding conversational groups, or F-formations, is a critical task for situated agents designed to interact with humans.

Metareasoning in Modular Software Systems: On-the-Fly Configuration using Reinforcement Learning with Rich Contextual Representations

no code implementations • 12 May 2019 • Aditya Modi, Debadeepta Dey, Alekh Agarwal, Adith Swaminathan, Besmira Nushi, Sean Andrist, Eric Horvitz

We address the opportunity to maximize the utility of an overall computing system by employing reinforcement learning to guide the configuration of the set of interacting modules that comprise the system.

Decision Making reinforcement-learning +1