no code implementations • 17 May 2024 • William E Bishop, Alice Li, Christopher Rawles, Oriana Riva
In the context of autonomous UI agents we then show that LLMs used in this manner are more than $76\%$ accurate at inferring various aspects of latent state, such as performed (vs. commanded) actions and task progression.
3 code implementations • 19 Jul 2023 • Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, Timothy Lillicrap
The dataset contains human demonstrations of device interactions, including the screens and actions, and corresponding natural language instructions.
