1 code implementation • 21 Aug 2024 • Zachary Ankner, Mansheej Paul, Brandon Cui, Jonathan D. Chang, Prithviraj Ammanabrolu
To enable reward models to reason explicitly about the quality of a response, we introduce Critique-out-Loud (CLoud) reward models.
1 code implementation • 17 Oct 2023 • Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannaneh Hajishirzi, Yejin Choi, Prithviraj Ammanabrolu
In this work, we study Reinforcement Learning from Personalized Human Feedback (RLPHF) problem, wherein LLMs are aligned to multiple (sometimes conflicting) preferences by modeling alignment as a Multi-Objective Reinforcement Learning (MORL) problem.
1 code implementation • NeurIPS 2023 • Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A. Smith, Mari Ostendorf, Hannaneh Hajishirzi
We introduce Fine-Grained RLHF, a framework that enables training and learning from reward functions that are fine-grained in two respects: (1) density, providing a reward after every segment (e. g., a sentence) is generated; and (2) incorporating multiple reward models associated with different feedback types (e. g., factual incorrectness, irrelevance, and information incompleteness).
2 code implementations • NeurIPS 2023 • Bill Yuchen Lin, Yicheng Fu, Karina Yang, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Prithviraj Ammanabrolu, Yejin Choi, Xiang Ren
The Swift module is a small encoder-decoder LM fine-tuned on the oracle agent's action trajectories, while the Sage module employs LLMs such as GPT-4 for subgoal planning and grounding.
1 code implementation • 24 May 2023 • Ximing Lu, Faeze Brahman, Peter West, Jaehun Jang, Khyathi Chandu, Abhilasha Ravichander, Lianhui Qin, Prithviraj Ammanabrolu, Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Yuchen Lin, Skyler Hallinan, Xiang Ren, Sean Welleck, Yejin Choi
While extreme-scale language models have demonstrated exceptional performance on a variety of language tasks, the degree of control over these language models through pure prompting can often be limited.
no code implementations • 28 Jan 2023 • Kolby Nottingham, Prithviraj Ammanabrolu, Alane Suhr, Yejin Choi, Hannaneh Hajishirzi, Sameer Singh, Roy Fox
Reinforcement learning (RL) agents typically learn tabula rasa, without prior knowledge of the world.
1 code implementation • CVPR 2023 • Youngjae Yu, Jiwan Chung, Heeseung Yun, Jack Hessel, Jae Sung Park, Ximing Lu, Rowan Zellers, Prithviraj Ammanabrolu, Ronan Le Bras, Gunhee Kim, Yejin Choi
Language models are capable of commonsense reasoning: while domain-specific models can learn from explicit knowledge (e. g. commonsense graphs [6], ethical norms [25]), and larger models like GPT-3 manifest broad commonsense reasoning capacity.
no code implementations • 20 Dec 2022 • Pei Zhou, Andrew Zhu, Jennifer Hu, Jay Pujara, Xiang Ren, Chris Callison-Burch, Yejin Choi, Prithviraj Ammanabrolu
We propose a novel task, G4C, to study teacher-student natural language interactions in a goal-driven and grounded environment.
1 code implementation • 13 Oct 2022 • Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, Prithviraj Ammanabrolu
In this work, we explore techniques for augmenting interactive agents with information from symbolic modules, much like humans use tools like calculators and GPS systems to assist with arithmetic and navigation.
3 code implementations • 3 Oct 2022 • Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kianté Brantley, Jack Hessel, Rafet Sifa, Christian Bauckhage, Hannaneh Hajishirzi, Yejin Choi
To help answer this, we first introduce an open-source modular library, RL4LMs (Reinforcement Learning for Language Models), for optimizing language generators with RL.
1 code implementation • 2 Jul 2022 • Zeqiu Wu, Ryu Parish, Hao Cheng, Sewon Min, Prithviraj Ammanabrolu, Mari Ostendorf, Hannaneh Hajishirzi
In an information-seeking conversation, a user may ask questions that are under-specified or unanswerable.
1 code implementation • 26 May 2022 • Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, Yejin Choi
Large-scale language models often learn behaviors that are misaligned with user expectations.
1 code implementation • 25 May 2022 • Youngjae Yu, Jiwan Chung, Heeseung Yun, Jack Hessel, JaeSung Park, Ximing Lu, Prithviraj Ammanabrolu, Rowan Zellers, Ronan Le Bras, Gunhee Kim, Yejin Choi
Large language models readily adapt to novel settings, even without task-specific training data.
no code implementations • NAACL 2022 • Prithviraj Ammanabrolu, Liwei Jiang, Maarten Sap, Hannaneh Hajishirzi, Yejin Choi
We focus on creating agents that act in alignment with socially beneficial norms and values in interactive narratives or text-based games -- environments wherein an agent perceives and interacts with a world through natural language.
1 code implementation • 14 Mar 2022 • Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, Prithviraj Ammanabrolu
We present ScienceWorld, a benchmark to test agents' scientific reasoning abilities in a new interactive text environment at the level of a standard elementary school science curriculum.
1 code implementation • 16 Dec 2021 • Xiangyu Peng, Mark O. Riedl, Prithviraj Ammanabrolu
We focus on the task of creating a reinforcement learning agent that is inherently explainable -- with the ability to produce immediate local explanations by thinking out loud while performing a task and analyzing entire trajectories post-hoc to produce causal explanations.
no code implementations • ACL 2022 • Prithviraj Ammanabrolu, Renee Jia, Mark O. Riedl
We teach goal-driven agents to interactively act and speak in situated environments by training on generated curriculums.
1 code implementation • AKBC Workshop CSKB 2021 • Prithviraj Ammanabrolu, Mark O. Riedl
We provide a dataset that enables the creation of learning agents that can build knowledge graph-based world models of interactive narratives.
no code implementations • NeurIPS 2021 • Prithviraj Ammanabrolu, Mark O. Riedl
World models improve a learning agent's ability to efficiently operate in interactive and situated environments.
no code implementations • SIGDIAL (ACL) 2021 • Wai Man Si, Prithviraj Ammanabrolu, Mark O. Riedl
This paper explores character-driven story continuation, in which the story emerges through characters' first- and second-person narration as well as dialogue -- requiring models to select language that is consistent with a character's persona and their relationships with other characters while following and advancing the story.
no code implementations • 18 Mar 2021 • Prithviraj Ammanabrolu, Mark O. Riedl
This paper provides a roadmap that explores the question of how to imbue learning agents with the ability to understand and generate contextually relevant natural language in service of achieving a goal.
no code implementations • 4 Dec 2020 • Sahith Dambekodi, Spencer Frazier, Prithviraj Ammanabrolu, Mark O. Riedl
We test our technique in the 9to05 game, which is an extreme version of a text based game that requires numerous interactions with common, everyday objects in common, everyday scenarios.
no code implementations • NAACL 2021 • Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim Rocktäschel, Jason Weston
We seek to create agents that both act and communicate with other agents in pursuit of a goal.
1 code implementation • 2 Sep 2020 • Prithviraj Ammanabrolu, Wesley Cheung, William Broniec, Mark O. Riedl
In this work, we introduce the concept of soft causal relations as causal relations inferred from commonsense reasoning.
1 code implementation • 12 Jun 2020 • Prithviraj Ammanabrolu, Ethan Tien, Matthew Hausknecht, Mark O. Riedl
Text-based games are long puzzles or quests, characterized by a sequence of sparse and potentially deceptive rewards.
1 code implementation • 19 Feb 2020 • Prithviraj Ammanabrolu, Ethan Tien, Zhaochen Luo, Mark O. Riedl
We compare our exploration strategies against strong baselines on the classic text-adventure game, Zork1, where prior agent have been unable to get past a bottleneck where the agent is eaten by a Grue.
1 code implementation • 28 Jan 2020 • Prithviraj Ammanabrolu, Wesley Cheung, Dan Tu, William Broniec, Mark O. Riedl
This knowledge graph is then automatically completed utilizing thematic knowledge and used to guide a neural language generation model that fleshes out the rest of the world.
1 code implementation • ICLR 2020 • Prithviraj Ammanabrolu, Matthew Hausknecht
Interactive Fiction games are text-based simulations in which an agent interacts with the world purely through natural language.
no code implementations • CCNLG (ACL) 2019 • Prithviraj Ammanabrolu, William Broniec, Alex Mueller, Jeremy Paul, Mark O. Riedl
Text-adventure games are typically structured as puzzles or quests wherein the player must execute certain actions in a certain order to succeed.
4 code implementations • 11 Sep 2019 • Matthew Hausknecht, Prithviraj Ammanabrolu, Marc-Alexandre Côté, Xingdi Yuan
A hallmark of human intelligence is the ability to understand and communicate with language.
1 code implementation • 8 Sep 2019 • Prithviraj Ammanabrolu, Ethan Tien, Wesley Cheung, Zhaochen Luo, William Ma, Lara J. Martin, Mark O. Riedl
Neural network based approaches to automated story plot generation attempt to learn how to generate novel plots from a corpus of natural language plot summaries.
Ranked #1 on
Event Expansion
on Scifi TV Shows
no code implementations • WS 2019 • Prithviraj Ammanabrolu, Mark O. Riedl
Text adventure games, in which players must make sense of the world through text descriptions and declare actions through text descriptions, provide a stepping stone toward grounding action in language.
no code implementations • WS 2019 • Prithviraj Ammanabrolu, Ethan Tien, Wesley Cheung, Zhaochen Luo, William Ma, Lara Martin, Mark Riedl
Prior work has shown that a semantic abstraction of sentences called events improves neural plot generation and and allows one to decompose the problem into: (1) the generation of a sequence of events (event-to-event) and (2) the transformation of these events into natural language sentences (event-to-sentence).
1 code implementation • NAACL 2019 • Prithviraj Ammanabrolu, Mark O. Riedl
Text-based adventure games provide a platform on which to explore reinforcement learning in the context of a combinatorial action space, such as natural language.
1 code implementation • 5 Jun 2017 • Lara J. Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, Mark O. Riedl
We then present a technique for automated story generation whereby we decompose the problem into the generation of successive events (event2event) and the generation of natural language sentences from events (event2sentence).