Program-Aided Reasoners (better) Know What They Know

1 code implementation • 16 Nov 2023 • Anubha Kabra, Sanketh Rangreji, Yash Mathur, Aman Madaan, Emmy Liu, Graham Neubig

Our analysis uncovers that prompting styles that produce lesser diversity in generations also have more calibrated results, and thus we also experiment with inducing lower generation diversity using temperature scaling and find that for certain temperatures, PAL is not only more accurate but is also more calibrated than COT.

Long-Horizon Dialogue Understanding for Role Identification in the Game of Avalon with Large Language Models

no code implementations • 9 Nov 2023 • Simon Stepputtis, Joseph Campbell, Yaqi Xie, Zhengyang Qi, Wenxin Sharon Zhang, Ruiyi Wang, Sanketh Rangreji, Michael Lewis, Katia Sycara

We discuss the capabilities of LLMs to utilize deceptive long-horizon conversations between six human players to determine each player's goal and motivation.

Decision Making Dialogue Understanding

SummQA at MEDIQA-Chat 2023:In-Context Learning with GPT-4 for Medical Summarization

1 code implementation • 30 Jun 2023 • Yash Mathur, Sanketh Rangreji, Raghav Kapoor, Medha Palavalli, Amanda Bertsch, Matthew R. Gormley

For full-note summarization (Task B), we use a similar solution with k=1.

In-Context Learning