Search Results for author:
Found 7 papers, 3 papers with code
FinanceBench: A New Benchmark for Financial Question Answering
We test 16 state of the art model configurations (including GPT-4-Turbo, Llama2 and Claude2, with vector stores and long context prompts) on a sample of 150 cases from FinanceBench, and manually review their answers (n=2, 400).
SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models
While some of the models do not give a single unsafe response, most give unsafe responses to more than 20% of the prompts, with over 50% unsafe responses in the extreme.
Perturbation Augmentation for Fairer NLP
Unwanted and often harmful social biases are becoming ever more salient in NLP research, affecting both models and datasets.
Many Episode Learning in a Modular Embodied Agent via End-to-End Interaction
In this work we give a case study of an embodied machine-learning (ML) powered agent that improves itself via interactions with crowd-workers.
Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents
At the heart of improving conversational AI is the open problem of how to evaluate conversations.
droidlet: modular, heterogenous, multi-modal agents
In recent years, there have been significant advances in building end-to-end Machine Learning (ML) systems that learn at scale.
