Search Results for author:
Found 11 papers, 3 papers with code
TRAIL: Trace Reasoning and Agentic Issue Localization
The increasing adoption of agentic workflows across diverse domains brings a critical need to scalably and systematically evaluate the complex traces these systems generate.
Browsing Lost Unformed Recollections: A Benchmark for Tip-of-the-Tongue Search and Reasoning
We introduce Browsing Lost Unformed Recollections, a tip-of-the-tongue known-item search and reasoning benchmark for general AI assistants.
GLIDER: Grading LLM Interactions and Decisions using Explainable Ranking
The LLM-as-judge paradigm is increasingly being adopted for automated evaluation of model outputs.
Lynx: An Open Source Hallucination Evaluation Model
Retrieval Augmented Generation (RAG) techniques aim to mitigate hallucinations in Large Language Models (LLMs).
FinanceBench: A New Benchmark for Financial Question Answering
We test 16 state of the art model configurations (including GPT-4-Turbo, Llama2 and Claude2, with vector stores and long context prompts) on a sample of 150 cases from FinanceBench, and manually review their answers (n=2, 400).
Ranked #1 on
How to refund a wrong transaction in PhonePe
on How to refund a wrong transaction in PhonePe
(using extra training data)
How to refund a wrong transaction in PhonePe
Question Answering
+2
SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models
While some of the models do not give a single unsafe response, most give unsafe responses to more than 20% of the prompts, with over 50% unsafe responses in the extreme.
Perturbation Augmentation for Fairer NLP
Unwanted and often harmful social biases are becoming ever more salient in NLP research, affecting both models and datasets.
Many Episode Learning in a Modular Embodied Agent via End-to-End Interaction
In this work we give a case study of an embodied machine-learning (ML) powered agent that improves itself via interactions with crowd-workers.
Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents
At the heart of improving conversational AI is the open problem of how to evaluate conversations.
droidlet: modular, heterogenous, multi-modal agents
In recent years, there have been significant advances in building end-to-end Machine Learning (ML) systems that learn at scale.
