CRAG -- Comprehensive RAG Benchmark

1 code implementation • 7 Jun 2024 • Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang, Lingkun Kong, Brian Moran, Jiaqi Wang, Yifan Ethan Xu, An Yan, Chenyu Yang, Eting Yuan, Hanwen Zha, Nan Tang, Lei Chen, Nicolas Scheffer, Yue Liu, Nirav Shah, Rakesh Wanga, Anuj Kumar, Wen-tau Yih, Xin Luna Dong

To bridge this gap, we introduce the Comprehensive RAG Benchmark (CRAG), a factual question answering benchmark of 4, 409 question-answer pairs and mock APIs to simulate web and Knowledge Graph (KG) search.

Hallucination Language Modelling +2

Ranking-Enhanced Unsupervised Sentence Representation Learning

1 code implementation • 9 Sep 2022 • Yeon Seonwoo, Guoyin Wang, Changmin Seo, Sajal Choudhary, Jiwei Li, Xiang Li, Puyang Xu, Sunghyun Park, Alice Oh

In this work, we show that the semantic meaning of a sentence is also determined by nearest-neighbor sentences that are similar to the input sentence.

Contrastive Learning Data Augmentation +5

Handling Long-Tail Queries with Slice-Aware Conversational Systems

no code implementations • 26 Apr 2021 • Cheng Wang, Sun Kim, Taiwoo Park, Sajal Choudhary, Sunghyun Park, Young-Bum Kim, Ruhi Sarikaya, Sungjin Lee

We have been witnessing the usefulness of conversational AI systems such as Siri and Alexa, directly impacting our daily lives.

Domain Aware Neural Dialog System

no code implementations • 2 Aug 2017 • Sajal Choudhary, Prerna Srivastava, Lyle Ungar, João Sedoc

We investigate the task of building a domain aware chat system which generates intelligent responses in a conversation comprising of different domains.