Know Your RAG: Dataset Taxonomy and Generation Strategies for Evaluating RAG Systems

no code implementations • 29 Nov 2024 • Rafael Teixeira de Lima, Shubham Gupta, Cesar Berrospi, Lokesh Mishra, Michele Dolfi, Peter Staar, Panagiotis Vagenas

In this paper, we show that using public question and answer (Q&A) datasets to assess retrieval performance can lead to non-optimal systems design, and that common tools for RAG dataset generation can lead to unbalanced data.

Dataset Generation RAG +2

INDUS: Effective and Efficient Language Models for Scientific Applications

no code implementations • 17 May 2024 • Bishwaranjan Bhattacharjee, Aashka Trivedi, Masayasu Muraoka, Muthukumaran Ramasubramanian, Takuma Udagawa, Iksha Gurung, Nishan Pantha, Rong Zhang, Bharath Dandala, Rahul Ramachandran, Manil Maskey, Kaylin Bugbee, Mike Little, Elizabeth Fancher, Irina Gerasimov, Armin Mehrabian, Lauren Sanders, Sylvain Costes, Sergi Blanco-Cuaresma, Kelly Lockhart, Thomas Allen, Felix Grezes, Megan Ansdell, Alberto Accomazzi, Yousef El-Kurdi, Davis Wertheimer, Birgit Pfitzmann, Cesar Berrospi Ramis, Michele Dolfi, Rafael Teixeira de Lima, Panagiotis Vagenas, S. Karthik Mukkavilli, Peter Staar, Sanaz Vahidinia, Ryan McGranaghan, Tsendgar Lee

The suite of models include: (1) an encoder model trained using domain-specific vocabulary and corpora to address NLP tasks, (2) a contrastive-learning based text embedding model trained using a diverse set of datasets to address information retrieval tasks and (3) smaller versions of these models created using knowledge distillation for applications which have latency or resource constraints.

Contrastive Learning Information Retrieval +4

ESG Accountability Made Easy: DocQA at Your Service

no code implementations • 30 Nov 2023 • Lokesh Mishra, Cesar Berrospi, Kasper Dinkla, Diego Antognini, Francesco Fusco, Benedikt Bothur, Maksym Lysak, Nikolaos Livathinos, Ahmed Nassar, Panagiotis Vagenas, Lucas Morin, Christoph Auer, Michele Dolfi, Peter Staar

We present Deep Search DocQA.

Question Answering

On the explainability of hospitalization prediction on a large COVID-19 patient dataset

no code implementations • 28 Oct 2021 • Ivan Girardi, Panagiotis Vagenas, Dario Arcos-Díaz, Lydia Bessaï, Alexander Büsser, Ludovico Furlan, Raffaello Furlan, Mauro Gatti, Andrea Giovannini, Ellen Hoeven, Chiara Marchiori

We develop various AI models to predict hospitalization on a large (over 110$k$) cohort of COVID-19 positive-tested US patients, sourced from March 2020 to February 2021.

Feature Importance