Search Results for author: Ashutosh Sathe

Found 6 papers, 1 papers with code

MAPLE: Multilingual Evaluation of Parameter Efficient Finetuning of Large Language Models

no code implementations15 Jan 2024 Divyanshu Aggarwal, Ashutosh Sathe, Ishaan Watts, Sunayana Sitaram

Prior work on multilingual evaluation has shown that there is a large gap between the performance of LLMs on English and other languages.

MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks

no code implementations13 Nov 2023 Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Maxamed Axmed, Kalika Bali, Sunayana Sitaram

We also perform a study on data contamination and find that several models are likely to be contaminated with multilingual evaluation benchmarks, necessitating approaches to detect and handle contamination while assessing the multilingual performance of LLMs.

Benchmarking

Benchmarking and Improving Text-to-SQL Generation under Ambiguity

1 code implementation20 Oct 2023 Adithya Bhaskar, Tushar Tomar, Ashutosh Sathe, Sunita Sarawagi

Research in Text-to-SQL conversion has been largely benchmarked against datasets where each text query corresponds to one correct SQL.

Benchmarking Natural Language Queries +2

Cannot find the paper you are looking for? You can Submit a new open access paper.