Search Results for author: Vamshi Krishna Bonagiri

Found 4 papers, 1 papers with code

COBIAS: Assessing the Contextual Reliability of Bias Benchmarks for Language Models

no code implementations22 Feb 2024 Priyanshul Govil, Hemang Jain, Vamshi Krishna Bonagiri, Aman Chadha, Ponnurangam Kumaraguru, Manas Gaur, Sanorita Dey

We develop the Context-Oriented Bias Indicator and Assessment Score (COBIAS) to measure a biased statement's reliability in detecting bias, based on the variance in model behavior across different contexts.

SaGE: Evaluating Moral Consistency in Large Language Models

1 code implementation21 Feb 2024 Vamshi Krishna Bonagiri, Sreeram Vennam, Priyanshul Govil, Ponnurangam Kumaraguru, Manas Gaur

To this extent, we construct the Moral Consistency Corpus (MCC), containing 50K moral questions, responses to them by LLMs, and the RoTs that these models followed.

Decision Making HellaSwag +2

Measuring Moral Inconsistencies in Large Language Models

no code implementations26 Jan 2024 Vamshi Krishna Bonagiri, Sreeram Vennam, Manas Gaur, Ponnurangam Kumaraguru

To address this issue, we propose a novel information-theoretic measure called Semantic Graph Entropy (SGE) to measure the consistency of an LLM in moral scenarios.

Decision Making Language Modeling +3

Cannot find the paper you are looking for? You can Submit a new open access paper.