no code implementations • 29 Sep 2024 • Vibhor Agarwal, Yiqiao Jin, Mohit Chandra, Munmun De Choudhury, Srijan Kumar, Nishanth Sastry
In this work, we conduct a pioneering study of hallucinations in LLM-generated responses to real-world healthcare queries from patients.
no code implementations • 14 Aug 2024 • Vibhor Agarwal, Yulong Pei, Salwa Alamir, Xiaomo Liu
We propose the first benchmark CodeMirage dataset for code hallucinations.
1 code implementation • 3 Apr 2024 • Vibhor Agarwal, Aravindh Raman, Nishanth Sastry, Ahmed M. Abdelmoniem, Gareth Tyson, Ignacio Castro
Recent work has exploited the conversational context of a post to improve this automatic tagging, e. g. using the replies to a post to help classify if it contains toxic speech.
no code implementations • 22 Jan 2024 • Vibhor Agarwal, Madhav Krishan Garg, Sahiti Dharmavaram, Dhruv Kumar
This study evaluates the effectiveness of various large language models (LLMs) in performing tasks common among undergraduate computer science students.
no code implementations • 21 Oct 2023 • Vibhor Agarwal, Yu Chen, Nishanth Sastry
Specifically, we design two novel algorithms that utilise both the graph structure of the online conversation as well as the semantic information from individual posts for retrieving relevant context nodes from the whole conversation.
no code implementations • 21 Oct 2023 • Vibhor Agarwal, Yu Chen, Nishanth Sastry
We develop 4 different prompts based on task description, hate definition, few-shot demonstrations and chain-of-thoughts for comprehensive experiments and conduct experiments on open-source LLMs such as LLaMA-1, LLaMA-2 chat, Vicuna as well as OpenAI's GPT-3. 5.
1 code implementation • 28 Aug 2023 • Vahid Ghafouri, Vibhor Agarwal, Yong Zhang, Nishanth Sastry, Jose Such, Guillermo Suarez-Tangil
The introduction of ChatGPT and the subsequent improvement of Large Language Models (LLMs) have prompted more and more individuals to turn to the use of ChatBots, both for information and assistance with decision-making.
no code implementations • 20 Dec 2022 • Wenjie Yin, Vibhor Agarwal, Aiqi Jiang, Arkaitz Zubiaga, Nishanth Sastry
During training, the model associates annotators with their label choices given a piece of text; during evaluation, when label information is not available, the model predicts the aggregated label given by the participating annotators by utilising the learnt association.
no code implementations • 16 Nov 2022 • Vibhor Agarwal, Anthony P. Young, Sagar Joglekar, Nishanth Sastry
We evaluate GraphNLI on two such tasks - polarity prediction and misogynistic hate speech detection - and found that our model consistently outperforms all relevant baselines for both tasks.
1 code implementation • 16 Feb 2022 • Vibhor Agarwal, Sagar Joglekar, Anthony P. Young, Nishanth Sastry
We then use these embeddings to predict the polarity relation between a reply and the post it is replying to.