Search Results for author: Raj Sanjay Shah

Found 16 papers, 2 papers with code

TN-Eval: Rubric and Evaluation Protocols for Measuring the Quality of Behavioral Therapy Notes

no code implementations26 Mar 2025 Raj Sanjay Shah, Lei Xu, Qianchu Liu, Jon Burnsky, Drew Bertagnolli, Chaitanya Shivade

To address this gap, we collaborated with licensed therapists to design a comprehensive rubric for evaluating therapy notes across key dimensions: completeness, conciseness, and faithfulness.

Hallucination

The potential -- and the pitfalls -- of using pre-trained language models as cognitive science theories

no code implementations22 Jan 2025 Raj Sanjay Shah, Sashank Varma

Many studies have evaluated the cognitive alignment of Pre-trained Language Models (PLMs), i. e., their correspondence to adult performance across a range of cognitive domains.

Understanding Graphical Perception in Data Visualization through Zero-shot Prompting of Vision-Language Models

no code implementations31 Oct 2024 Grace Guo, Jenna Jiayi Kang, Raj Sanjay Shah, Hanspeter Pfister, Sashank Varma

Vision Language Models (VLMs) have been successful at many chart comprehension tasks that require attending to both the images of charts and their accompanying textual descriptions.

Data Visualization

Development of Cognitive Intelligence in Pre-trained Language Models

no code implementations1 Jul 2024 Raj Sanjay Shah, Khushi Bhardwaj, Sashank Varma

The increasing cognitive alignment of these models has made them candidates for cognitive science theories.

Incremental Comprehension of Garden-Path Sentences by Large Language Models: Semantic Interpretation, Syntactic Re-Analysis, and Attention

no code implementations25 May 2024 Andrew Li, Xianle Feng, Siddhant Narang, Austin Peng, Tianle Cai, Raj Sanjay Shah, Sashank Varma

The overall goal is to evaluate whether humans and LLMs are aligned in their processing of garden-path sentences and in the lingering misinterpretations past the point of disambiguation, especially when extra-syntactic information (e. g., a comma delimiting a clause boundary) is present to guide processing.

Question Answering Sentence +1

Multi-Level Feedback Generation with Large Language Models for Empowering Novice Peer Counselors

no code implementations21 Mar 2024 Alicja Chaszczewicz, Raj Sanjay Shah, Ryan Louie, Bruce A Arnow, Robert Kraut, Diyi Yang

We further design a self-improvement method on top of large language models to enhance the automatic generation of feedback.

Natural Mitigation of Catastrophic Interference: Continual Learning in Power-Law Learning Environments

no code implementations18 Jan 2024 Atith Gandhi, Raj Sanjay Shah, Vijay Marupudi, Sashank Varma

The benefits of this environment include simplicity, rehearsal that is agnostic to both tasks and models, and the lack of a need for extra neural circuitry.

Continual Learning

Pre-training LLMs using human-like development data corpus

no code implementations8 Nov 2023 Khushi Bhardwaj, Raj Sanjay Shah, Sashank Varma

Pre-trained Large Language Models (LLMs) have shown success in a diverse set of language inference and understanding tasks.

Language Acquisition

Human Behavioral Benchmarking: Numeric Magnitude Comparison Effects in Large Language Models

no code implementations18 May 2023 Raj Sanjay Shah, Vijay Marupudi, Reba Koenen, Khushi Bhardwaj, Sashank Varma

This research shows the utility of understanding LLMs using behavioral benchmarks and points the way to future work on the number representations of LLMs and their cognitive plausibility.

Benchmarking

Modeling Motivational Interviewing Strategies On An Online Peer-to-Peer Counseling Platform

no code implementations9 Nov 2022 Raj Sanjay Shah, Faye Holt, Shirley Anugrah Hayati, Aastha Agarwal, Yi-Chia Wang, Robert E. Kraut, Diyi Yang

This work provides a deeper understanding of the use of motivational interviewing techniques on peer-to-peer counselor platforms and sheds light on how to build better training programs for volunteer counselors on online platforms.

WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain

1 code implementation31 Oct 2022 Raj Sanjay Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, Diyi Yang

To this end, we contribute the Financial Language Understanding Evaluation (FLUE), an open-source comprehensive suite of benchmarks for the financial domain.

FLUE Language Modeling +1

Cannot find the paper you are looking for? You can Submit a new open access paper.