Search Results for author:
Found 15 papers, 11 papers with code
How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis
We develop NegotiationArena: a flexible framework for evaluating and probing the negotiation abilities of LLM agents.
ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome Management
One early application has been to medicine, where LLMs have been investigated to streamline clinical workflows and facilitate clinical analysis and decision-making.
KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval
Motivated by rising concerns around factual incorrectness and hallucinations of LLMs, we present KITAB, a new dataset for measuring constraint satisfaction abilities of language models.
Diversity of Thought Improves Reasoning Abilities of LLMs
Large language models (LLMs) are documented to struggle in settings that require complex reasoning.
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
We investigate the internal behavior of Transformer-based Large Language Models (LLMs) when they generate factually incorrect text.
Discover and Cure: Concept-aware Mitigation of Spurious Correlation
Deep neural networks often rely on spurious correlations to make predictions, which hinders generalization beyond training environments.
GPT detectors are biased against non-native English writers
In this study, we evaluate the performance of several widely-used GPT detectors using writing samples from native and non-native English writers.
Leveraging medical Twitter to build a visual–language foundation model for pathology AI
This is the largest public dataset for pathology images annotated with natural text.
SkinCon: A skin disease dataset densely annotated by domain experts for fine-grained model debugging and analysis
To provide a medical dataset densely annotated by domain experts with annotations useful across multiple disease processes, we developed SkinCon: a skin disease dataset densely annotated by dermatologists.
Holistic Evaluation of Language Models
We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models.
When and why vision-language models behave like bags-of-words, and what to do about it?
ARO consists of Visual Genome Attribution, to test the understanding of objects' properties; Visual Genome Relation, to test for relational understanding; and COCO & Flickr30k-Order, to test for order sensitivity.
Post-hoc Concept Bottleneck Models
When concept annotations are not available on the training data, we show that PCBM can transfer concepts from other datasets or from natural language descriptions of concepts via multimodal models.
Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanations
Understanding and explaining the mistakes made by trained models is critical to many machine learning objectives, such as improving robustness, addressing concept drift, and mitigating biases.
ImageNet performance correlates with pose estimation robustness and generalization on out-of-domain data
Neural networks are highly effective tools for pose estimation.
Learning Maximally Predictive Prototypes in Multiple Instance Learning
In this work, we propose a simple model that provides permutation invariant maximally predictive prototype generator from a given dataset, which leads to interpretability of the solution and concrete insights to the nature and the solution of a problem.
