Search Results for author:
Found 14 papers, 3 papers with code
Why Don't Prompt-Based Fairness Metrics Correlate?
CAIRO augments the original prompts of a given fairness metric by using several pre-trained language models and then selects the combination of the augmented prompts that achieves the highest correlation across metrics.
Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations
Large language models (LLMs) are susceptible to a variety of risks, from non-faithful output to biased and toxic generations.
Fairness-Aware Structured Pruning in Transformers
The increasing size of large language models (LLMs) has introduced challenges in their training and inference.
SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models
Current datasets for unwanted social bias auditing are limited to studying protected demographic features such as race and gender.
Subtle Misogyny Detection and Mitigation: An Expert-Annotated Dataset
Using novel approaches to dataset development, the Biasly dataset captures the nuance and subtlety of misogyny in ways that are unique within the literature.
Keeping Up with the Language Models: Systematic Benchmark Extension for Bias Auditing
We demonstrate that the newly created dataset BBNLI-next is more challenging than BBNLI: on average, BBNLI-next reduces the accuracy of state-of-the-art NLI models from 95. 3%, as observed by BBNLI, to a strikingly low 57. 5%.
Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendations
In this study, we investigate the level of implicit race information available to ML models and human experts and the implications of model-detectable differences in clinical notes.
Downstream Fairness Caveats with Synthetic Healthcare Data
In this paper, we evaluate the fairness of models generated on two healthcare datasets for gender and race biases.
Ground-Truth, Whose Truth? -- Examining the Challenges with Annotating Toxic Text Datasets
The use of machine learning (ML)-based language models (LMs) to monitor content online is on the rise.
Your fairness may vary: Pretrained language model fairness in toxic text classification
Through the analysis of more than a dozen pretrained language models of varying sizes on two toxic text classification tasks (English), we demonstrate that focusing on accuracy measures alone can lead to models with wide variation in fairness characteristics.
Biomedical Interpretable Entity Representations
Pre-trained language models induce dense entity representations that offer strong performance on entity-centric NLP tasks, but such representations are not immediately interpretable.
Drug Repurposing for Cancer: An NLP Approach to Identify Low-Cost Therapies
More than 200 generic drugs approved by the U. S. Food and Drug Administration for non-cancer indications have shown promise for treating cancer.
How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?
Our findings suggest that besides the glitches in the collaboration readiness, technology readiness, and coupling of work dimensions, the tensions that exist in the common ground building process influence the collaboration outcomes, and then persist in the actual collaboration process.
Teaching machines to understand data science code by semantic enrichment of dataflow graphs
Your computer is continuously executing programs, but does it really understand them?
