Search Results for author:
Found 14 papers, 10 papers with code
The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers
Evaluation of large language models (LLMs) for code has primarily relied on static benchmarks, including HumanEval (Chen et al., 2021), which measure the ability of LLMs to generate complete code that passes unit tests.
Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposium
The organization of the research roundtables at the conference involved 17 Senior Chairs and 19 Junior Chairs across 11 tables.
Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study
Augmentations were evaluated for errors by clinicians, and we found misleading errors occur, with errors more common in real donated notes than synthetic notes, illustrating the importance of carefully written clinical notes.
Effective Human-AI Teams via Learned Natural Language Rules and Onboarding
In this work, we propose to learn rules, grounded in data regions and described in natural language, that illustrate how the human should collaborate with the AI.
When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming
Using data from 535 programmers, we perform a retrospective evaluation of CDHF and show that we can avoid displaying a significant fraction of suggestions that would have been rejected.
Closing the Gap in High-Risk Pregnancy Care Using Machine Learning and Human-AI Collaboration
This work presents the implementation of a real-world ML-based system to assist care managers in identifying pregnant patients at risk of complications.
Who Should Predict? Exact Algorithms For Learning to Defer to Humans
We show that prior approaches can fail to find a human-AI system with low misclassification error even when there exists a linear classifier and rejector that have zero error (the realizable setting).
Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming
However, to fully realize their potential, we must understand how programmers interact with these systems and identify ways to improve that interaction.
Sample Efficient Learning of Predictors that Complement Humans
One of the goals of learning algorithms is to complement and reduce the burden on human decision makers.
Teaching Humans When To Defer to a Classifier via Exemplars
For this collaboration to perform properly, the human decision maker must have a mental model of when and when not to rely on the agent.
Consistent Estimators for Learning to Defer to an Expert
Learning algorithms are often used in conjunction with expert decision makers in practical scenarios, however this fact is largely ignored when designing these algorithms.
Fair Learning with Private Demographic Data
Sensitive attributes such as race are rarely available to learners in real world settings as their collection is often restricted by laws and regulations.
Neural Arabic Question Answering
Our system for open domain question answering in Arabic (SOQAL) is based on two components: (1) a document retriever using a hierarchical TF-IDF approach and (2) a neural reading comprehension model using the pre-trained bi-directional transformer BERT.
From Fair Decision Making to Social Equality
In this paper, we propose a simple yet revealing model that encompasses (1) a selection process where an institution chooses from multiple groups according to their qualifications so as to maximize an institutional utility and (2) dynamics that govern the evolution of the groups' qualifications according to the imposed policies.
