Search Results for author:
Found 64 papers, 21 papers with code
Robust Black Box Explanations Under Distribution Shift
As machine learning black boxes are increasingly being deployed in real-world applications, there has been a growing interest in developing post hoc explanations that summarize the behaviors of these black box models.
Manipulating Large Language Models to Increase Product Visibility
We demonstrate that adding a strategic text sequence (STS) -- a carefully crafted message -- to a product's information page can significantly increase its likelihood of being listed as the LLM's top recommendation.
Data Poisoning Attacks on Off-Policy Policy Evaluation Methods
Off-policy Evaluation (OPE) methods are a crucial tool for evaluating policies in high-stakes domains such as healthcare, where exploration is often infeasible, unethical, or expensive.
Towards Safe and Aligned Large Language Models for Medicine
The capabilities of large language models (LLMs) have been progressing at a breathtaking speed, leaving even their own developers grappling with the depth of their potential and risks.
Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems
Retrieval-Augmented Generation (RAG) improves pre-trained models by incorporating external knowledge at test time to enable customized adaptation.
OpenHEXAI: An Open-Source Framework for Human-Centered Evaluation of Explainable Machine Learning
However, properly evaluating the effectiveness of the XAI methods inevitably requires the involvement of human subjects, and conducting human-centered benchmarks is challenging in a number of ways: designing and implementing user studies is complex; numerous design choices in the design space of user study lead to problems of reproducibility; and running user studies can be challenging and even daunting for machine learning researchers.
Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)
CLIP embeddings have demonstrated remarkable performance across a wide range of computer vision tasks.
Towards Uncovering How Large Language Model Works: An Explainability Perspective
Large language models (LLMs) have led to breakthroughs in language tasks, yet the internal mechanisms that enable their remarkable generalization and reasoning abilities remain opaque.
Understanding the Effects of Iterative Prompting on Truthfulness
The development of Large Language Models (LLMs) has notably transformed numerous sectors, offering impressive text generation capabilities.
Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models
We highlight that the current trend towards increasing the plausibility of explanations, primarily driven by the demand for user-friendly interfaces, may come at the cost of diminishing their faithfulness.
A Study on the Calibration of In-context Learning
Accurate uncertainty quantification is crucial for the safe deployment of machine learning models, and prior research has demonstrated improvements in the calibration of modern language models (LMs).
Quantifying Uncertainty in Natural Language Explanations of Large Language Models
In this work, we make one of the first attempts at quantifying the uncertainty in explanations of LLMs.
Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications
Recent literature has suggested the potential of using large language models (LLMs) to make classifications for tabular tasks.
In-Context Unlearning: Language Models as Few Shot Unlearners
In this work, we propose a new class of unlearning methods for LLMs we call ''In-Context Unlearning'', providing inputs in context and without having to update model parameters.
Are Large Language Models Post Hoc Explainers?
To this end, several approaches have been proposed in recent literature to explain the behavior of complex predictive models in a post hoc fashion.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
+1
On the Trade-offs between Adversarial Robustness and Actionable Explanations
As machine learning models are increasingly being employed in various high-stakes settings, it becomes important to ensure that predictions of these models are not only adversarially robust, but also readily explainable to relevant stakeholders.
Certifying LLM Safety against Adversarial Prompting
We defend against three attack modes: i) adversarial suffix, where an adversarial sequence is appended at the end of a harmful prompt; ii) adversarial insertion, where the adversarial sequence is inserted anywhere in the middle of the prompt; and iii) adversarial infusion, where adversarial tokens are inserted at arbitrary positions in the prompt, not necessarily as a contiguous block.
Accurate, Explainable, and Private Models: Providing Recourse While Minimizing Training Data Leakage
We present two novel methods to generate differentially private recourse: Differentially Private Model (DPM) and Laplace Recourse (LR).
Discriminative Feature Attributions: Bridging Post Hoc Explainability and Inherent Interpretability
This strategy naturally combines the ease of use of post hoc explanations with the faithfulness of inherently interpretable models.
Efficient Estimation of Average-Case Robustness for Multi-Class Classification
These estimators linearize models in the local region around an input and analytically compute the robustness of the resulting linear models.
Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributions
Chain-of-thought (CoT) prompting has been shown to empirically improve the accuracy of large language models (LLMs) on various question answering tasks.
On Minimizing the Impact of Dataset Shifts on Actionable Explanations
To this end, we conduct rigorous theoretical analysis to demonstrate that model curvature, weight decay parameters while training, and the magnitude of the dataset shift are key factors that determine the extent of explanation (in)stability.
Consistent Explanations in the Face of Model Indeterminacy via Ensembling
This work addresses the challenge of providing consistent explanations for predictive models in the presence of model indeterminacy, which arises due to the existence of multiple (nearly) equally well-performing models for a given dataset and task.
Word-Level Explanations for Analyzing Bias in Text-to-Image Models
We introduce a method for computing scores for each word in the prompt; these scores represent its influence on biases in the model's output.
Post Hoc Explanations of Language Models Can Improve Language Models
Large Language Models (LLMs) have demonstrated remarkable capabilities in performing complex tasks.
Towards Bridging the Gaps between the Right to Explanation and the Right to be Forgotten
The Right to Explanation and the Right to be Forgotten are two important principles outlined to regulate algorithmic decision making and data usage in real-world applications.
On the Privacy Risks of Algorithmic Recourse
As predictive models are increasingly being employed to make consequential decisions, there is a growing emphasis on developing techniques that can provide algorithmic recourse to affected individuals.
Towards Robust Off-Policy Evaluation via Human Inputs
When deployment environments are expected to undergo changes (that is, dataset shifts), it is important for OPE methods to perform robust evaluation of the policies amidst such changes.
Evaluating Explainability for Graph Neural Networks
As post hoc explanations are increasingly used to understand the behavior of graph neural networks (GNNs), it becomes crucial to evaluate the quality and reliability of GNN explanations.
TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations
In real-world evaluations with humans, 73% of healthcare workers (e. g., doctors and nurses) agreed they would use TalkToModel over baseline point-and-click systems for explainability in a disease prediction task, and 85% of ML professionals agreed TalkToModel was easier to use for computing explanations.
OpenXAI: Towards a Transparent Evaluation of Model Explanations
OpenXAI comprises of the following key components: (i) a flexible synthetic data generator and a collection of diverse real-world datasets, pre-trained models, and state-of-the-art feature attribution methods, and (ii) open-source implementations of eleven quantitative metrics for evaluating faithfulness, stability (robustness), and fairness of explanation methods, in turn providing comparisons of several explanation methods across a wide variety of metrics, models, and datasets.
Efficiently Training Low-Curvature Neural Networks
To achieve this, we minimize a data-independent upper bound on the curvature of a neural network, which decomposes overall curvature in terms of curvatures and slopes of its constituent layers.
A Human-Centric Take on Model Monitoring
Predictive models are increasingly used to make various consequential decisions in high-stakes domains such as healthcare, finance, and policy.
Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations
By bringing diverse explanation methods into a common framework, this work (1) advances the conceptual understanding of these methods, revealing their shared local function approximation objective, properties, and relation to one another, and (2) guides the use of these methods in practice, providing a principled approach to choose among methods and paving the way for the creation of new ones.
Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanations
We then leverage these properties to propose a novel evaluation framework which can quantitatively measure disparities in the quality of explanations output by state-of-the-art methods.
Rethinking Stability for Attribution-based Explanations
As attribution-based explanation methods are increasingly used to establish model trustworthiness in high-stakes situations, it is critical to ensure that these explanations are stable, e. g., robust to infinitesimal perturbations to an input.
Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse
To this end, we propose a novel objective function which simultaneously minimizes the gap between the achieved (resulting) and desired recourse invalidation rates, minimizes recourse costs, and also ensures that the resulting recourse achieves a positive model prediction.
The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective
To this end, we first conduct interviews with data scientists to understand what constitutes disagreement between explanations generated by different methods for the same model prediction, and introduce a novel quantitative framework to formalize this understanding.
Rethinking Explainability as a Dialogue: A Practitioner's Perspective
Overall, we hope our work serves as a starting place for researchers and engineers to design interactive explainability systems.
What will it take to generate fairness-preserving explanations?
In situations where explanations of black-box models may be useful, the fairness of the black-box is also often a relevant concern.
Feature Attributions and Counterfactual Explanations Can Be Manipulated
As machine learning models are increasingly used in critical decision-making settings (e. g., healthcare, finance), there has been a growing emphasis on developing methods to explain model predictions.
Exploring Counterfactual Explanations Through the Lens of Adversarial Examples: A Theoretical and Empirical Analysis
As machine learning (ML) models become more widely deployed in high-stakes applications, counterfactual explanations have emerged as key tools for providing actionable model explanations in practice.
Probing GNN Explainers: A Rigorous Theoretical and Empirical Analysis of GNN Explanation Methods
As Graph Neural Networks (GNNs) are increasingly being employed in critical real-world applications, several methods have been proposed in recent literature to explain the predictions of these models.
Counterfactual Explanations Can Be Manipulated
In this work, we introduce the first framework that describes the vulnerabilities of counterfactual explanations and shows how they can be manipulated.
Learning Under Adversarial and Interventional Shifts
Most of the existing work focuses on optimizing for either adversarial shifts or interventional shifts.
Towards Robust and Reliable Algorithmic Recourse
To address this problem, we propose a novel framework, RObust Algorithmic Recourse (ROAR), that leverages adversarial training for finding recourses that are robust to model shifts.
Towards a Unified Framework for Fair and Stable Graph Representation Learning
In this work, we establish a key connection between counterfactual fairness and stability and leverage it to propose a novel framework, NIFTY (uNIfying Fairness and stabiliTY), which can be used with any GNN to learn fair and stable representations.
Towards the Unification and Robustness of Perturbation and Gradient Based Explanations
As machine learning black boxes are increasingly being deployed in critical domains such as healthcare and criminal justice, there has been a growing emphasis on developing techniques for explaining these black boxes in a post hoc manner.
Algorithmic Recourse in the Wild: Understanding the Impact of Data and Model Shifts
Our theoretical results establish a lower bound on the probability of recourse invalidation due to model shifts, and show the existence of a tradeoff between this invalidation probability and typical notions of "cost" minimized by modern recourse generation algorithms.
Does Fair Ranking Improve Minority Outcomes? Understanding the Interplay of Human and Algorithmic Biases in Online Hiring
In this work, we analyze various sources of gender biases in online hiring platforms, including the job context and inherent biases of employers and establish how these factors interact with ranking algorithms to affect hiring decisions.
Learning Models for Actionable Recourse
As machine learning models are increasingly deployed in high-stakes domains such as legal and financial decision-making, there has been growing interest in post-hoc methods for generating counterfactual explanations.
When Does Uncertainty Matter?: Understanding the Impact of Predictive Uncertainty in ML Assisted Decision Making
As machine learning (ML) models are increasingly being employed to assist human decision makers, it becomes critical to provide these decision makers with relevant inputs which can help them decide if and how to incorporate model predictions into their decision making.
Robust and Stable Black Box Explanations
To the best of our knowledge, this work makes the first attempt at generating post hoc explanations that are robust to a general class of adversarial perturbations that are of practical interest.
Incorporating Interpretable Output Constraints in Bayesian Neural Networks
Domains where supervised models are deployed often come with task-specific constraints, such as prior expert knowledge on the ground-truth function, or desiderata like safety and fairness.
Beyond Individualized Recourse: Interpretable and Interactive Summaries of Actionable Recourses
As predictive models are increasingly being deployed in high-stakes decision-making, there has been a lot of interest in developing algorithms which can provide recourses to affected individuals.
Reliable Post hoc Explanations: Modeling Uncertainty in Explainability
In this paper, we address the aforementioned challenges by developing a novel Bayesian framework for generating local explanations along with their associated uncertainty.
Fair Influence Maximization: A Welfare Optimization Approach
Under this framework, the trade-off between fairness and efficiency can be controlled by a single inequality aversion design parameter.
"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations
Our work is the first to empirically establish how user trust in black box models can be manipulated via misleading explanations.
Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
Our approach can be used to scaffold any biased classifier in such a way that its predictions on the input data distribution still remain biased, but the post hoc explanations of the scaffolded classifier look innocuous.
Interpretable & Explorable Approximations of Black Box Models
To the best of our knowledge, this is the first approach which can produce global explanations of the behavior of any given black box model through joint optimization of unambiguity, fidelity, and interpretability, while also allowing users to explore model behavior based on their preferences.
Confusions over Time: An Interpretable Bayesian Model to Characterize Trends in Decision Making
We propose Confusions over Time (CoT), a novel generative framework which facilitates a multi-granular analysis of the decision making process.
Learning Cost-Effective and Interpretable Regimes for Treatment Recommendation
We formulate this as a problem of learning a decision list -- a sequence of if-then-else rules -- which maps characteristics of subjects (eg., diagnostic test results of patients) to treatments.
Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
Predictive models deployed in the real world may assign incorrect labels to instances with high confidence.
Learning Cost-Effective Treatment Regimes using Markov Decision Processes
We formulate this as a problem of learning a decision list -- a sequence of if-then-else rules -- which maps characteristics of subjects (eg., diagnostic test results of patients) to treatments.
