Search Results for author:
Found 34 papers, 16 papers with code
On the Impact of Fine-Tuning on Chain-of-Thought Reasoning
Large language models have emerged as powerful tools for general intelligence, showcasing advanced natural language processing capabilities that find applications across diverse domains.
Towards Operationalizing Right to Data Protection
The widespread practice of indiscriminate data scraping to fine-tune language models (LMs) raises significant legal and ethical concerns, particularly regarding compliance with data protection laws such as the General Data Protection Regulation (GDPR).
On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
In this work, we explore the promise of three broad approaches commonly employed to steer the behavior of LLMs to enhance the faithfulness of the CoT reasoning generated by LLMs: in-context learning, fine-tuning, and activation editing.
MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models
As large language models (LLMs) develop increasingly sophisticated capabilities and find applications in medical settings, it becomes important to assess their medical safety due to their far-reaching implications for personal and public health, patient safety, and human rights.
Understanding the Effects of Iterative Prompting on Truthfulness
The development of Large Language Models (LLMs) has notably transformed numerous sectors, offering impressive text generation capabilities.
Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models
We highlight that the current trend towards increasing the plausibility of explanations, primarily driven by the demand for user-friendly interfaces, may come at the cost of diminishing their faithfulness.
Quantifying Uncertainty in Natural Language Explanations of Large Language Models
In this work, we make one of the first attempts at quantifying the uncertainty in explanations of LLMs.
In-Context Explainers: Harnessing LLMs for Explaining Black Box Models
Despite their effectiveness in enhancing the performance of LLMs on diverse language and tabular tasks, these methods have not been thoroughly explored for their potential to generate post hoc explanations.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
+2
On the Trade-offs between Adversarial Robustness and Actionable Explanations
As machine learning models are increasingly being employed in various high-stakes settings, it becomes important to ensure that predictions of these models are not only adversarially robust, but also readily explainable to relevant stakeholders.
Certifying LLM Safety against Adversarial Prompting
We defend against three attack modes: i) adversarial suffix, where an adversarial sequence is appended at the end of a harmful prompt; ii) adversarial insertion, where the adversarial sequence is inserted anywhere in the middle of the prompt; and iii) adversarial infusion, where adversarial tokens are inserted at arbitrary positions in the prompt, not necessarily as a contiguous block.
Counterfactual Explanation Policies in RL
In this work, we present COUNTERPOL, the first framework to analyze RL policies using counterfactual explanations in the form of minimal changes to the policy that lead to the desired outcome.
Explaining RL Decisions with Trajectories
To do so, we encode trajectories in offline training data individually as well as collectively (encoding a set of trajectories).
Explain like I am BM25: Interpreting a Dense Model's Ranked-List with a Sparse Approximation
Neural retrieval models (NRMs) have been shown to outperform their statistical counterparts owing to their ability to capture semantic meaning via dense document representations.
DeAR: Debiasing Vision-Language Models with Additive Residuals
These biases manifest as the skewed similarity between the representations for specific text concepts and images of people of different identity groups and, therefore, limit the usefulness of such models in real-world high-stakes applications.
GNNDelete: A General Strategy for Unlearning in Graph Neural Networks
Deleted Edge Consistency ensures that the influence of deleted elements is removed from both model weights and neighboring representations, while Neighborhood Influence guarantees that the remaining model knowledge is preserved after deletion.
Towards Estimating Transferability using Hard Subsets
As transfer learning techniques are increasingly used to transfer knowledge from the source model to the target task, it becomes important to quantify which source models are suitable for a given target task without performing computationally expensive fine tuning.
Towards Training GNNs using Explanation Directed Message Passing
With the increasing use of Graph Neural Networks (GNNs) in critical real-world applications, several post hoc explanation methods have been proposed to understand their predictions.
Evaluating Explainability for Graph Neural Networks
As post hoc explanations are increasingly used to understand the behavior of graph neural networks (GNNs), it becomes crucial to evaluate the quality and reliability of GNN explanations.
OpenXAI: Towards a Transparent Evaluation of Model Explanations
OpenXAI comprises of the following key components: (i) a flexible synthetic data generator and a collection of diverse real-world datasets, pre-trained models, and state-of-the-art feature attribution methods, and (ii) open-source implementations of eleven quantitative metrics for evaluating faithfulness, stability (robustness), and fairness of explanation methods, in turn providing comparisons of several explanation methods across a wide variety of metrics, models, and datasets.
Rethinking Stability for Attribution-based Explanations
As attribution-based explanation methods are increasingly used to establish model trustworthiness in high-stakes situations, it is critical to ensure that these explanations are stable, e. g., robust to infinitesimal perturbations to an input.
A Tale Of Two Long Tails
As machine learning models are increasingly employed to assist human decision-makers, it becomes critical to communicate the uncertainty associated with these model predictions.
Exploring Counterfactual Explanations Through the Lens of Adversarial Examples: A Theoretical and Empirical Analysis
As machine learning (ML) models become more widely deployed in high-stakes applications, counterfactual explanations have emerged as key tools for providing actionable model explanations in practice.
Probing GNN Explainers: A Rigorous Theoretical and Empirical Analysis of GNN Explanation Methods
As Graph Neural Networks (GNNs) are increasingly being employed in critical real-world applications, several methods have been proposed in recent literature to explain the predictions of these models.
Towards a Unified Framework for Fair and Stable Graph Representation Learning
In this work, we establish a key connection between counterfactual fairness and stability and leverage it to propose a novel framework, NIFTY (uNIfying Fairness and stabiliTY), which can be used with any GNN to learn fair and stable representations.
Towards the Unification and Robustness of Perturbation and Gradient Based Explanations
As machine learning black boxes are increasingly being deployed in critical domains such as healthcare and criminal justice, there has been a growing emphasis on developing techniques for explaining these black boxes in a post hoc manner.
Estimating Example Difficulty Using Variance of Gradients
In this work, we propose Variance of Gradients (VoG) as a valuable and efficient metric to rank data by difficulty and to surface a tractable subset of the most challenging examples for human-in-the-loop auditing.
The shape and simplicity biases of adversarially robust ImageNet-trained CNNs
Increasingly more similarities between human vision and convolutional neural networks (CNNs) have been revealed in the past few years.
SAM: The Sensitivity of Attribution Methods to Hyperparameters
Attribution methods can provide powerful insights into the reasons for a classifier's decision.
Deep-URL: A Model-Aware Approach To Blind Deconvolution Based On Deep Unfolded Richardson-Lucy Network
The lack of interpretability in current deep learning models causes serious concerns as they are extensively used for various life-critical applications.
Explaining image classifiers by removing input features using generative models
Perturbation-based explanation methods often measure the contribution of an input feature to an image classifier's outputs by heuristically removing it via e. g. blurring, adding noise, or graying out, which often produce unrealistic, out-of-samples.
Removing input features via a generative model to explain their attributions to classifier's decisions
Interpretability methods often measure the contribution of an input feature to an image classifier's decisions by heuristically removing it via e. g. blurring, adding noise, or graying out, which often produce unrealistic, out-of-samples.
Improving Adversarial Robustness by Encouraging Discriminative Features
Intuitively, the center loss encourages DNNs to simultaneously learns a center for the deep features of each class, and minimize the distances between the intra-class deep features and their corresponding class centers.
An Explainable Adversarial Robustness Metric for Deep Learning Neural Networks
Instead of simply measuring a DNN's adversarial robustness in the input domain, as previous works, the proposed NSS is built on top of insightful mathematical understanding of the adversarial attack and gives a more explicit explanation of the robustness.
Convergence of backpropagation with momentum for network architectures with skip connections
We study a class of deep neural networks with networks that form a directed acyclic graph (DAG).
