Search Results for author:
Found 52 papers, 23 papers with code
We Can't Understand AI Using our Existing Vocabulary
This means interpretability can be framed as a communication problem: humans must be able to reference and control machine concepts, and communicate human concepts to machines.
Proactive Agents for Multi-Turn Text-to-Image Generation Under Uncertainty
To address this, we propose a design for proactive T2I agents equipped with an interface to (1) actively ask clarification questions when uncertain, and (2) present their understanding of user intent as an understandable belief graph that a user can edit.
Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero
Artificial Intelligence (AI) systems have made remarkable progress, attaining super-human performance across various domains.
Getting aligned on representational alignment
These questions pertaining to the study of representational alignment are at the heart of some of the most promising research areas in contemporary cognitive science, neuroscience, and machine learning.
Don't trust your eyes: on the (un)reliability of feature visualizations
Today, visualization methods form the foundation of our knowledge about the internal workings of neural networks, as a type of mechanistic interpretability.
Model evaluation for extreme risks
Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities.
Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models
This finding raises questions about how past work relies on Causal Tracing to select which model layers to edit.
Impossibility Theorems for Feature Attribution
Despite a sea of interpretability methods that can produce plausible explanations, the field has also empirically seen many failure cases of such methods.
On the Relationship Between Explanation and Prediction: A Causal View
Our work borrows tools from causal inference to systematically assay this relationship.
Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation
We investigate whether three types of post hoc model explanations--feature attribution, concept activation, and training point ranking--are effective for detecting a model's reliance on spurious signals in the training data.
Advanced Methods for Connectome-Based Predictive Modeling of Human Intelligence: A Novel Approach Based on Individual Differences in Cortical Topography
Investigating the proposed modeling framework's efficacy, we find that advanced connectome-based predictive modeling generates neuroscience predictions that account for a significantly greater proportion of variance in general intelligence scores than previously established methods, advancing our scientific understanding of the network architecture that underlies human intelligence.
Human-Centered Concept Explanations for Neural Networks
We start by introducing concept explanations including the class of Concept Activation Vectors (CAV) which characterize concepts using vectors in appropriate spaces of neural activations, and discuss different properties of useful concepts, and approaches to measure the usefulness of concept vectors.
Subgoal-Based Explanations for Unreliable Intelligent Decision Support Systems
Intelligent decision support (IDS) systems leverage artificial intelligence techniques to generate recommendations that guide human users through the decision making phases of a task.
Analyzing a Caching Model
Machine Learning has been successfully applied in systems applications such as memory prefetching and caching, where learned models have been shown to outperform heuristics.
Acquisition of Chess Knowledge in AlphaZero
In this work we provide evidence that human knowledge is acquired by the AlphaZero neural network as it trains on the game of chess.
Best of both worlds: local and global explanations with human-understandable concepts
While there are many methods focused on either one, few frameworks can provide both local and global explanations in a consistent manner.
DISSECT: Disentangled Simultaneous Explanations via Concept Traversals
Explaining deep learning model inferences is a promising venue for scientific understanding, improving safety, uncovering hidden biases, evaluating fairness, and beyond, as argued by many scholars.
Debugging Tests for Model Explanations
For several explanation methods, we assess their ability to: detect spurious correlation artifacts (data contamination), diagnose mislabeled training examples (data contamination), differentiate between a (partially) re-initialized model and a trained one (model contamination), and detect out-of-distribution inputs (test-time contamination).
Concept Bottleneck Models
We seek to learn models that we can interact with using high-level concepts: if the model did not think there was a bone spur in the x-ray, would it still predict severe arthritis?
On Completeness-aware Concept-Based Explanations in Deep Neural Networks
Next, we propose a concept discovery method that aims to infer a complete set of concepts that are additionally encouraged to be interpretable, which addresses the limitations of existing methods on concept explanations.
On Concept-Based Explanations in Deep Neural Networks
Next, we propose a concept discovery method that considers two additional constraints to encourage the interpretability of the discovered concepts.
Benchmarking Attribution Methods with Relative Feature Importance
Despite active development, quantitative evaluation of feature attribution methods remains difficult due to the lack of ground truth: we do not know which input features are in fact important to a model.
Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems
We then provide a mechanism to generate the smallest set of changes that will improve an individual's outcome.
Explaining Classifiers with Causal Concept Effect (CaCE)
To overcome this problem, we define the Causal Concept Effect (CaCE) as the causal effect of (the presence or absence of) a human-interpretable concept on a deep neural net's predictions.
Visualizing and Measuring the Geometry of BERT
Transformer architectures show significant promise for natural language processing.
Neural Networks Trained on Natural Scenes Exhibit Gestalt Closure
The Gestalt laws of perceptual organization, which describe how visual elements in an image are grouped and interpreted, have traditionally been thought of as innate despite their ecological validity.
Towards Automatic Concept-based Explanations
Interpretability has become an important topic of research as more machine learning (ML) models are deployed and widely used to make important decisions.
An Evaluation of the Human-Interpretability of Explanation
Recent years have seen a boom in interest in machine learning systems that can provide a human-understandable rationale for their predictions or decisions.
Interpreting Black Box Predictions using Fisher Kernels
Research in both machine learning and psychology suggests that salient examples can help humans to interpret learning models.
Sanity Checks for Saliency Maps
We find that reliance, solely, on visual assessment can be misleading.
Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values
Explaining the output of a complicated machine learning model like a deep neural network (DNN) is a central challenge in machine learning.
Proceedings of the 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018)
This is the Proceedings of the 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018), which was held in Stockholm, Sweden, July 14, 2018.
A Benchmark for Interpretability Methods in Deep Neural Networks
We propose an empirical measure of the approximate accuracy of feature importance estimates in deep neural networks.
xGEMs: Generating Examplars to Explain Black-Box Models
This work proposes xGEMs or manifold guided exemplars, a framework to understand black-box classifier behavior by exploring the landscape of the underlying data manifold as data points cross decision boundaries.
To Trust Or Not To Trust A Classifier
Knowing when a classifier's prediction can be trusted is useful in many applications and critical for safely using AI.
Human-in-the-Loop Interpretability Prior
We often desire our models to be interpretable as well as accurate.
How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
Recent years have seen a boom in interest in machine learning systems that can provide a human-understandable rationale for their predictions or decisions.
TCAV: Relative concept importance testing with Linear Concept Activation Vectors
In particular, this framework enables non-machine learning experts to express concepts of interests and test hypotheses using examples (e. g., a set of pictures that illustrate the concept).
Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
The interpretation of deep learning models is a challenge due to their size, complexity, and often opaque internal state.
The (Un)reliability of saliency methods
Saliency methods aim to explain the predictions of deep neural networks.
Proceedings of the 2017 ICML Workshop on Human Interpretability in Machine Learning (WHI 2017)
This is the Proceedings of the 2017 ICML Workshop on Human Interpretability in Machine Learning (WHI 2017), which was held in Sydney, Australia, August 10, 2017.
SmoothGrad: removing noise by adding noise
Explaining the output of a deep network remains a challenge.
Learning how to explain neural networks: PatternNet and PatternAttribution
We show that these methods do not produce the theoretically correct explanation for a linear model.
Towards A Rigorous Science of Interpretable Machine Learning
As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs.
BIG-bench Machine Learning
Interpretable Machine Learning
+1
Examples are not enough, learn to criticize! Criticism for Interpretability
Example-based explanations are widely used in the effort to improve the interpretability of highly complex distributions.
Proceedings of NIPS 2016 Workshop on Interpretable Machine Learning for Complex Systems
This is the Proceedings of NIPS 2016 Workshop on Interpretable Machine Learning for Complex Systems, held in Barcelona, Spain on December 9, 2016
Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016)
This is the Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), which was held in New York, NY, June 23, 2016.
Mind the Gap: A Generative Approach to Interpretable Feature Selection and Extraction
We present the Mind the Gap Model (MGM), an approach for interpretable feature extraction and selection.
The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
We present the Bayesian Case Model (BCM), a general framework for Bayesian case-based reasoning (CBR) and prototype classification and clustering.
Inferring Robot Task Plans from Human Team Meetings: A Generative Modeling Approach with Logic-Based Prior
We present an algorithm that reduces this translation burden by inferring the final plan from a processed form of the human team's planning conversation.
