Search Results for author: Marco Tulio Ribeiro

Found 14 papers, 8 papers with code

Adaptive Testing and Debugging of NLP Models

no code implementations ACL 2022 Marco Tulio Ribeiro, Scott Lundberg

Current approaches to testing and debugging NLP models rely on highly variable human creativity and extensive labor, or only work for a very restrictive class of bugs.

ExSum: From Local Explanations to Model Understanding

1 code implementation30 Apr 2022 Yilun Zhou, Marco Tulio Ribeiro, Julie Shah

Interpretability methods are developed to understand the working mechanisms of black-box models, which is crucial to their responsible deployment.

Finding and Fixing Spurious Patterns with Explanations

no code implementations3 Jun 2021 Gregory Plumb, Marco Tulio Ribeiro, Ameet Talwalkar

Machine learning models often use spurious patterns such as "relying on the presence of a person to detect a tennis racket," which do not generalize.

Data Augmentation

Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models

1 code implementation ACL 2021 Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, Daniel S. Weld

While counterfactual examples are useful for analysis and training of NLP models, current generation methods either rely on manual labor to create very few counterfactuals, or only instantiate limited types of perturbations such as paraphrases or word substitutions.

Text Generation

Beyond Accuracy: Behavioral Testing of NLP models with CheckList

4 code implementations ACL 2020 Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, Sameer Singh

Although measuring held-out accuracy has been the primary approach to evaluate generalization, it often overestimates the performance of NLP models, while alternative approaches for evaluating models either focus on individual tasks or on specific behaviors.

Question Answering Sentiment Analysis

Are Red Roses Red? Evaluating Consistency of Question-Answering Models

1 code implementation ACL 2019 Marco Tulio Ribeiro, Carlos Guestrin, Sameer Singh

Although current evaluation of question-answering systems treats predictions in isolation, we need to consider the relationship between predictions to measure true understanding.

Question Answering Visual Question Answering +1

Errudite: Scalable, Reproducible, and Testable Error Analysis

1 code implementation ACL 2019 Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, Daniel Weld

Though error analysis is crucial to understanding and improving NLP models, the common practice of manual, subjective categorization of a small sample of errors can yield biased and incomplete conclusions.

Semantically Equivalent Adversarial Rules for Debugging NLP models

1 code implementation ACL 2018 Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin

Complex machine learning models for NLP are often brittle, making different predictions for input instances that are extremely similar semantically.

Data Augmentation Question Answering +3

Programs as Black-Box Explanations

no code implementations22 Nov 2016 Sameer Singh, Marco Tulio Ribeiro, Carlos Guestrin

Recent work in model-agnostic explanations of black-box machine learning has demonstrated that interpretability of complex models does not have to come at the cost of accuracy or model flexibility.

Program induction

Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction Invariance

no code implementations17 Nov 2016 Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin

At the core of interpretable machine learning is the question of whether humans are able to make accurate predictions about a model's behavior.

Interpretable Machine Learning

Model-Agnostic Interpretability of Machine Learning

no code implementations16 Jun 2016 Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin

Understanding why machine learning models behave the way they do empowers both system designers and end-users in many ways: in model selection, feature engineering, in order to trust and act upon the predictions, and in more intuitive user interfaces.

Feature Engineering Model Selection

Cannot find the paper you are looking for? You can Submit a new open access paper.