Search Results for author: Nithum Thain

Found 19 papers, 7 papers with code

Can We Improve Model Robustness through Secondary Attribute Counterfactuals?

no code implementations EMNLP 2021 Ananth Balashankar, Xuezhi Wang, Ben Packer, Nithum Thain, Ed Chi, Alex Beutel

By implementing RDI in the context of toxicity detection, we find that accounting for secondary attributes can significantly improve robustness, with improvements in sliced accuracy on the original dataset up to 7% compared to existing robustness methods.

coreference-resolution Data Augmentation +1

Improving Classifier Robustness through Active Generation of Pairwise Counterfactuals

no code implementations22 May 2023 Ananth Balashankar, Xuezhi Wang, Yao Qin, Ben Packer, Nithum Thain, Jilin Chen, Ed H. Chi, Alex Beutel

We demonstrate that with a small amount of human-annotated counterfactual data (10%), we can generate a counterfactual augmentation dataset with learned labels, that provides an 18-20% improvement in robustness and a 14-21% reduction in errors on 6 out-of-domain datasets, comparable to that of a fully human-annotated counterfactual dataset for both sentiment classification and question paraphrase tasks.

Data Augmentation Sentiment Analysis +1

Gradient-Based Automated Iterative Recovery for Parameter-Efficient Tuning

no code implementations13 Feb 2023 Maximilian Mozes, Tolga Bolukbasi, Ann Yuan, Frederick Liu, Nithum Thain, Lucas Dixon

In this paper, we explore the use of TracIn to improve model performance in the parameter-efficient tuning (PET) setting.

Decision Making Transfer Learning

Towards Agile Text Classifiers for Everyone

no code implementations13 Feb 2023 Maximilian Mozes, Jessica Hoffmann, Katrin Tomanek, Muhamed Kouate, Nithum Thain, Ann Yuan, Tolga Bolukbasi, Lucas Dixon

Text-based safety classifiers are widely used for content moderation and increasingly to tune generative language model behavior - a topic of growing concern for the safety of digital assistants and chatbots.

Language Modelling text-classification +1

Measuring Recommender System Effects with Simulated Users

no code implementations12 Jan 2021 Sirui Yao, Yoni Halpern, Nithum Thain, Xuezhi Wang, Kang Lee, Flavien Prost, Ed H. Chi, Jilin Chen, Alex Beutel

Using this simulation framework, we can (a) isolate the effect of the recommender system from the user preferences, and (b) examine how the system performs not just on average for an "average user" but also the extreme experiences under atypical user behavior.

Collaborative Filtering Recommendation Systems

Fairness without Demographics through Adversarially Reweighted Learning

5 code implementations NeurIPS 2020 Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, Ed H. Chi

Much of the previous machine learning (ML) fairness literature assumes that protected features such as race and sex are present in the dataset, and relies upon them to mitigate fairness concerns.


Classifying Constructive Comments

1 code implementation11 Apr 2020 Varada Kolhatkar, Nithum Thain, Jeffrey Sorensen, Lucas Dixon, Maite Taboada

The quality of the annotation scheme and the resulting dataset is evaluated using measurements of inter-annotator agreement, expert assessment of a sample, and by the constructiveness sub-characteristics, which we show provide a proxy for the general constructiveness concept.

Domain Adaptation

Practical Compositional Fairness: Understanding Fairness in Multi-Component Recommender Systems

no code implementations5 Nov 2019 Xuezhi Wang, Nithum Thain, Anu Sinha, Flavien Prost, Ed H. Chi, Jilin Chen, Alex Beutel

In addition to the theoretical results, we find on multiple datasets -- including a large-scale real-world recommender system -- that the overall system's end-to-end fairness is largely achievable by improving fairness in individual components.

Fairness Recommendation Systems

Debiasing Embeddings for Reduced Gender Bias in Text Classification

no code implementations WS 2019 Flavien Prost, Nithum Thain, Tolga Bolukbasi

(Bolukbasi et al., 2016) demonstrated that pretrained word embeddings can inherit gender bias from the data they were trained on.

General Classification text-classification +2

ConvAI at SemEval-2019 Task 6: Offensive Language Identification and Categorization with Perspective and BERT

no code implementations SEMEVAL 2019 John Pavlopoulos, Nithum Thain, Lucas Dixon, Ion Androutsopoulos

This paper presents the application of two strong baseline systems for toxicity detection and evaluates their performance in identifying and categorizing offensive language in social media.

Language Identification

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

4 code implementations11 Mar 2019 Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, Lucy Vasserman

Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in society at large.

BIG-bench Machine Learning Fairness +3

WikiConv: A Corpus of the Complete Conversational History of a Large Online Collaborative Community

no code implementations EMNLP 2018 Yiqing Hua, Cristian Danescu-Niculescu-Mizil, Dario Taraborelli, Nithum Thain, Jeffery Sorensen, Lucas Dixon

We present a corpus that encompasses the complete history of conversations between contributors to Wikipedia, one of the largest online collaborative communities.

Conversations Gone Awry: Detecting Early Signs of Conversational Failure

no code implementations ACL 2018 Justine Zhang, Jonathan P. Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Nithum Thain, Dario Taraborelli

One of the main challenges online social systems face is the prevalence of antisocial behavior, such as harassment and personal attacks.

Ex Machina: Personal Attacks Seen at Scale

3 code implementations27 Oct 2016 Ellery Wulczyn, Nithum Thain, Lucas Dixon

The damage personal attacks cause to online discourse motivates many platforms to try to curb the phenomenon.

Cannot find the paper you are looking for? You can Submit a new open access paper.