Search Results for author:
Found 22 papers, 8 papers with code
Can We Improve Model Robustness through Secondary Attribute Counterfactuals?
By implementing RDI in the context of toxicity detection, we find that accounting for secondary attributes can significantly improve robustness, with improvements in sliced accuracy on the original dataset up to 7% compared to existing robustness methods.
Six Attributes of Unhealthy Conversations
We present a new dataset of approximately 44000 comments labeled by crowdworkers.
Gemma: Open Models Based on Gemini Research and Technology
This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models.
Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics
We explore a strategy to handle controversial topics in LLM-based chatbots based on Wikipedia's Neutral Point of View (NPOV) principle: acknowledge the absence of a single true answer and surface multiple perspectives.
ConstitutionalExperts: Training a Mixture of Principle-based Prompts
We also show that we can improve overall performance by learning unique prompts for different semantic regions of the training data and using a mixture-of-experts (MoE) architecture to route inputs at inference time.
Improving Classifier Robustness through Active Generation of Pairwise Counterfactuals
We demonstrate that with a small amount of human-annotated counterfactual data (10%), we can generate a counterfactual augmentation dataset with learned labels, that provides an 18-20% improvement in robustness and a 14-21% reduction in errors on 6 out-of-domain datasets, comparable to that of a fully human-annotated counterfactual dataset for both sentiment classification and question paraphrase tasks.
Gradient-Based Automated Iterative Recovery for Parameter-Efficient Tuning
In this paper, we explore the use of TracIn to improve model performance in the parameter-efficient tuning (PET) setting.
Towards Agile Text Classifiers for Everyone
Text-based safety classifiers are widely used for content moderation and increasingly to tune generative language model behavior - a topic of growing concern for the safety of digital assistants and chatbots.
Plex: Towards Reliability using Pretrained Large Model Extensions
A recent trend in artificial intelligence is the use of pretrained models for language and vision tasks, which have achieved extraordinary performance but also puzzling failures.
Measuring Recommender System Effects with Simulated Users
Using this simulation framework, we can (a) isolate the effect of the recommender system from the user preferences, and (b) examine how the system performs not just on average for an "average user" but also the extreme experiences under atypical user behavior.
Six Attributes of Unhealthy Conversation
We present a new dataset of approximately 44000 comments labeled by crowdworkers.
Fairness without Demographics through Adversarially Reweighted Learning
Much of the previous machine learning (ML) fairness literature assumes that protected features such as race and sex are present in the dataset, and relies upon them to mitigate fairness concerns.
Toxicity Detection: Does Context Really Matter?
Moderation is crucial to promoting healthy on-line discussions.
Classifying Constructive Comments
The quality of the annotation scheme and the resulting dataset is evaluated using measurements of inter-annotator agreement, expert assessment of a sample, and by the constructiveness sub-characteristics, which we show provide a proxy for the general constructiveness concept.
Practical Compositional Fairness: Understanding Fairness in Multi-Component Recommender Systems
In addition to the theoretical results, we find on multiple datasets -- including a large-scale real-world recommender system -- that the overall system's end-to-end fairness is largely achievable by improving fairness in individual components.
Debiasing Embeddings for Reduced Gender Bias in Text Classification
(Bolukbasi et al., 2016) demonstrated that pretrained word embeddings can inherit gender bias from the data they were trained on.
ConvAI at SemEval-2019 Task 6: Offensive Language Identification and Categorization with Perspective and BERT
This paper presents the application of two strong baseline systems for toxicity detection and evaluates their performance in identifying and categorizing offensive language in social media.
Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification
Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in society at large.
Limitations of Pinned AUC for Measuring Unintended Bias
This report examines the Pinned AUC metric introduced and highlights some of its limitations.
WikiConv: A Corpus of the Complete Conversational History of a Large Online Collaborative Community
We present a corpus that encompasses the complete history of conversations between contributors to Wikipedia, one of the largest online collaborative communities.
Conversations Gone Awry: Detecting Early Signs of Conversational Failure
One of the main challenges online social systems face is the prevalence of antisocial behavior, such as harassment and personal attacks.
Ex Machina: Personal Attacks Seen at Scale
The damage personal attacks cause to online discourse motivates many platforms to try to curb the phenomenon.
