Search Results for author:
Found 46 papers, 3 papers with code
Fairness without Demographics through Adversarially Reweighted Learning
Much of the previous machine learning (ML) fairness literature assumes that protected features such as race and sex are present in the dataset, and relies upon them to mitigate fairness concerns.
The Case for Learned Index Structures
Indexes are models: a B-Tree-Index can be seen as a model to map a key to the position of a record within a sorted array, a Hash-Index as a model to map a key to a position of a record within an unsorted array, and a BitMap-Index as a model to indicate if a data record exists or not.
Top-K Off-Policy Correction for a REINFORCE Recommender System
The contributions of the paper are: (1) scaling REINFORCE to a production recommender system with an action space on the orders of millions; (2) applying off-policy correction to address data biases in learning from logged feedback collected from multiple behavior policies; (3) proposing a novel top-K off-policy correction to account for our policy recommending multiple items at a time; (4) showcasing the value of exploration.
The Many Faces of Link Fraud
Most past work on social network link fraud detection tries to separate genuine users from fraudsters, implicitly assuming that there is only one type of fraudulent behavior.
BIRDNEST: Bayesian Inference for Ratings-Fraud Detection
To combine these 2 approaches, we formulate our Bayesian Inference for Rating Data (BIRD) model, a flexible Bayesian model of user rating behavior.
Explaining reviews and ratings with PACO: Poisson Additive Co-Clustering
With this novel technique we propose a new Bayesian model for joint collaborative filtering of ratings and text reviews through a sum of simple co-clusterings.
ACCAMS: Additive Co-Clustering to Approximate Matrices Succinctly
Matrix completion and approximation are popular tools to capture a user's preferences for recommendation and to approximate missing data.
Spotting Suspicious Link Behavior with fBox: An Adversarial Perspective
How can we detect suspicious users in large online networks?
Counterfactual Fairness in Text Classification through Robustness
In this paper, we study counterfactual fairness in text classification, which asks the question: How would the prediction change if the sensitive attribute referenced in the example were different?
Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements
In this paper we provide a case-study on the application of fairness in machine learning research to a production classification system, and offer new insights in how to measure and address algorithmic fairness issues.
Towards Neural Mixture Recommender for Long Range Dependent User Sequences
Our approach employs a mixture of models, each with a different temporal range.
Fairness in Recommendation Ranking through Pairwise Comparisons
Recommender systems are one of the most pervasive applications of machine learning in industry, with many services using them to match users to products or information.
Transfer of Machine Learning Fairness across Domains
A model trained for one setting may be picked up and used in many others, particularly as is common with pre-training and cloud APIs.
Toward a better trade-off between performance and fairness with kernel-based distribution matching
As recent literature has demonstrated how classifiers often carry unintended biases toward some subgroups, deploying machine learned models to users demands careful consideration of the social consequences.
Practical Compositional Fairness: Understanding Fairness in Multi-Component Recommender Systems
In addition to the theoretical results, we find on multiple datasets -- including a large-scale real-world recommender system -- that the overall system's end-to-end fairness is largely achievable by improving fairness in individual components.
Improving Calibration through the Relationship with Adversarial Robustness
To this end, we propose Adversarial Robustness based Adaptive Label Smoothing (AR-AdaLS) that integrates the correlations of adversarial robustness and calibration into training by adaptively softening labels for an example based on how easily it can be attacked by an adversary.
What are effective labels for augmented data? Improving robustness with AutoLabel
Despite this, most existing work simply reuses the original label from the clean data, and the choice of label accompanying the augmented data is relatively less explored.
CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation
Experiments on real-world NLP datasets demonstrate that our method can generate more diverse and fluent adversarial texts, compared to many existing adversarial text generation approaches.
Measuring and Reducing Gendered Correlations in Pre-trained Models
Pre-trained models have revolutionized natural language understanding.
Underspecification Presents Challenges for Credibility in Modern Machine Learning
Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains.
Learned Indexes for a Google-scale Disk-based Database
There is great excitement about learned index structures, but understandable skepticism about the practicality of a new method uprooting decades of research on B-Trees.
Measuring Recommender System Effects with Simulated Users
Using this simulation framework, we can (a) isolate the effect of the recommender system from the user preferences, and (b) examine how the system performs not just on average for an "average user" but also the extreme experiences under atypical user behavior.
Evaluating Fairness of Machine Learning Models Under Uncertain and Incomplete Information
An alternate approach that is commonly used is to separately train an attribute classifier on data with sensitive attribute information, and then use it later in the ML pipeline to evaluate the bias of a given classifier.
Towards Content Provider Aware Recommender Systems: A Simulation Study on the Interplay between User and Provider Utilities
We then build a REINFORCE recommender agent, coined EcoAgent, to optimize a joint objective of user utility and the counterfactual utility lift of the provider associated with the recommended content, which we show to be equivalent to maximizing overall user utility and the utilities of all providers on the platform under some mild assumptions.
Measuring Model Fairness under Noisy Covariates: A Theoretical Perspective
In this work we study the problem of measuring the fairness of a machine learning model under noisy information.
Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning
This presents a multi-dimensional Pareto frontier on (1) the trade-off between group fairness and accuracy with respect to each task, as well as (2) the trade-offs across multiple tasks.
Recurrent Recommender Networks
Recommender systems traditionally assume that user profiles and movie attributes are static.
Understanding and Improving Robustness of Vision Transformers through Patch-based Negative Augmentation
We show that patch-based negative augmentation consistently improves robustness of ViTs across a wide set of ImageNet based robustness benchmarks.
Can We Improve Model Robustness through Secondary Attribute Counterfactuals?
By implementing RDI in the context of toxicity detection, we find that accounting for secondary attributes can significantly improve robustness, with improvements in sliced accuracy on the original dataset up to 7% compared to existing robustness methods.
Flexible text generation for counterfactual fairness probing
A common approach for testing fairness issues in text-based classifiers is through the use of counterfactuals: does the classifier output change if a sensitive attribute in the input is changed?
Simpson's Paradox in Recommender Fairness: Reconciling differences between per-user and aggregated evaluations
We reconcile these notions and show that the tension is due to differences in distributions of users where items are relevant, and break down the important factors of the user's recommendations.
A Human-ML Collaboration Framework for Improving Video Content Reviews
We deal with the problem of localized in-video taxonomic human annotation in the video content moderation domain, where the goal is to identify video segments that violate granular policies, e. g., community guidelines on an online video platform.
Striving for data-model efficiency: Identifying data externalities on group performance
Building trustworthy, effective, and responsible machine learning systems hinges on understanding how differences in training data and modeling decisions interact to impact predictive performance.
What Are Effective Labels for Augmented Data? Improving Calibration and Robustness with AutoLabel
A wide breadth of research has devised data augmentation approaches that can improve both accuracy and generalization performance for neural networks.
Towards Robust Prompts on Vision-Language Models
With the advent of vision-language models (VLMs) that can perform in-context and prompt-based learning, how can we design prompting approaches that robustly generalize to distribution shift and can be used on novel classes outside the support set of the prompts?
Improving Classifier Robustness through Active Generation of Pairwise Counterfactuals
We demonstrate that with a small amount of human-annotated counterfactual data (10%), we can generate a counterfactual augmentation dataset with learned labels, that provides an 18-20% improvement in robustness and a 14-21% reduction in errors on 6 out-of-domain datasets, comparable to that of a fully human-annotated counterfactual dataset for both sentiment classification and question paraphrase tasks.
Let's Do a Thought Experiment: Using Counterfactuals to Improve Moral Reasoning
Language models still struggle on moral reasoning, despite their impressive performance in many other tasks.
Towards A Scalable Solution for Improving Multi-Group Fairness in Compositional Classification
Despite the rich literature on machine learning fairness, relatively little attention has been paid to remediating complex systems, where the final prediction is the combination of multiple classifiers and where multiple groups are present.
Learning from Negative User Feedback and Measuring Responsiveness for Sequential Recommenders
In this work, we incorporate explicit and implicit negative user feedback into the training objective of sequential recommenders in the retrieval stage using a "not-to-recommend" loss function that optimizes for the log-likelihood of not recommending items with negative feedback.
Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Voting
A crucial challenge for generative large language models (LLMs) is diversity: when a user's prompt is under-specified, models may follow implicit assumptions while generating a response, which may result in homogenization of the responses, as well as certain demographic groups being under-represented or even erased from the generated responses.
Controlled Decoding from Language Models
KL-regularized reinforcement learning (RL) is a popular alignment framework to control the language model responses towards high reward outcomes.
Language Modelling
Multi-Objective Reinforcement Learning
+1
Improving Few-shot Generalization of Safety Classifiers via Data Augmented Parameter-Efficient Fine-Tuning
As large language models (LLMs) are widely adopted, new safety issues and policies emerge, to which existing safety classifiers do not generalize well.
Break it, Imitate it, Fix it: Robustness by Generating Human-Like Attacks
We demonstrate the advantages of this system on the ANLI and hate speech detection benchmark datasets - both collected via an iterative, adversarial human-and-model-in-the-loop procedure.
Multi-Group Fairness Evaluation via Conditional Value-at-Risk Testing
Here, the sample complexity for estimating the worst-case performance gap across groups (e. g., the largest difference in error rates) increases exponentially with the number of group-denoting sensitive attributes.
Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Images
First, a text-guided approach is used to extract a person-diversity representation from a pre-trained image-text model.
