Search Results for author:
Found 23 papers, 8 papers with code
Implicit Rate-Constrained Optimization of Non-decomposable Objectives
We consider a popular family of constrained optimization problems arising in machine learning that involve optimizing a non-decomposable evaluation metric with a certain thresholded form, while constraining another metric of interest.
Churn Reduction via Distillation
In real-world systems, models are frequently updated as more data becomes available, and in addition to achieving high accuracy, the goal is to also maintain a low difference in predictions compared to the base model (i. e. predictive "churn").
Distilling Double Descent
Distillation is the technique of training a "student" model based on examples that are labeled by a separate "teacher" model, which itself is trained on a labeled dataset.
Approximate Heavily-Constrained Learning with Lagrange Multiplier Models
In machine learning applications such as ranking fairness or fairness over intersectional groups, one often encounters optimization problems with an extremely large number of constraints.
Robust Optimization for Fairness with Noisy Protected Groups
Second, we introduce two new approaches using robust optimization that, unlike the naive approach of only relying on $\hat{G}$, are guaranteed to satisfy fairness criteria on the true protected groups G while minimizing a training objective.
Optimizing Generalized Rate Metrics with Three Players
We present a general framework for solving a large class of learning problems with non-linear functions of classification rates.
On Making Stochastic Classifiers Deterministic
Stochastic classifiers arise in a number of machine learning problems, and have become especially prominent of late, as they often result from constrained optimization problems, e. g. for fairness, churn, or custom losses.
Optimizing Generalized Rate Metrics through Game Equilibrium
We present a general framework for solving a large class of learning problems with non-linear functions of classification rates.
Pairwise Fairness for Ranking and Regression
We present pairwise fairness metrics for ranking models and regression models that form analogues of statistical fairness notions such as equal opportunity, equal accuracy, and statistical parity.
Diminishing Returns Shape Constraints for Interpretability and Regularization
We investigate machine learning models that can provide diminishing returns and accelerating returns guarantees to capture prior knowledge or policies about how outputs should depend on inputs.
Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals
This new formulation leads to an algorithm that produces a stochastic classifier by playing a two-player non-zero-sum game solving for what we call a semi-coarse correlated equilibrium, which in turn corresponds to an approximately optimal and feasible solution to the constrained optimization problem.
Constrained Interacting Submodular Groupings
We introduce the problem of grouping a finite ground set into blocks where each block is a subset of the ground set and where: (i) the blocks are individually highly valued by a submodular function (both robustly and in the average case) while satisfying block-specific matroid constraints; and (ii) block scores interact where blocks are jointly scored highly, thus making the blocks mutually non-redundant.
Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints
Classifiers can be trained with data-dependent constraints to satisfy fairness goals, reduce churn, achieve a targeted false positive rate, or other policy goals.
Proxy Fairness
We consider the problem of improving fairness when one lacks access to a dataset labeled with protected groups, making it difficult to take advantage of strategies that can improve fairness but require protected group labels, either at training or runtime.
Interpretable Set Functions
We propose learning flexible but interpretable functions that aggregate a variable-length set of permutation-invariant feature vectors to predict a label.
Two-Player Games for Efficient Non-Convex Constrained Optimization
For both the proxy-Lagrangian and Lagrangian formulations, however, we prove that this classifier, instead of having unbounded size, can be taken to be a distribution over no more than m+1 models (where m is the number of constraints).
Fast and Flexible Monotonic Functions with Ensembles of Lattices
For many machine learning problems, there are some inputs that are known to be positively (or negatively) related to the output, and in such cases training the model to respect that monotonic relationship can provide regularization, and makes the model more interpretable.
Satisfying Real-world Goals with Dataset Constraints
The goal of minimizing misclassification error on a training set is often just one of several real-world goals that might be defined on different datasets.
A Light Touch for Heavily Constrained SGD
Minimizing empirical risk subject to a set of constraints can be a useful strategy for learning restricted classes of functions, such as monotonic functions, submodular functions, classifiers that guarantee a certain class label for some subset of examples, etc.
Monotonic Calibrated Interpolated Look-Up Tables
Real-world machine learning applications may require functions that are fast-to-evaluate and interpretable.
Stochastic Optimization for Machine Learning
In Part ii, we will consider the unsupervised problem of Principal Component Analysis, for which the learning task is to find the directions which contain most of the variance of the data distribution.
Stochastic Optimization of PCA with Capped MSG
We study PCA as a stochastic optimization problem and propose a novel stochastic approximation algorithm which we refer to as "Matrix Stochastic Gradient" (MSG), as well as a practical variant, Capped MSG.
Better Mini-Batch Algorithms via Accelerated Gradient Methods
Mini-batch algorithms have recently received significant attention as a way to speed-up stochastic convex optimization problems.
