Search Results for author: Mikhail Yurochkin

Found 58 papers, 28 papers with code

Aligners: Decoupling LLMs and Alignment

no code implementations7 Mar 2024 Lilian Ngweta, Mayank Agarwal, Subha Maity, Alex Gittens, Yuekai Sun, Mikhail Yurochkin

Large Language Models (LLMs) need to be aligned with human expectations to ensure their safety and utility in most applications.

Asymmetry in Low-Rank Adapters of Foundation Models

1 code implementation26 Feb 2024 Jiacheng Zhu, Kristjan Greenewald, Kimia Nadjahi, Haitz Sáez de Ocáriz Borde, Rickard Brüel Gabrielsson, Leshem Choshen, Marzyeh Ghassemi, Mikhail Yurochkin, Justin Solomon

Specifically, when updating the parameter matrices of a neural network by adding a product $BA$, we observe that the $B$ and $A$ matrices have distinct functions: $A$ extracts features from the input, while $B$ uses these features to create the desired output.

tinyBenchmarks: evaluating LLMs with fewer examples

2 code implementations22 Feb 2024 Felipe Maia Polo, Lucas Weber, Leshem Choshen, Yuekai Sun, Gongjun Xu, Mikhail Yurochkin

The versatility of large language models (LLMs) led to the creation of diverse benchmarks that thoroughly test a variety of language models' abilities.

Multiple-choice

Estimating Fréchet bounds for validating programmatic weak supervision

no code implementations7 Dec 2023 Felipe Maia Polo, Mikhail Yurochkin, Moulinath Banerjee, Subha Maity, Yuekai Sun

We develop methods for estimating Fr\'echet bounds on (possibly high-dimensional) distribution classes in which some variables are continuous-valued.

Fusing Models with Complementary Expertise

no code implementations2 Oct 2023 Hongyi Wang, Felipe Maia Polo, Yuekai Sun, Souvik Kundu, Eric Xing, Mikhail Yurochkin

Training AI models that generalize across tasks and domains has long been among the open problems driving AI research.

Multiple-choice text-classification +2

An Investigation of Representation and Allocation Harms in Contrastive Learning

1 code implementation2 Oct 2023 Subha Maity, Mayank Agarwal, Mikhail Yurochkin, Yuekai Sun

In this paper, we demonstrate that contrastive learning (CL), a popular variant of SSL, tends to collapse representations of minority groups with certain majority groups.

Contrastive Learning Self-Supervised Learning +1

GeRA: Label-Efficient Geometrically Regularized Alignment

no code implementations1 Oct 2023 Dustin Klebe, Tal Shnitzer, Mikhail Yurochkin, Leonid Karlinsky, Justin Solomon

We introduce a semi-supervised Geometrically Regularized Alignment (GeRA) method to align the embedding spaces of pretrained unimodal encoders in a label-efficient way.

Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairness

no code implementations1 Mar 2023 Zahra Ashktorab, Benjamin Hoover, Mayank Agarwal, Casey Dugan, Werner Geyer, Hao Bang Yang, Mikhail Yurochkin

While several toolkits exist to aid machine learning practitioners in addressing fairness issues, little is known about the strategies practitioners employ to evaluate model fairness and what factors influence their assessment, particularly in the context of text classification.

Fairness text-classification +1

Simple Disentanglement of Style and Content in Visual Representations

1 code implementation20 Feb 2023 Lilian Ngweta, Subha Maity, Alex Gittens, Yuekai Sun, Mikhail Yurochkin

Learning visual representations with interpretable features, i. e., disentangled representations, remains a challenging problem.

Disentanglement Domain Generalization

Sampling with Mollified Interaction Energy Descent

2 code implementations24 Oct 2022 Lingxiao Li, Qiang Liu, Anna Korba, Mikhail Yurochkin, Justin Solomon

These energies rely on mollifier functions -- smooth approximations of the Dirac delta originated from PDE theory.

Outlier-Robust Group Inference via Gradient Space Clustering

1 code implementation13 Oct 2022 Yuchen Zeng, Kristjan Greenewald, Kangwook Lee, Justin Solomon, Mikhail Yurochkin

Traditional machine learning models focus on achieving good performance on the overall training distribution, but they often underperform on minority groups.

Clustering

How does overparametrization affect performance on minority groups?

1 code implementation7 Jun 2022 Subha Maity, Saptarshi Roy, Songkai Xue, Mikhail Yurochkin, Yuekai Sun

The benefits of overparameterization for the overall performance of modern machine learning (ML) models are well known.

regression

Understanding new tasks through the lens of training data via exponential tilting

1 code implementation26 May 2022 Subha Maity, Mikhail Yurochkin, Moulinath Banerjee, Yuekai Sun

However, it is conceivable that the training data can be reweighted to be more representative of the new (target) task.

Model Selection

Domain Adaptation meets Individual Fairness. And they get along

no code implementations1 May 2022 Debarghya Mukherjee, Felix Petersen, Mikhail Yurochkin, Yuekai Sun

In this paper, we leverage this connection between algorithmic fairness and distribution shifts to show that algorithmic fairness interventions can help ML models overcome distribution shifts, and that domain adaptation methods (for overcoming distribution shifts) can mitigate algorithmic biases.

Domain Adaptation Fairness

Log-Euclidean Signatures for Intrinsic Distances Between Unaligned Datasets

1 code implementation3 Feb 2022 Tal Shnitzer, Mikhail Yurochkin, Kristjan Greenewald, Justin Solomon

We use manifold learning to compare the intrinsic geometric structures of different datasets by comparing their diffusion operators, symmetric positive-definite (SPD) matrices that relate to approximations of the continuous Laplace-Beltrami operator from discrete samples.

Rewiring with Positional Encodings for Graph Neural Networks

no code implementations29 Jan 2022 Rickard Brüel-Gabrielsson, Mikhail Yurochkin, Justin Solomon

As a conservative alternative, we use positional encodings to expand receptive fields to $r$-hop neighborhoods.

Learning Proximal Operators to Discover Multiple Optima

1 code implementation28 Jan 2022 Lingxiao Li, Noam Aigerman, Vladimir G. Kim, Jiajin Li, Kristjan Greenewald, Mikhail Yurochkin, Justin Solomon

We present an end-to-end method to learn the proximal operator of a family of training problems so that multiple local minima can be quickly obtained from initial guesses by iterating the learned operator, emulating the proximal-point algorithm that has fast convergence.

object-detection Object Detection

Propagating Distributions through Neural Networks

no code implementations29 Sep 2021 Felix Petersen, Christian Borgelt, Mikhail Yurochkin, Hilde Kuehne, Oliver Deussen

We propose a new approach to propagating probability distributions through neural networks.

regression

$k$-Mixup Regularization for Deep Learning via Optimal Transport

no code implementations29 Sep 2021 Kristjan Greenewald, Anming Gu, Mikhail Yurochkin, Justin Solomon, Edward Chien

To better leverage the structure of the data, we extend mixup to $k$-mixup by perturbing $k$-batches of training points in the direction of other $k$-batches using displacement interpolation, i. e. interpolation under the Wasserstein metric.

Adversarial Robustness

Your fairness may vary: Pretrained language model fairness in toxic text classification

no code implementations Findings (ACL) 2022 Ioana Baldini, Dennis Wei, Karthikeyan Natesan Ramamurthy, Mikhail Yurochkin, Moninder Singh

Through the analysis of more than a dozen pretrained language models of varying sizes on two toxic text classification tasks (English), we demonstrate that focusing on accuracy measures alone can lead to models with wide variation in fairness characteristics.

Fairness Language Modelling +2

Measuring the robustness of Gaussian processes to kernel choice

no code implementations11 Jun 2021 William T. Stephenson, Soumya Ghosh, Tin D. Nguyen, Mikhail Yurochkin, Sameer K. Deshpande, Tamara Broderick

We demonstrate in both synthetic and real-world examples that decisions made with a GP can exhibit non-robustness to kernel choice, even when prior draws are qualitatively interchangeable to a user.

Gaussian Processes

k-Mixup Regularization for Deep Learning via Optimal Transport

1 code implementation5 Jun 2021 Kristjan Greenewald, Anming Gu, Mikhail Yurochkin, Justin Solomon, Edward Chien

Our empirical results show that training with $k$-mixup further improves generalization and robustness across several network architectures and benchmark datasets of differing modalities.

Adversarial Robustness Hyperparameter Optimization

Individually Fair Gradient Boosting

no code implementations ICLR 2021 Alexander Vargo, Fan Zhang, Mikhail Yurochkin, Yuekai Sun

Gradient boosting is a popular method for machine learning from tabular data, which arise often in applications where algorithmic fairness is a concern.

Fairness

Statistical inference for individual fairness

1 code implementation ICLR 2021 Subha Maity, Songkai Xue, Mikhail Yurochkin, Yuekai Sun

As we rely on machine learning (ML) models to make more consequential decisions, the issue of ML models perpetuating or even exacerbating undesirable historical biases (e. g., gender and racial biases) has come to the fore of the public's attention.

Adversarial Attack Fairness

Individually Fair Ranking

no code implementations19 Mar 2021 Amanda Bower, Hamid Eftekhari, Mikhail Yurochkin, Yuekai Sun

We develop an algorithm to train individually fair learning-to-rank (LTR) models.

Fairness Learning-To-Rank

Outlier Robust Optimal Transport

no code implementations1 Jan 2021 Debarghya Mukherjee, Aritra Guha, Justin Solomon, Yuekai Sun, Mikhail Yurochkin

In light of recent advances in solving the OT problem, OT distances are widely used as loss functions in minimum distance estimation.

Outlier Detection

Online Semi-Supervised Learning with Bandit Feedback

no code implementations ICLR Workshop LLD 2019 Sohini Upadhyay, Mikhail Yurochkin, Mayank Agarwal, Yasaman Khazaeni, DjallelBouneffouf

We formulate a new problem at the intersectionof semi-supervised learning and contextual bandits, motivated by several applications including clini-cal trials and ad recommendations.

Imputation Multi-Armed Bandits

There is no trade-off: enforcing fairness can improve accuracy

no code implementations28 Sep 2020 Subha Maity, Debarghya Mukherjee, Mikhail Yurochkin, Yuekai Sun

If the algorithmic biases in an ML model are due to sampling biases in the training data, then enforcing algorithmic fairness may improve the performance of the ML model on unbiased test data.

Fairness

Continuous Regularized Wasserstein Barycenters

1 code implementation NeurIPS 2020 Lingxiao Li, Aude Genevay, Mikhail Yurochkin, Justin Solomon

Leveraging a new dual formulation for the regularized Wasserstein barycenter problem, we introduce a stochastic algorithm that constructs a continuous approximation of the barycenter.

Model Fusion with Kullback--Leibler Divergence

1 code implementation ICML 2020 Sebastian Claici, Mikhail Yurochkin, Soumya Ghosh, Justin Solomon

Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors and proceeds using a simple assign-and-average approach.

Federated Learning

Two Simple Ways to Learn Individual Fairness Metrics from Data

no code implementations19 Jun 2020 Debarghya Mukherjee, Mikhail Yurochkin, Moulinath Banerjee, Yuekai Sun

Individual fairness is an intuitive definition of algorithmic fairness that addresses some of the drawbacks of group fairness.

Fairness Vocal Bursts Valence Prediction

Auditing ML Models for Individual Bias and Unfairness

no code implementations11 Mar 2020 Songkai Xue, Mikhail Yurochkin, Yuekai Sun

We consider the task of auditing ML models for individual bias/unfairness.

Federated Learning with Matched Averaging

1 code implementation ICLR 2020 Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, Yasaman Khazaeni

Federated learning allows edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store the data in the cloud.

Federated Learning

Alleviating Label Switching with Optimal Transport

1 code implementation NeurIPS 2019 Pierre Monteiller, Sebastian Claici, Edward Chien, Farzaneh Mirzazadeh, Justin Solomon, Mikhail Yurochkin

Label switching is a phenomenon arising in mixture model posterior inference that prevents one from meaningfully assessing posterior statistics using standard Monte Carlo procedures.

On Efficient Multilevel Clustering via Wasserstein Distances

1 code implementation19 Sep 2019 Viet Huynh, Nhat Ho, Nhan Dam, XuanLong Nguyen, Mikhail Yurochkin, Hung Bui, and Dinh Phung

We propose a novel approach to the problem of multilevel clustering, which aims to simultaneously partition data in each group and discover grouping patterns among groups in a potentially large hierarchically structured corpus of data.

Clustering

Training individually fair ML models with Sensitive Subspace Robustness

2 code implementations ICLR 2020 Mikhail Yurochkin, Amanda Bower, Yuekai Sun

We consider training machine learning models that are fair in the sense that their performance is invariant under certain sensitive perturbations to the inputs.

BIG-bench Machine Learning Fairness

Hierarchical Optimal Transport for Document Representation

1 code implementation NeurIPS 2019 Mikhail Yurochkin, Sebastian Claici, Edward Chien, Farzaneh Mirzazadeh, Justin Solomon

The ability to measure similarity between documents enables intelligent summarization and analysis of large corpora.

Bayesian Nonparametric Federated Learning of Neural Networks

1 code implementation28 May 2019 Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Trong Nghia Hoang, Yasaman Khazaeni

In federated learning problems, data is scattered across different servers and exchanging or pooling it is often impractical or prohibited.

Federated Learning General Classification +1

Dirichlet Simplex Nest and Geometric Inference

1 code implementation27 May 2019 Mikhail Yurochkin, Aritra Guha, Yuekai Sun, XuanLong Nguyen

We propose Dirichlet Simplex Nest, a class of probabilistic models suitable for a variety of data types, and develop fast and provably accurate inference algorithms by accounting for the model's convex geometry and low dimensional simplicial structure.

Probabilistic Federated Neural Matching

no code implementations ICLR 2019 Mikhail Yurochkin, Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, Yasaman Khazaeni

In federated learning problems, data is scattered across different servers and exchanging or pooling it is often impractical or prohibited.

Federated Learning General Classification +1

Scalable inference of topic evolution via models for latent geometric structures

1 code implementation NeurIPS 2019 Mikhail Yurochkin, Zhiwei Fan, Aritra Guha, Paraschos Koutris, XuanLong Nguyen

We develop new models and algorithms for learning the temporal dynamics of the topic polytopes and related geometric objects that arise in topic model based inference.

UPS: optimizing Undirected Positive Sparse graph for neural graph filtering

no code implementations ICLR 2018 Mikhail Yurochkin, Dung Thai, Hung Hai Bui, XuanLong Nguyen

In this work we propose a novel approach for learning graph representation of the data using gradients obtained via backpropagation.

Multi-way Interacting Regression via Factorization Machines

1 code implementation NeurIPS 2017 Mikhail Yurochkin, XuanLong Nguyen, Nikolaos Vasiloglou

We propose a Bayesian regression method that accounts for multi-way interactions of arbitrary orders among the predictor variables.

regression

Multilevel Clustering via Wasserstein Means

1 code implementation ICML 2017 Nhat Ho, XuanLong Nguyen, Mikhail Yurochkin, Hung Hai Bui, Viet Huynh, Dinh Phung

We propose a novel approach to the problem of multilevel clustering, which aims to simultaneously partition data in each group and discover grouping patterns among groups in a potentially large hierarchically structured corpus of data.

Clustering

Geometric Dirichlet Means algorithm for topic inference

no code implementations NeurIPS 2016 Mikhail Yurochkin, XuanLong Nguyen

We propose a geometric algorithm for topic learning and inference that is built on the convex geometry of topics arising from the Latent Dirichlet Allocation (LDA) model and its nonparametric extensions.

Clustering Variational Inference

Cannot find the paper you are looking for? You can Submit a new open access paper.