Budget-Constrained Bounds for Mini-Batch Estimation of Optimal Transport

no code implementations • 24 Oct 2022 • David Alvarez-Melis, Nicolò Fusi, Lester Mackey, Tal Wagner

Optimal Transport (OT) is a fundamental tool for comparing probability distributions, but its exact computation remains prohibitive for large datasets.

Initialization and Regularization of Factorized Neural Layers

1 code implementation • ICLR 2021 • Mikhail Khodak, Neil Tenenholtz, Lester Mackey, Nicolò Fusi

In model compression, we show that they enable low-rank methods to significantly outperform both unstructured sparsity and tensor methods on the task of training low-memory residual networks; analogs of the schemes also improve the performance of tensor decomposition techniques.

Knowledge Distillation Model Compression +2

Dataset Dynamics via Gradient Flows in Probability Space

1 code implementation • 24 Oct 2020 • David Alvarez-Melis, Nicolò Fusi

Various machine learning tasks, from generative modeling to domain adaptation, revolve around the concept of dataset transformation and manipulation.

BIG-bench Machine Learning Domain Adaptation +1

Geometric Dataset Distances via Optimal Transport

1 code implementation • NeurIPS 2020 • David Alvarez-Melis, Nicolò Fusi

The notion of task similarity is at the core of various machine learning paradigms, such as domain adaptation and meta-learning.

Domain Adaptation Meta-Learning +1