Search Results for author:
Found 32 papers, 13 papers with code
Scaffold with Stochastic Gradients: New Analysis with Linear Speed-Up
This paper proposes a novel analysis for the Scaffold algorithm, a popular method for dealing with data heterogeneity in federated learning.
Humanity's Last Exam
However, benchmarks are not keeping pace in difficulty: LLMs now achieve over 90\% accuracy on popular benchmarks like MMLU, limiting informed measurement of state-of-the-art LLM capabilities.
Ranked #4 on
Humanity's Last Exam
on Humanity's Last Exam
Refined Analysis of Federated Averaging's Bias and Federated Richardson-Romberg Extrapolation
In this paper, we present a novel analysis of FedAvg with constant step size, relying on the Markov property of the underlying process.
Unified Breakdown Analysis for Byzantine Robust Gossip
We introduce a practical robust aggregation rule, coined $\rm CS_{ours}$, such that $\rm CS_{ours}\text{-}RG$ has a near-optimal breakdown.
Byzantine-Robust Gossip: Insights from a Dual Approach
Distributed approaches have many computational benefits, but they are vulnerable to attacks from a subset of devices transmitting incorrect information.
Random features models: a way to study the success of naive imputation
Constant (naive) imputation is still widely used in practice as this is a first easy-to-use technique to deal with missing data.
Sliced-Wasserstein Estimation with Spherical Harmonics as Control Variates
Spherical harmonics are polynomials on the sphere that form an orthonormal basis of the set of square-integrable functions on the sphere.
Compression with Exact Error Distribution for Federated Learning
Compression schemes have been extensively used in Federated Learning (FL) to reduce the communication cost of distributed learning.
Proving Linear Mode Connectivity of Neural Networks via Optimal Transport
The energy landscape of high-dimensional non-convex optimization problems is crucial to understanding the effectiveness of modern deep neural network architectures.
Compressed and distributed least-squares regression: convergence rates with applications to Federated Learning
In this paper, we investigate the impact of compression on stochastic gradient algorithms for machine learning, a technique widely used in distributed and federated learning.
Conformal Prediction with Missing Values
This motivates our novel generalized conformalized quantile regression framework, missing data augmentation, which yields prediction intervals that are valid conditionally to the patterns of missing values, despite their exponential number.
Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning
Stochastic Approximation (SA) is a classical algorithm that has had since the early days a huge impact on signal processing, and nowadays on machine learning, due to the necessity to deal with a large amount of data observed with uncertainties.
FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings
In this work, we propose a novel cross-silo dataset suite focused on healthcare, FLamby (Federated Learning AMple Benchmark of Your cross-silo strategies), to bridge the gap between theory and practice of cross-silo FL.
Adaptive Conformal Predictions for Time Series
While recent works tackled this issue, we argue that Adaptive Conformal Inference (ACI, Gibbs and Cand{\`e}s, 2021), developed for distribution-shift time series, is a good procedure for time series with general dependency.
Minimax rate of consistency for linear models with missing values
Missing values arise in most real-world data sets due to the aggregation of multiple sources and intrinsically missing information (sensor failure, unanswered questions in surveys...).
PEPit: computer-assisted worst-case analyses of first-order optimization methods in Python
PEPit is a Python package aiming at simplifying the access to worst-case analyses of a large family of first-order optimization methods possibly involving gradient, projection, proximal, or linear optimization oracles, along with their approximate, or Bregman variants.
Federated-EM with heterogeneity mitigation and variance reduction
The Expectation Maximization (EM) algorithm is the default algorithm for inference in latent variable models.
Differentially Private Federated Learning on Heterogeneous Data
Federated Learning (FL) is a paradigm for large-scale distributed learning which faces two key challenges: (i) efficient training from highly heterogeneous user data, and (ii) protecting the privacy of participating users.
Federated Expectation Maximization with heterogeneity mitigation and variance reduction
The Expectation Maximization (EM) algorithm is the default algorithm for inference in latent variable models.
QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning
The objective of Federated Learning (FL) is to perform statistical inference for data which are decentralised and stored locally on networked clients.
$\texttt{DoStoVoQ}$: Doubly Stochastic Voronoi Vector Quantization SGD for Federated Learning
The growing size of models and datasets have made distributed implementation of stochastic gradient descent (SGD) an active field of research.
Preserved central model for faster bidirectional compression in distributed settings
To obtain this improvement, we design MCM, an algorithm such that the downlink compression only impacts local models, while the global model is preserved.
Debiasing Averaged Stochastic Gradient Descent to handle missing values
Stochastic gradient algorithm is a key ingredient of many machine learning methods, particularly appropriate for large-scale learning.
On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent
Constant step-size Stochastic Gradient Descent exhibits two phases: a transient phase during which iterates make fast progress towards the optimum, followed by a stationary phase during which iterates oscillate around the optimal point.
Bidirectional compression in heterogeneous settings for distributed or federated learning with partial participation: tight convergence guarantees
We introduce a framework - Artemis - to tackle the problem of learning in a distributed or federated setting with communication constraints and device partial participation.
Communication trade-offs for Local-SGD with large step size
Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning.
Communication trade-offs for synchronized distributed SGD with large step size
Synchronous mini-batch SGD is state-of-the-art for large-scale distributed machine learning.
Unsupervised Scalable Representation Learning for Multivariate Time Series
Time series constitute a challenging data type for machine learning algorithms, due to their highly variable lengths and sparse labeling in practice.
Context Mover's Distance & Barycenters: Optimal Transport of Contexts for Building Representations
We present a framework for building unsupervised representations of entities and their compositions, where each entity is viewed as a probability distribution rather than a vector embedding.
Wasserstein is all you need
We propose a unified framework for building unsupervised representations of individual objects or entities (and their compositions), by associating with each object both a distributional as well as a point estimate (vector embedding).
Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chains
We consider the minimization of an objective function given access to unbiased estimates of its gradient through stochastic gradient descent (SGD) with constant step-size.
Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression
We consider the optimization of a quadratic objective function whose gradients are only accessible through a stochastic oracle that returns the gradient at any given point plus a zero-mean finite variance random error.
