Search Results for author:
Found 22 papers, 15 papers with code
Benchopt: Reproducible, efficient and collaborative optimization benchmarks
Numerical validation is at the core of machine learning research as it allows to assess the actual impact of new methods, and to confirm the agreement between theory and practice.
Do Residual Neural Networks discretize Neural Ordinary Differential Equations?
As a byproduct of our analysis, we consider the use of a memory-free discrete adjoint method to train a ResNet by recovering the activations on the fly through a backward pass of the network, and show that this method theoretically succeeds at large depth if the residual functions are Lipschitz with the input.
A framework for bilevel optimization that enables stochastic and global variance reduction algorithms
However, computing the gradient of the value function involves solving a linear system, which makes it difficult to derive unbiased stochastic estimates.
Shared Independent Component Analysis for Multi-Subject Neuroimaging
While ShICA-J is based on second-order statistics, we further propose to leverage non-Gaussianity of the components using a maximum-likelihood method, ShICA-ML, that is both more accurate and more costly.
Sinkformers: Transformers with Doubly Stochastic Attention
We show that the row-wise stochastic attention matrices in classical Transformers get close to doubly stochastic matrices as the number of epochs increases, justifying the use of Sinkhorn normalization as an informative prior.
Kernel Stein Discrepancy Descent
We investigate the properties of its Wasserstein gradient flow to approximate a target probability distribution $\pi$ on $\mathbb{R}^d$, known up to a normalization constant.
Adaptive Multi-View ICA: Estimation of noise levels for optimal inference
By contrast, we propose Adaptive multiView ICA (AVICA), a noisy ICA model where each view is a linear mixture of shared independent sources with additive noise on the sources.
Momentum Residual Neural Networks
We show on CIFAR and ImageNet that Momentum ResNets have the same accuracy as ResNets, while having a much smaller memory footprint, and show that pre-trained Momentum ResNets are promising for fine-tuning models.
Ranked #119 on
Image Classification
on CIFAR-10
Fast and accurate optimization on the orthogonal manifold without retraction
We consider the problem of minimizing a function over the manifold of orthogonal matrices.
Deep orthogonal linear networks are shallow
We consider the problem of training a deep orthogonal linear network, which consists of a product of orthogonal matrices, with no non-linearity in-between.
Spectral independent component analysis with noise modeling for M/EEG source separation
Signals are modelled as a linear mixing of independent sources corrupted by additive noise, where sources and the noise are stationary Gaussian time series.
Modeling Shared Responses in Neuroimaging Studies through MultiView ICA
Group studies involving large cohorts of subjects are important to draw general conclusions about brain functional organization.
mvlearn: Multiview Machine Learning in Python
As data are generated more and more from multiple disparate sources, multiview data sets, where each sample has features in distinct views, have ballooned in recent years.
Super-efficiency of automatic differentiation for functions defined as a minimum
In most cases, the minimum has no closed-form, and an approximation is obtained via an iterative algorithm.
Manifold-regression to predict from MEG/EEG brain signals without source modeling
We show that Wasserstein and geometric distances allow perfect out-of-sample prediction on the generative models.
Learning step sizes for unfolded sparse coding
We demonstrate that for a large class of unfolded algorithms, if the algorithm converges to the solution of the Lasso, its last layers correspond to ISTA with learned step sizes.
Beyond Pham's algorithm for joint diagonalization
The approximate joint diagonalization of a set of matrices consists in finding a basis in which these matrices are as diagonal as possible.
A Quasi-Newton algorithm on the orthogonal manifold for NMF with transform learning
Nonnegative matrix factorization (NMF) is a popular method for audio spectral unmixing.
Accelerating likelihood optimization for ICA on real signals
We study optimization methods for solving the maximum likelihood formulation of independent component analysis (ICA).
Stochastic algorithms with descent guarantees for ICA
We derive an online algorithm for the streaming setting, and an incremental algorithm for the finite sum setting, with the following benefits.
Faster ICA under orthogonal constraint
Independent Component Analysis (ICA) is a technique for unsupervised exploration of multi-channel data widely used in observational sciences.
Faster independent component analysis by preconditioning with Hessian approximations
Independent Component Analysis (ICA) is a technique for unsupervised exploration of multi-channel data that is widely used in observational sciences.
