Search Results for author:
Found 15 papers, 1 papers with code
The twin peaks of learning neural networks
Recent works demonstrated the existence of a double-descent phenomenon for the generalization error of neural networks, where highly overparameterized models escape overfitting and achieve good test performance, at odds with the standard bias-variance trade-off described by statistical learning theory.
The star-shaped space of solutions of the spherical negative perceptron
Empirical studies on the landscape of neural networks have shown that low-energy configurations are often found in complex connected structures, where zero-energy paths between pairs of distant solutions can be constructed.
Optimal transfer protocol by incremental layer defrosting
The simplest transfer learning protocol is based on ``freezing" the feature-extractor layers of a network pre-trained on a data-rich source task, and then adapting only the last layers to a data-poor target task.
Bias-inducing geometries: an exactly solvable data model with fairness implications
Then, we consider a novel mitigation strategy based on a matched inference approach, consisting in the introduction of coupled learning models.
An Analytical Theory of Curriculum Learning in Teacher-Student Networks
To study the former, we provide an exact description of the online learning setting, confirming the long-standing experimental observation that curricula can modestly speed up learning.
Probing transfer learning with a model of synthetic correlated datasets
Transfer learning can significantly improve the sample efficiency of neural networks, by exploiting the relatedness between a data-scarce target task and a data-abundant source task.
Solvable Model for Inheriting the Regularization through Knowledge Distillation
In recent years the empirical success of transfer learning with neural networks has stimulated an increasing interest in obtaining a theoretical understanding of its core properties.
Large deviations for the perceptron model and consequences for active learning
These large deviations then provide optimal achievable performance boundaries for any active learning algorithm.
Generalized Approximate Survey Propagation for High-Dimensional Estimation
In Generalized Linear Estimation (GLE) problems, we seek to estimate a signal that is observed through a linear transform followed by a component-wise, possibly nonlinear and noisy, channel.
Gaussian Process Prior Variational Autoencoders
In this work, we introduce a new model, the Gaussian Process (GP) Prior Variational Autoencoder (GPPVAE), to specifically address this issue.
On the role of synaptic stochasticity in training low-precision neural networks
Stochasticity and limited precision of synaptic weights in neural network models are key aspects of both biological and hardware modeling of learning processes.
Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes
We define a novel measure, which we call the "robust ensemble" (RE), which suppresses trapping by isolated configurations and amplifies the role of these dense regions.
Learning may need only a few bits of synaptic precision
Learning in neural networks poses peculiar challenges when using discretized rather then continuous synaptic states.
Local entropy as a measure for sampling solutions in Constraint Satisfaction Problems
We introduce a novel Entropy-driven Monte Carlo (EdMC) strategy to efficiently sample solutions of random Constraint Satisfaction Problems (CSPs).
Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses
We also show that the dense regions are surprisingly accessible by simple learning protocols, and that these synaptic configurations are robust to perturbations and generalize better than typical solutions.
