Search Results for author:
Found 9 papers, 1 papers with code
Do deep neural networks have an inbuilt Occam's razor?
The remarkable performance of overparameterized deep neural networks (DNNs) must arise from an interplay between network architecture, training algorithms, and structure in the data.
Double-descent curves in neural networks: a new perspective using Gaussian processes
Double-descent curves in neural networks describe the phenomenon that the generalisation error initially descends with increasing parameters, then grows after reaching an optimal number of parameters which is less than the number of data points, but then descends again in the overparameterized regime.
Generalization bounds for deep learning
Here we introduce desiderata for techniques that predict generalization errors for deep learning models in supervised learning.
Is SGD a Bayesian sampler? Well, almost
Our main findings are that $P_{SGD}(f\mid S)$ correlates remarkably well with $P_B(f\mid S)$ and that $P_B(f\mid S)$ is strongly biased towards low-error and low complexity functions.
From genotypes to organisms: State-of-the-art and perspectives of a cornerstone in evolutionary dynamics
Understanding how genotypes map onto phenotypes, fitness, and eventually organisms is arguably the next major missing piece in a fully predictive theory of evolution.
Neural networks are a priori biased towards Boolean functions with low entropy
Understanding the inductive bias of neural networks is critical to explaining their ability to generalise.
Simplicity bias in the parameter-function map of deep neural networks
Deep neural networks can be viewed as a mapping from the space of parameters (the weights) to the space of functions (how inputs get transformed to outputs by the network).
Deep learning generalizes because the parameter-function map is biased towards simple functions
We then provide clear evidence for this strong simplicity bias in a model DNN for Boolean functions, as well as in much larger fully connected and convolutional networks applied to CIFAR10 and MNIST.
Modelling the Self-Assembly of Virus Capsids
We use computer simulations to study a model, first proposed by Wales [1], for the reversible and monodisperse self-assembly of simple icosahedral virus capsid structures.
Biomolecules
