Search Results for author:
Found 17 papers, 9 papers with code
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
We introduce RecurrentGemma, a family of open language models which uses Google's novel Griffin architecture.
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
Recurrent neural networks (RNNs) have fast inference and scale efficiently on long sequences, but they are difficult to train and hard to scale.
Applications of flow models to the generation of correlated lattice QCD ensembles
Machine-learned normalizing flows can be used in the context of lattice quantum field theory to generate statistically correlated ensembles of lattice gauge fields at different action parameters.
Normalizing flows for lattice gauge theory in arbitrary space-time dimension
Applications of normalizing flows to the sampling of field configurations in lattice gauge theory have so far been explored almost exclusively in two space-time dimensions.
Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation
Skip connections and normalisation layers form two standard architectural components that are ubiquitous for the training of Deep Neural Networks (DNNs), but whose precise roles are poorly understood.
Aspects of scaling and scalability for flow-based sampling of lattice QCD
Recent applications of machine-learned normalizing flows to sampling in lattice field theory suggest that such methods may be able to mitigate critical slowing down and topological freezing.
Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers
However, this method (called Deep Kernel Shaping) isn't fully compatible with ReLUs, and produces networks that overfit significantly more than ResNets on ImageNet.
SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision
Using SyMetric, we identify a set of architectural choices that significantly improve the performance of a previously proposed model for inferring latent dynamics from pixels, the Hamiltonian Generative Network (HGN).
Which priors matter? Benchmarking models for learning latent dynamics
Learning dynamics is at the heart of many important applications of machine learning (ML), such as robotics and autonomous driving.
Better, Faster Fermionic Neural Networks
The Fermionic Neural Network (FermiNet) is a recently-developed neural network architecture that can be used as a wavefunction Ansatz for many-electron systems, and has already demonstrated high accuracy on small systems.
Disentangling by Subspace Diffusion
We present a novel nonparametric algorithm for symmetry-based disentangling of data manifolds, the Geometric Manifold Component Estimator (GEOMANCER).
Hamiltonian Generative Networks
The Hamiltonian formalism plays a central role in classical and quantum physics.
Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting
In order to make our method scalable, we leverage recent block-diagonal Kronecker factored approximations to the curvature.
A Scalable Laplace Approximation for Neural Networks
Pytorch implementations of Bayes By Backprop, MC Dropout, SGLD, the Local Reparametrization Trick, KF-Laplace and more
Practical Gauss-Newton Optimisation for Deep Learning
We present an efficient block-diagonal ap- proximation to the Gauss-Newton matrix for feedforward neural networks.
Nesterov's Accelerated Gradient and Momentum as approximations to Regularised Update Descent
We present a unifying framework for adapting the update direction in gradient-based iterative optimization methods.
Dealing with a large number of classes -- Likelihood, Discrimination or Ranking?
We consider training probabilistic classifiers in the case of a large number of classes.
