Search Results for author:
Found 28 papers, 17 papers with code
Optimizing Data Distribution and Kernel Performance for Efficient Training of Chemistry Foundation Models: A Case Study with MACE
For the training phase, we identify symmetric tensor contraction as the key computational kernel in MACE and optimize this kernel to improve the overall performance.
Proteina: Scaling Flow-based Protein Structure Generative Models
Here, we develop Proteina, a new large-scale flow-based protein backbone generator that utilizes hierarchical fold class labels for conditioning and relies on a tailored scalable transformer architecture with up to 5x as many parameters as previous models.
BioNeMo Framework: a modular, high-performance library for AI model development in drug discovery
We introduce the BioNeMo Framework to facilitate the training of computational biology and chemistry AI models across hundreds of GPUs.
EquiJump: Protein Dynamics Simulation via SO(3)-Equivariant Stochastic Interpolants
Our approach unifies diverse sampling methods and is benchmarked against existing models on trajectory data of fast folding proteins.
Symphony: Symmetry-Equivariant Point-Centered Spherical Harmonics for 3D Molecule Generation
We present Symphony, an $E(3)$-equivariant autoregressive generative model for 3D molecular geometries that iteratively builds a molecule from molecular fragments.
Ranked #4 on
Unconditional Molecule Generation
on QM9
Ophiuchus: Scalable Modeling of Protein Structures through Hierarchical Coarse-graining SO(3)-Equivariant Autoencoders
Three-dimensional native states of natural proteins display recurring and hierarchical patterns.
Leveraging SO(3)-steerable convolutions for pose-robust semantic segmentation in 3D medical data
Convolutional neural networks (CNNs) allow for parameter sharing and translational equivariance by using convolutional kernels in their linear layers.
Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning
They show that SGD noise can be detrimental or instead useful depending on the training regime.
e3nn: Euclidean Neural Networks
We present e3nn, a generalized framework for creating E(3) equivariant trainable functions, also known as Euclidean neural networks.
How memory architecture affects learning in a simple POMDP: the two-hypothesis testing problem
Reinforcement learning is generally difficult for partially observable Markov decision processes (POMDPs), which occurs when the agent's observation is partial or noisy.
SE(3)-equivariant prediction of molecular wavefunctions and electronic densities
Machine learning has enabled the prediction of quantum chemical properties with high accuracy and efficiency, allowing to bypass computationally costly ab initio calculations.
Relative stability toward diffeomorphisms indicates performance in deep nets
Understanding why deep nets can classify data in large dimensions remains a challenge.
E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentials
This work presents Neural Equivariant Interatomic Potentials (NequIP), an E(3)-equivariant neural network approach for learning interatomic potentials from ab-initio calculations for molecular dynamics simulations.
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training
In this manuscript, we review recent results elucidating (i, ii) and the perspective they offer on the (still unexplained) curse of dimensionality paradox.
Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties
Equivariant neural networks (ENNs) are graph neural networks embedded in $\mathbb{R}^3$ and are well suited for predicting molecular properties.
Geometric compression of invariant manifolds in neural nets
We confirm these predictions both for a one-hidden layer FC network trained on the stripe model and for a 16-layers CNN trained on MNIST, for which we also find $\beta_\text{Feature}>\beta_\text{Lazy}$.
Finding Symmetry Breaking Order Parameters with Euclidean Neural Networks
Curie's principle states that "when effects show certain asymmetry, this asymmetry must be found in the causes that gave rise to them".
Disentangling feature and lazy training in deep neural networks
Two distinct limits for deep learning have been derived as the network width $h\rightarrow \infty$, depending on how the weights of the last layer scale with $h$.
Asymptotic learning curves of kernel methods: empirical data v.s. Teacher-Student paradigm
We extract $a$ from real data by performing kernel PCA, leading to $\beta\approx0. 36$ for MNIST and $\beta\approx0. 07$ for CIFAR10, in good agreement with observations.
Scaling description of generalization with number of parameters in deep learning
At this threshold, we argue that $\|f_{N}\|$ diverges.
A General Theory of Equivariant CNNs on Homogeneous Spaces
Feature maps in these networks represent fields on a homogeneous base space, and layers are equivariant maps between spaces of fields.
A jamming transition from under- to over-parametrization affects loss landscape and generalization
We argue that in fully-connected networks a phase transition delimits the over- and under-parametrized regimes where fitting can or cannot be achieved.
The jamming transition as a paradigm to understand the loss landscape of deep neural networks
In the vicinity of this transition, properties of the curvature of the minima of the loss are critical.
3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
We prove that equivariant convolutions are the most general equivariant linear maps between fields over R^3.
Comparing Dynamics: Deep Neural Networks versus Glassy Systems
We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems.
Intertwiners between Induced Representations (with Applications to the Theory of Equivariant Neural Networks)
In algebraic terms, the feature spaces in regular G-CNNs transform according to a regular representation of the group G, whereas the feature spaces in Steerable G-CNNs transform according to the more general induced representations of G. In order to make the network equivariant, each layer in a G-CNN is required to intertwine between the induced representations associated with its input and output space.
Spherical CNNs
Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images.
Convolutional Networks for Spherical Signals
Many areas of science and egineering deal with signals with other symmetries, such as rotation invariant data on the sphere.
