Search Results for author:
Found 37 papers, 19 papers with code
Graph neural networks and non-commuting operators
Our theoretical results extend well-known transferability theorems for GNNs to the case of several simultaneous graphs (GtNNs) and provide a strict improvement on what is currently known even in the GNN case.
Thinner Latent Spaces: Detecting dimension and imposing invariance through autoencoder gradient constraints
Conformal Autoencoders are a neural network architecture that imposes orthogonality conditions between the gradients of latent variables towards achieving disentangled representations of data.
Learning equivariant tensor functions with applications to sparse vector recovery
Our numerical results show that the proposed equivariant machine learning models can learn spectral methods that outperform the best theoretically known spectral methods in some regimes.
Is machine learning good or bad for the natural sciences?
The question in the title is being asked of all of the natural sciences; that is, we are calling on the scientific communities to take a step back and consider the role and value of ML in their fields; the (partial) answers we give here come from the particular perspective of physics.
A Galois theorem for machine learning: Functions on symmetric matrices and point clouds via lightweight invariant features
In this work, we present a mathematical formulation for machine learning of (1) functions on symmetric matrices that are invariant with respect to the action of permutations by conjugation, and (2) functions on point clouds that are invariant with respect to rotations, reflections, and permutations of the points.
Approximately Equivariant Graph Networks
However, these two symmetries are fundamentally different: The translation equivariance of CNNs corresponds to symmetries of the fixed domain acting on the image signals (sometimes known as active symmetries), whereas in GNNs any permutation acts on both the graph signals and the graph domain (sometimes described as passive symmetries).
Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning
Specifically, in the contrastive learning setting, we introduce an equivariance objective and theoretically prove that its minima forces augmentations on input space to correspond to rotations on the spherical embedding space.
Fine-grained Expressivity of Graph Neural Networks
In particular, we characterize the expressive power of MPNNs in terms of the tree distance, which is a graph distance based on the concept of fractional isomorphisms, and substructure counts via tree homomorphisms, showing that these concepts have the same expressive power as the $1$-WL and MPNNs on graphons.
GeometricImageNet: Extending convolutional neural networks to vector and tensor images
We use representation theory to quantify the dimension of the space of equivariant polynomial functions on 2-dimensional vector images.
Towards fully covariant machine learning
We discuss links to causal modeling, and argue that the implementation of passive symmetries is particularly valuable when the goal of the learning problem is to generalize out of sample.
Sketch-and-solve approaches to k-means clustering by semidefinite programming
This lower bound is data-driven; it does not make any assumption on the data nor how it is generated.
A Spectral Analysis of Graph Neural Networks on Dense and Sparse Graphs
In this work we propose a random graph model that can produce graphs at different levels of sparsity.
Deep Learning is Provably Robust to Symmetric Label Noise
We conjecture that for general label noise, mitigation strategies that make use of the noisy data will outperform those that ignore the noisy data.
Shuffled linear regression through graduated convex relaxation
The shuffled linear regression problem aims to recover linear relationships in datasets where the correspondence between input and output is unknown.
Machine learning and invariant theory
Inspired by constraints from physical law, equivariant machine learning restricts the learning to a hypothesis class where all the functions are equivariant with respect to some group action.
From Local to Global: Spectral-Inspired Graph Neural Networks
Graph Neural Networks (GNNs) are powerful deep learning methods for Non-Euclidean data.
MarkerMap: nonlinear marker selection for single-cell studies
Single-cell RNA-seq data allow the quantification of cell type differences across a growing set of biological contexts.
Dimensionless machine learning: Imposing exact units equivariance
Units equivariance (or units covariance) is the exact symmetry that follows from the requirement that relationships among measured quantities of physics relevance must obey self-consistent dimensional scalings.
Prospective Learning: Principled Extrapolation to the Future
We conjecture that certain sequences of tasks are not retrospectively learnable (in which the data distribution is fixed), but are prospectively learnable (in which distributions may be dynamic), suggesting that prospective learning is more difficult in kind than retrospective learning.
A Short Tutorial on The Weisfeiler-Lehman Test And Its Variants
Graph neural networks are designed to learn functions on graphs.
A simple equivariant machine learning method for dynamics based on scalars
Physical systems obey strict symmetry principles.
Scalars are universal: Equivariant machine learning, structured like classical physics
There has been enormous progress in the last few years in designing neural networks that respect the fundamental symmetries and coordinate freedoms of physical law.
Fitting very flexible models: Linear regression with large numbers of parameters
We emphasize that it is often valuable to choose far more parameters than data points, despite folk rules to the contrary: Suitably regularized models with enormous numbers of parameters generalize well and make good predictions for held-out data; over-fitting is not (mainly) a problem of having too many parameters.
Dimensionality reduction, regularization, and generalization in overparameterized regressions
This realization brought back the study of linear models for regression, including ordinary least squares (OLS), which, like deep learning, shows a "double-descent" behavior: (1) The risk (expected out-of-sample prediction error) can grow arbitrarily when the number of parameters $p$ approaches the number of samples $n$, and (2) the risk decreases with $p$ for $p>n$, sometimes achieving a lower value than the lowest risk for $p<n$.
Can Graph Neural Networks Count Substructures?
We also prove positive results for k-WL and k-IGNs as well as negative results for k-WL with a finite number of iterations.
MREC: a fast and versatile framework for aligning and matching point clouds with applications to single cell molecular data
Comparing and aligning large datasets is a pervasive problem occurring across many different knowledge domains.
Experimental performance of graph neural networks on random instances of max-cut
In particular we consider Graph Neural Networks (GNNs) -- a class of neural networks designed to learn functions on graphs -- and we apply them to the max-cut problem on random regular graphs.
On the equivalence between graph isomorphism testing and function approximation with GNNs
We further develop a framework of the expressive power of GNNs that incorporates both of these viewpoints using the language of sigma-algebra, through which we compare the expressive power of different types of GNNs together with other graph isomorphism tests.
Ranked #33 on
Graph Regression
on ZINC-500k
SqueezeFit: Label-aware dimensionality reduction by semidefinite programming
Given labeled points in a high-dimensional vector space, we seek a low-dimensional subspace such that projecting onto this subspace maintains some prescribed distance between points of differing labels.
SUNLayer: Stable denoising with generative networks
It has been experimentally established that deep neural networks can be used to produce good generative models for real world data.
Monte Carlo approximation certificates for k-means clustering
Efficient algorithms for $k$-means clustering frequently converge to suboptimal partitions, and given a partition, it is difficult to detect $k$-means optimality.
Revised Note on Learning Algorithms for Quadratic Assignment with Graph Neural Networks
Inverse problems correspond to a certain type of optimization problems formulated over appropriate input distributions.
A polynomial-time relaxation of the Gromov-Hausdorff distance
The Gromov-Hausdorff distance provides a metric on the set of isometry classes of compact metric spaces.
Clustering subgaussian mixtures by semidefinite programming
We introduce a model-free relax-and-round algorithm for k-means clustering based on a semidefinite relaxation due to Peng and Wei.
Probably certifiably correct k-means clustering
First, we prove that Peng and Wei's semidefinite relaxation of k-means is tight with high probability under a distribution of planted clusters called the stochastic ball model.
On the tightness of an SDP relaxation of k-means
Recently, Awasthi et al. introduced an SDP relaxation of the $k$-means problem in $\mathbb R^m$.
Relax, no need to round: integrality of clustering formulations
Under the same distributional model, the $k$-means LP relaxation fails to recover such clusters at separation as large as $\Delta = 4$.
