Search Results for author:
Found 77 papers, 30 papers with code
Amplifying human performance in combinatorial competitive programming
Recent years have seen a significant surge in complex AI systems for competitive programming, capable of performing at admirable levels against human competitors.
NAR-*ICP: Neural Execution of Classical ICP-based Pointcloud Registration Algorithms
This study explores the intersection of neural networks and classical robotics algorithms through the Neural Algorithmic Reasoning (NAR) framework, allowing to train neural networks to effectively reason like classical robotics algorithms by learning to execute them.
Round and Round We Go! What makes Rotary Positional Encodings useful?
Positional Encodings (PEs) are a critical component of Transformer-based Large Language Models (LLMs), providing the attention mechanism with important sequence-position information.
Cayley Graph Propagation
One well regarded family of bottleneck-free graphs within the mathematical community are expander graphs, with prior work$\unicode{x2014}$Expander Graph Propagation (EGP)$\unicode{x2014}$proposing the use of a well-known expander graph family$\unicode{x2014}$the Cayley graphs of the $\mathrm{SL}(2,\mathbb{Z}_n)$ special linear group$\unicode{x2014}$as a computational template for GNNs.
Positional Attention: Out-of-Distribution Generalization and Expressivity for Neural Algorithmic Reasoning
There has been a growing interest in the ability of neural networks to solve algorithmic tasks, such as arithmetic, summary statistics, and sorting.
softmax is not enough (for sharp out-of-distribution)
A key property of reasoning systems is the ability to make sharp decisions on their input data.
Recurrent Aggregators in Neural Algorithmic Reasoning
Neural algorithmic reasoning (NAR) is an emerging field that seeks to design neural networks that mimic classical algorithmic computations.
Commute-Time-Optimised Graphs for GNNs
We explore graph rewiring methods that optimise commute time.
Transformers meet Neural Algorithmic Reasoners
Transformers have revolutionized machine learning with their simple yet effective architecture.
Transformers need glasses! Information over-squashing in language tasks
We rely on a theoretical signal propagation analysis -- specifically, we analyse the representations of the last token in the final layer of the Transformer, as this is the representation used for next-token prediction.
The CLRS-Text Algorithmic Reasoning Language Benchmark
Three years ago, a similar issue was identified and rectified in the field of neural algorithmic reasoning, with the advent of the CLRS benchmark.
Temporal Graph Rewiring with Expander Graphs
Evolving relations in real-world networks are often modelled by temporal graphs.
Position: Categorical Deep Learning is an Algebraic Theory of All Architectures
We present our position on the elusive quest for a general-purpose framework for specifying and studying deep learning architectures.
Position: Topological Deep Learning is the New Frontier for Relational Learning
At the same time, this paper serves as an invitation to the scientific community to actively participate in TDL research to unlock the potential of this emerging field.
Finding Increasingly Large Extremal Graphs with AlphaZero and Tabu Search
This work studies a central extremal graph theory problem inspired by a 1975 conjecture of Erd\H{o}s, which aims to find graphs with a given size (number of nodes) that maximize the number of edges without having 3- or 4-cycles.
TacticAI: an AI assistant for football tactics
The utility of TacticAI is validated by a qualitative study conducted with football domain experts at Liverpool FC.
Half-Hop: A graph upsampling approach for slowing down message passing
Message passing neural networks have shown a lot of success on graph-structured data.
Ranked #1 on
Node Classification
on AMZ Comp
Neural Priority Queues for Graph Neural Networks
Graph Neural Networks (GNNs) have shown considerable success in neural algorithmic reasoning.
Ranked #27 on
Graph Regression
on Peptides-struct
Latent Space Representations of Neural Algorithmic Reasoners
Neural Algorithmic Reasoning (NAR) is a research area focused on designing neural architectures that can reliably capture classical computation, usually by learning to execute algorithms.
Parallel Algorithms Align with Neural Execution
Neural algorithmic reasoners are parallel processors.
Recursive Algorithmic Reasoning
Learning models that execute algorithms can enable us to address a key problem in deep learning: generalizing to out-of-distribution data.
Asynchronous Algorithmic Alignment with Cocycles
In this work, we explicitly separate the concepts of node state update and message function invocation.
How does over-squashing affect the power of GNNs?
In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity.
Geometric Epitope and Paratope Prediction
Antibody-antigen interactions play a crucial role in identifying and neutralizing harmful foreign molecules.
Neural Algorithmic Reasoning with Causal Regularisation
We evaluate our method on the CLRS algorithmic reasoning benchmark, where we show up to 3$\times$ improvements on the OOD test data.
Dual Algorithmic Reasoning
We demonstrate that simultaneously learning the dual definition of these optimisation problems in algorithmic learning allows for better learning and qualitatively better solutions.
Time-Warping Invariant Quantum Recurrent Neural Networks via Quantum-Classical Adaptive Gating
Adaptive gating plays a key role in temporal data processing via classical recurrent neural networks (RNN), as it facilitates retention of past information necessary to predict the future, providing a mechanism that preserves invariance to time warping transformations.
Everything is Connected: Graph Neural Networks
In many ways, graphs are the main modality of data we receive from nature.
Learnable Commutative Monoids for Graph Neural Networks
And with this, we construct an aggregator of $O(\log V)$ depth, yielding exponential improvements for both parallelism and dependency length while achieving performance competitive with recurrent aggregators.
Learning Graph Search Heuristics
At training time, we aggregate datasets of search trajectories and ground-truth shortest path distances, which we use to train a specialized graph neural network-based heuristic function using backpropagation through steps of the pathfinding process.
Continuous Neural Algorithmic Planners
Neural algorithmic reasoning studies the problem of learning algorithms with neural networks, especially with graph architectures.
Learning to Configure Computer Networks with Neural Algorithmic Reasoning
We present a new method for scaling automatic configuration of computer networks.
Expander Graph Propagation
Deploying graph neural networks (GNNs) on whole-graph classification or regression tasks is known to be challenging: it often requires computing node features that are mindful of both local interactions in their neighbourhood and the global context of the graph structure.
A Generalist Neural Algorithmic Learner
The cornerstone of neural algorithmic reasoning is the ability to solve algorithmic tasks, especially in a way that generalises out of distribution.
Sheaf Neural Networks with Connection Laplacians
A Sheaf Neural Network (SNN) is a type of Graph Neural Network (GNN) that operates on a sheaf, an object that equips a graph with vector spaces over its nodes and edges and linear maps between these spaces.
Ranked #10 on
Node Classification
on Wisconsin
Utility of Equivariant Message Passing in Cortical Mesh Segmentation
Our evaluation shows that GNNs outperform EGNNs on aligned meshes, due to their ability to leverage the presence of a global coordinate system.
The CLRS Algorithmic Reasoning Benchmark
Learning representations of algorithms is an emerging area of machine learning, seeking to bridge concepts from neural networks with classical algorithms.
Learning heuristics for A*
At training time, we exploit multi-task learning to learn jointly the Dijkstra's algorithm and a consistent heuristic function for the A* search algorithm.
Graph Neural Networks are Dynamic Programmers
Recent advances in neural algorithmic reasoning with graph neural networks (GNNs) are propped up by the notion of algorithmic alignment.
Message passing all the way up
The message passing framework is the foundation of the immense success enjoyed by graph neural networks (GNNs) in recent years.
Neural Algorithmic Reasoners are Implicit Planners
We find that prior approaches either assume that the environment is provided in such a tabular form -- which is highly restrictive -- or infer "local neighbourhoods" of states to run value iteration over -- for which we discover an algorithmic bottleneck effect.
Simple GNN Regularisation for 3D Molecular Property Prediction and Beyond
We introduce “Noisy Nodes”, a very simple technique for improved training of GNNs, in which we corrupt the input graph with noise, and add a noise correcting node-level loss.
Initial Structure to Relaxed Energy (IS2RE), Direct
Molecular Property Prediction
+1
Neural Distance Embeddings for Biological Sequences
The development of data-dependent heuristics and representations for biological sequences that reflect their evolutionary distance is critical for large-scale biological research.
Relating Graph Neural Networks to Structural Causal Models
Causality can be described in terms of a structural causal model (SCM) that carries information on the variables of interest and their mechanistic relations.
ETA Prediction with Graph Neural Networks in Google Maps
Travel-time prediction constitutes a task of high importance in transportation networks, with web mapping services like Google Maps regularly serving vast quantities of travel time queries from users and enterprises alike.
Large-scale graph representation learning with very deep GNNs and self-supervision
In doing so, we demonstrate evidence of scalable self-supervised graph representation learning, and utility of very deep GNNs -- both very important open issues.
Reasoning-Modulated Representations
Neural networks leverage robust internal representations in order to generalise.
Algorithmic Concept-based Explainable Reasoning
Recent research on graph neural network (GNN) models successfully applied GNNs to classical graph algorithms and combinatorial optimisation problems.
Simple GNN Regularisation for 3D Molecular Property Prediction & Beyond
From this observation we derive "Noisy Nodes", a simple technique in which we corrupt the input graph with noise, and add a noise correcting node-level loss.
Ranked #4 on
Initial Structure to Relaxed Energy (IS2RE)
on OC20
Neural message passing for joint paratope-epitope prediction
Antibodies are proteins in the immune system which bind to antigens to detect and neutralise them.
Neural Algorithmic Reasoning
Algorithms have been fundamental to recent global technological advances and, in particular, they have been the cornerstone of technical advances in one field rapidly being applied to another.
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods.
Bootstrapped Representation Learning on Graphs
Current state-of-the-art self-supervised learning methods for graph neural networks are based on contrastive learning.
Persistent Message Passing
Graph neural networks (GNNs) are a powerful inductive bias for modelling algorithmic reasoning procedures and data structures.
Combinatorial optimization and reasoning with graph neural networks
Combinatorial optimization is a well-established area in operations research and computer science.
Large-Scale Representation Learning on Graphs via Bootstrapping
To address these challenges, we introduce Bootstrapped Graph Latents (BGRL) - a graph representation learning method that learns by predicting alternative augmentations of the input.
Predicting Patient Outcomes with Graph Representation Learning
Recent work on predicting patient outcomes in the Intensive Care Unit (ICU) has focused heavily on the physiological time series data, largely ignoring sparse data such as diagnoses and medications.
A step towards neural genome assembly
De novo genome assembly focuses on finding connections between a vast amount of short sequences in order to reconstruct the original genome.
On the role of planning in model-based deep reinforcement learning
These results indicate where and how to utilize planning in reinforcement learning settings, and highlight a number of open questions for future MBRL research.
Deep Reinforcement Learning
Model-based Reinforcement Learning
+2
XLVIN: eXecuted Latent Value Iteration Nets
Value Iteration Networks (VINs) have emerged as a popular method to incorporate planning algorithms within deep reinforcement learning, enabling performance improvements on tasks requiring long-range reasoning and understanding of environment dynamics.
Deep Reinforcement Learning
Graph Representation Learning
+1
Hierachial Protein Function Prediction with Tails-GNNs
Protein function prediction may be framed as predicting subgraphs (with certain closure properties) of a directed acyclic graph describing the hierarchy of protein functions.
Pointer Graph Networks
This static input structure is often informed purely by insight of the machine learning practitioner, and might not be optimal for the actual task the GNN is solving.
Principal Neighbourhood Aggregation for Graph Nets
Graph Neural Networks (GNNs) have been shown to be effective models for different predictive tasks on graph-structured data.
Ranked #1 on
Graph Regression
on KIT
The PlayStation Reinforcement Learning Environment (PSXLE)
We propose a new benchmark environment for evaluating Reinforcement Learning (RL) algorithms: the PlayStation Learning Environment (PSXLE), a PlayStation emulator modified to expose a simple control API that enables rich game-state representations.
Neural Execution of Graph Algorithms
Graph Neural Networks (GNNs) are a powerful representational tool for solving problems on graph-structured inputs.
Drug-Drug Adverse Effect Prediction with Graph Co-Attention
Complex or co-existing diseases are commonly treated using drug combinations, which can lead to higher risk of adverse side effects.
Ranked #3 on
Drug–drug Interaction Extraction
on DrugBank
Spatio-Temporal Deep Graph Infomax
Spatio-temporal graphs such as traffic networks or gene regulatory systems present challenges for the existing deep learning methods due to the complexity of structural changes over time.
ChronoMID - Cross-Modal Neural Networks for 3-D Temporal Medical Imaging Data
ChronoMID builds on the success of cross-modal convolutional neural networks (X-CNNs), making the novel application of the technique to medical imaging data.
Towards Sparse Hierarchical Graph Classifiers
Recent advances in representation learning on graphs, mainly leveraging graph convolutional networks, have brought a substantial improvement on many graph-based benchmark tasks.
Deep Graph Infomax
We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised manner.
Ranked #47 on
Node Classification
on Citeseer
Attentive cross-modal paratope prediction
Antibodies are a critical part of the immune system, having the function of directly neutralising or tagging undesirable objects (the antigens) for future destruction.
Automatic Inference of Cross-modal Connection Topologies for X-CNNs
The base approach learns the topology in a data-driven manner, by using measurements performed on the base CNN and supplied data.
Quantifying the Effects of Enforcing Disentanglement on Variational Autoencoders
In this paper we quantify the effects of the parameter $\beta$ on the model performance and disentanglement.
Graph Attention Networks
We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations.
Ranked #1 on
Heterogeneous Node Classification
on ACM (Heterogeneous Node Classification)
(Macro-F1 metric)
Cross-modal Recurrent Models for Weight Objective Prediction from Multimodal Time-series Data
We analyse multimodal time-series data corresponding to weight, sleep and steps measurements.
XFlow: Cross-modal Deep Neural Networks for Audiovisual Classification
Our work improves on existing multimodal deep learning algorithms in two essential ways: (1) it presents a novel method for performing cross-modality (before features are learned from individual modalities) and (2) extends the previously proposed cross-connections which only transfer information between streams that process compatible data.
X-CNN: Cross-modal Convolutional Neural Networks for Sparse Datasets
In this paper we propose cross-modal convolutional neural networks (X-CNNs), a novel biologically inspired type of CNN architectures, treating gradient descent-specialised CNNs as individual units of processing in a larger-scale network topology, while allowing for unconstrained information flow and/or weight sharing between analogous hidden layers of the network---thus generalising the already well-established concept of neural network ensembles (where information typically may flow only between the output layers of the individual networks).
