Search Results for author: Petar Veličković

Found 65 papers, 23 papers with code

Categorical Deep Learning: An Algebraic Theory of Architectures

no code implementations23 Feb 2024 Bruno Gavranović, Paul Lessard, Andrew Dudzik, Tamara von Glehn, João G. M. Araújo, Petar Veličković

We present our position on the elusive quest for a general-purpose framework for specifying and studying deep learning architectures.

Position

Finding Increasingly Large Extremal Graphs with AlphaZero and Tabu Search

no code implementations6 Nov 2023 Abbas Mehrabian, Ankit Anand, Hyunjik Kim, Nicolas Sonnerat, Matej Balog, Gheorghe Comanici, Tudor Berariu, Andrew Lee, Anian Ruoss, Anna Bulanova, Daniel Toyama, Sam Blackwell, Bernardino Romera Paredes, Petar Veličković, Laurent Orseau, Joonkyung Lee, Anurag Murty Naredla, Doina Precup, Adam Zsolt Wagner

This work studies a central extremal graph theory problem inspired by a 1975 conjecture of Erd\H{o}s, which aims to find graphs with a given size (number of nodes) that maximize the number of edges without having 3- or 4-cycles.

Decision Making Graph Generation

Neural Priority Queues for Graph Neural Networks

no code implementations18 Jul 2023 Rishabh Jain, Petar Veličković, Pietro Liò

Graph Neural Networks (GNNs) have shown considerable success in neural algorithmic reasoning.

Graph Regression

Latent Space Representations of Neural Algorithmic Reasoners

1 code implementation17 Jul 2023 Vladimir V. Mirjanić, Razvan Pascanu, Petar Veličković

Neural Algorithmic Reasoning (NAR) is a research area focused on designing neural architectures that can reliably capture classical computation, usually by learning to execute algorithms.

Learning to Execute

Recursive Algorithmic Reasoning

1 code implementation1 Jul 2023 Jonas Jürß, Dulhan Jayalath, Petar Veličković

Learning models that execute algorithms can enable us to address a key problem in deep learning: generalizing to out-of-distribution data.

Asynchronous Algorithmic Alignment with Cocycles

no code implementations27 Jun 2023 Andrew Dudzik, Tamara von Glehn, Razvan Pascanu, Petar Veličković

In this work, we explicitly separate the concepts of node state update and message function invocation.

How does over-squashing affect the power of GNNs?

no code implementations6 Jun 2023 Francesco Di Giovanni, T. Konstantin Rusch, Michael M. Bronstein, Andreea Deac, Marc Lackenby, Siddhartha Mishra, Petar Veličković

In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity.

Geometric Epitope and Paratope Prediction

no code implementations28 May 2023 Marco Pegoraro, Clémentine Dominé, Emanuele Rodolà, Petar Veličković, Andreea Deac

Antibody-antigen interactions play a crucial role in identifying and neutralizing harmful foreign molecules.

Neural Algorithmic Reasoning with Causal Regularisation

no code implementations20 Feb 2023 Beatrice Bevilacqua, Kyriacos Nikiforou, Borja Ibarz, Ioana Bica, Michela Paganini, Charles Blundell, Jovana Mitrovic, Petar Veličković

We evaluate our method on the CLRS algorithmic reasoning benchmark, where we show up to 3$\times$ improvements on the OOD test data.

Data Augmentation

Dual Algorithmic Reasoning

no code implementations9 Feb 2023 Danilo Numeroso, Davide Bacciu, Petar Veličković

We demonstrate that simultaneously learning the dual definition of these optimisation problems in algorithmic learning allows for better learning and qualitatively better solutions.

Time-Warping Invariant Quantum Recurrent Neural Networks via Quantum-Classical Adaptive Gating

no code implementations19 Jan 2023 Ivana Nikoloska, Osvaldo Simeone, Leonardo Banchi, Petar Veličković

Adaptive gating plays a key role in temporal data processing via classical recurrent neural networks (RNN), as it facilitates retention of past information necessary to predict the future, providing a mechanism that preserves invariance to time warping transformations.

Learnable Commutative Monoids for Graph Neural Networks

no code implementations16 Dec 2022 Euan Ong, Petar Veličković

And with this, we construct an aggregator of $O(\log V)$ depth, yielding exponential improvements for both parallelism and dependency length while achieving performance competitive with recurrent aggregators.

Learning Graph Search Heuristics

no code implementations Learning on Graphs 2022 Michal Pándy, Weikang Qiu, Gabriele Corso, Petar Veličković, Rex Ying, Jure Leskovec, Pietro Liò

At training time, we aggregate datasets of search trajectories and ground-truth shortest path distances, which we use to train a specialized graph neural network-based heuristic function using backpropagation through steps of the pathfinding process.

Graph Representation Learning Imitation Learning

Continuous Neural Algorithmic Planners

no code implementations29 Nov 2022 Yu He, Petar Veličković, Pietro Liò, Andreea Deac

Neural algorithmic reasoning studies the problem of learning algorithms with neural networks, especially with graph architectures.

Continuous Control

Expander Graph Propagation

no code implementations6 Oct 2022 Andreea Deac, Marc Lackenby, Petar Veličković

Deploying graph neural networks (GNNs) on whole-graph classification or regression tasks is known to be challenging: it often requires computing node features that are mindful of both local interactions in their neighbourhood and the global context of the graph structure.

Graph Classification Graph Representation Learning +1

Sheaf Neural Networks with Connection Laplacians

no code implementations17 Jun 2022 Federico Barbero, Cristian Bodnar, Haitz Sáez de Ocáriz Borde, Michael Bronstein, Petar Veličković, Pietro Liò

A Sheaf Neural Network (SNN) is a type of Graph Neural Network (GNN) that operates on a sheaf, an object that equips a graph with vector spaces over its nodes and edges and linear maps between these spaces.

Node Classification

Utility of Equivariant Message Passing in Cortical Mesh Segmentation

1 code implementation7 Jun 2022 Dániel Unyi, Ferdinando Insalata, Petar Veličković, Bálint Gyires-Tóth

Our evaluation shows that GNNs outperform EGNNs on aligned meshes, due to their ability to leverage the presence of a global coordinate system.

Segmentation

The CLRS Algorithmic Reasoning Benchmark

1 code implementation31 May 2022 Petar Veličković, Adrià Puigdomènech Badia, David Budden, Razvan Pascanu, Andrea Banino, Misha Dashevskiy, Raia Hadsell, Charles Blundell

Learning representations of algorithms is an emerging area of machine learning, seeking to bridge concepts from neural networks with classical algorithms.

Learning to Execute

Learning heuristics for A*

no code implementations11 Apr 2022 Danilo Numeroso, Davide Bacciu, Petar Veličković

At training time, we exploit multi-task learning to learn jointly the Dijkstra's algorithm and a consistent heuristic function for the A* search algorithm.

Multi-Task Learning

Graph Neural Networks are Dynamic Programmers

no code implementations29 Mar 2022 Andrew Dudzik, Petar Veličković

Recent advances in neural algorithmic reasoning with graph neural networks (GNNs) are propped up by the notion of algorithmic alignment.

Abstract Algebra Learning to Execute

Message passing all the way up

no code implementations22 Feb 2022 Petar Veličković

The message passing framework is the foundation of the immense success enjoyed by graph neural networks (GNNs) in recent years.

Graph Representation Learning

Neural Algorithmic Reasoners are Implicit Planners

no code implementations NeurIPS 2021 Andreea Deac, Petar Veličković, Ognjen Milinković, Pierre-Luc Bacon, Jian Tang, Mladen Nikolić

We find that prior approaches either assume that the environment is provided in such a tabular form -- which is highly restrictive -- or infer "local neighbourhoods" of states to run value iteration over -- for which we discover an algorithmic bottleneck effect.

Self-Supervised Learning

Neural Distance Embeddings for Biological Sequences

1 code implementation NeurIPS 2021 Gabriele Corso, Rex Ying, Michal Pándy, Petar Veličković, Jure Leskovec, Pietro Liò

The development of data-dependent heuristics and representations for biological sequences that reflect their evolutionary distance is critical for large-scale biological research.

Multiple Sequence Alignment

Relating Graph Neural Networks to Structural Causal Models

no code implementations9 Sep 2021 Matej Zečević, Devendra Singh Dhami, Petar Veličković, Kristian Kersting

Causality can be described in terms of a structural causal model (SCM) that carries information on the variables of interest and their mechanistic relations.

Causal Inference

ETA Prediction with Graph Neural Networks in Google Maps

no code implementations25 Aug 2021 Austin Derrow-Pinion, Jennifer She, David Wong, Oliver Lange, Todd Hester, Luis Perez, Marc Nunkesser, Seongjae Lee, Xueying Guo, Brett Wiltshire, Peter W. Battaglia, Vishal Gupta, Ang Li, Zhongwen Xu, Alvaro Sanchez-Gonzalez, Yujia Li, Petar Veličković

Travel-time prediction constitutes a task of high importance in transportation networks, with web mapping services like Google Maps regularly serving vast quantities of travel time queries from users and enterprises alike.

Graph Representation Learning

Algorithmic Concept-based Explainable Reasoning

1 code implementation15 Jul 2021 Dobrik Georgiev, Pietro Barbiero, Dmitry Kazhdan, Petar Veličković, Pietro Liò

Recent research on graph neural network (GNN) models successfully applied GNNs to classical graph algorithms and combinatorial optimisation problems.

Neural message passing for joint paratope-epitope prediction

no code implementations31 May 2021 Alice Del Vecchio, Andreea Deac, Pietro Liò, Petar Veličković

Antibodies are proteins in the immune system which bind to antigens to detect and neutralise them.

Neural Algorithmic Reasoning

no code implementations6 May 2021 Petar Veličković, Charles Blundell

Algorithms have been fundamental to recent global technological advances and, in particular, they have been the cornerstone of technical advances in one field rapidly being applied to another.

Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

5 code implementations27 Apr 2021 Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković

The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods.

Protein Folding

Persistent Message Passing

no code implementations ICLR Workshop GTRL 2021 Heiko Strathmann, Mohammadamin Barekatain, Charles Blundell, Petar Veličković

Graph neural networks (GNNs) are a powerful inductive bias for modelling algorithmic reasoning procedures and data structures.

Inductive Bias

Large-Scale Representation Learning on Graphs via Bootstrapping

3 code implementations ICLR 2022 Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Mehdi Azabou, Eva L. Dyer, Rémi Munos, Petar Veličković, Michal Valko

To address these challenges, we introduce Bootstrapped Graph Latents (BGRL) - a graph representation learning method that learns by predicting alternative augmentations of the input.

Contrastive Learning Graph Representation Learning +1

Predicting Patient Outcomes with Graph Representation Learning

1 code implementation11 Jan 2021 Emma Rocheteau, Catherine Tong, Petar Veličković, Nicholas Lane, Pietro Liò

Recent work on predicting patient outcomes in the Intensive Care Unit (ICU) has focused heavily on the physiological time series data, largely ignoring sparse data such as diagnoses and medications.

Graph Representation Learning Length-of-Stay prediction +3

A step towards neural genome assembly

1 code implementation NeurIPS Workshop LMCA 2020 Lovro Vrček, Petar Veličković, Mile Šikić

De novo genome assembly focuses on finding connections between a vast amount of short sequences in order to reconstruct the original genome.

Graph Representation Learning

XLVIN: eXecuted Latent Value Iteration Nets

no code implementations25 Oct 2020 Andreea Deac, Petar Veličković, Ognjen Milinković, Pierre-Luc Bacon, Jian Tang, Mladen Nikolić

Value Iteration Networks (VINs) have emerged as a popular method to incorporate planning algorithms within deep reinforcement learning, enabling performance improvements on tasks requiring long-range reasoning and understanding of environment dynamics.

Graph Representation Learning Self-Supervised Learning

Hierachial Protein Function Prediction with Tails-GNNs

no code implementations24 Jul 2020 Stefan Spalević, Petar Veličković, Jovana Kovačević, Mladen Nikolić

Protein function prediction may be framed as predicting subgraphs (with certain closure properties) of a directed acyclic graph describing the hierarchy of protein functions.

Inductive Bias Protein Function Prediction

Pointer Graph Networks

no code implementations NeurIPS 2020 Petar Veličković, Lars Buesing, Matthew C. Overlan, Razvan Pascanu, Oriol Vinyals, Charles Blundell

This static input structure is often informed purely by insight of the machine learning practitioner, and might not be optimal for the actual task the GNN is solving.

The PlayStation Reinforcement Learning Environment (PSXLE)

1 code implementation12 Dec 2019 Carlos Purves, Cătălina Cangea, Petar Veličković

We propose a new benchmark environment for evaluating Reinforcement Learning (RL) algorithms: the PlayStation Learning Environment (PSXLE), a PlayStation emulator modified to expose a simple control API that enables rich game-state representations.

OpenAI Gym reinforcement-learning +1

Neural Execution of Graph Algorithms

no code implementations ICLR 2020 Petar Veličković, Rex Ying, Matilde Padovano, Raia Hadsell, Charles Blundell

Graph Neural Networks (GNNs) are a powerful representational tool for solving problems on graph-structured inputs.

Drug-Drug Adverse Effect Prediction with Graph Co-Attention

1 code implementation2 May 2019 Andreea Deac, Yu-Hsiang Huang, Petar Veličković, Pietro Liò, Jian Tang

Complex or co-existing diseases are commonly treated using drug combinations, which can lead to higher risk of adverse side effects.

Spatio-Temporal Deep Graph Infomax

no code implementations12 Apr 2019 Felix L. Opolka, Aaron Solomon, Cătălina Cangea, Petar Veličković, Pietro Liò, R. Devon Hjelm

Spatio-temporal graphs such as traffic networks or gene regulatory systems present challenges for the existing deep learning methods due to the complexity of structural changes over time.

Representation Learning Traffic Prediction

ChronoMID - Cross-Modal Neural Networks for 3-D Temporal Medical Imaging Data

no code implementations12 Jan 2019 Alexander G. Rakowski, Petar Veličković, Enrico Dall'Ara, Pietro Liò

ChronoMID builds on the success of cross-modal convolutional neural networks (X-CNNs), making the novel application of the technique to medical imaging data.

General Classification

Towards Sparse Hierarchical Graph Classifiers

1 code implementation3 Nov 2018 Cătălina Cangea, Petar Veličković, Nikola Jovanović, Thomas Kipf, Pietro Liò

Recent advances in representation learning on graphs, mainly leveraging graph convolutional networks, have brought a substantial improvement on many graph-based benchmark tasks.

General Classification Graph Classification +3

Deep Graph Infomax

11 code implementations ICLR 2019 Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, R. Devon Hjelm

We present Deep Graph Infomax (DGI), a general approach for learning node representations within graph-structured data in an unsupervised manner.

General Classification Node Classification

Attentive cross-modal paratope prediction

no code implementations12 Jun 2018 Andreea Deac, Petar Veličković, Pietro Sormanni

Antibodies are a critical part of the immune system, having the function of directly neutralising or tagging undesirable objects (the antigens) for future destruction.

Computational Efficiency

Automatic Inference of Cross-modal Connection Topologies for X-CNNs

1 code implementation2 May 2018 Laurynas Karazija, Petar Veličković, Pietro Liò

The base approach learns the topology in a data-driven manner, by using measurements performed on the base CNN and supplied data.

Quantifying the Effects of Enforcing Disentanglement on Variational Autoencoders

1 code implementation24 Nov 2017 Momchil Peychev, Petar Veličković, Pietro Liò

In this paper we quantify the effects of the parameter $\beta$ on the model performance and disentanglement.

Disentanglement

Graph Attention Networks

90 code implementations ICLR 2018 Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio

We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations.

 Ranked #1 on Node Classification on Pubmed (Validation metric)

Document Classification Graph Attention +8

XFlow: Cross-modal Deep Neural Networks for Audiovisual Classification

1 code implementation2 Sep 2017 Cătălina Cangea, Petar Veličković, Pietro Liò

Our work improves on existing multimodal deep learning algorithms in two essential ways: (1) it presents a novel method for performing cross-modality (before features are learned from individual modalities) and (2) extends the previously proposed cross-connections which only transfer information between streams that process compatible data.

Classification General Classification +2

X-CNN: Cross-modal Convolutional Neural Networks for Sparse Datasets

no code implementations1 Oct 2016 Petar Veličković, Duo Wang, Nicholas D. Lane, Pietro Liò

In this paper we propose cross-modal convolutional neural networks (X-CNNs), a novel biologically inspired type of CNN architectures, treating gradient descent-specialised CNNs as individual units of processing in a larger-scale network topology, while allowing for unconstrained information flow and/or weight sharing between analogous hidden layers of the network---thus generalising the already well-established concept of neural network ensembles (where information typically may flow only between the output layers of the individual networks).

Data Augmentation

Cannot find the paper you are looking for? You can Submit a new open access paper.