Search Results for author:
Found 57 papers, 41 papers with code
SPGNN: Recognizing Salient Subgraph Patterns via Enhanced Graph Convolution and Pooling
We propose a novel Subgraph Pattern GNN (SPGNN) architecture that incorporates these enhancements.
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models
However, LoRA approximates Delta W through the product of two matrices, A, initialized with Gaussian noise, and B, initialized with zeros, while PiSSA initializes A and B with principal singular values and vectors of the original matrix W. PiSSA can better approximate the outcomes of full-parameter fine-tuning at the beginning by changing the essential parts while freezing the "noisy" parts.
Case-Based or Rule-Based: How Do Transformers Do the Math?
Through carefully designed intervention experiments on five math tasks, we confirm that transformers are performing case-based reasoning, no matter whether scratchpad is used, which aligns with the previous observations that transformers use subgraph matching/shortcut learning to reason.
Highly Accurate Disease Diagnosis and Highly Reproducible Biomarker Identification with PathFormer
However, we found two major limitations of existing GNNs in omics data analysis, i. e., limited-prediction (diagnosis) accuracy and limited-reproducible biomarker identification capacity across multiple datasets.
On the Completeness of Invariant Geometric Deep Learning Models
Our results fill the gap in the theoretical power of invariant models, contributing to a rigorous and comprehensive understanding of their capabilities.
Latent Graph Diffusion: A Unified Framework for Generation and Prediction on Graphs
We first propose Latent Graph Diffusion (LGD), a generative model that can generate node, edge, and graph-level features of all categories simultaneously.
PyTorch Geometric High Order: A Unified Library for High Order Graph Neural Network
We introduce PyTorch Geometric High Order (PyGHO), a library for High Order Graph Neural Networks (HOGNNs) that extends PyTorch Geometric (PyG).
Chain of Images for Intuitively Reasoning
The human brain is naturally equipped to comprehend and interpret visual information rapidly.
LooGLE: Can Long-Context Language Models Understand Long Contexts?
In this paper, we present LooGLE, a Long Context Generic Language Evaluation benchmark for LLMs' long context understanding.
Facilitating Graph Neural Networks with Random Walk on Simplicial Complexes
Second, on $1$-simplices or edge level, we bridge edge-level random walk and Hodge $1$-Laplacians and design corresponding edge PE respectively.
Neural Attention: Enhancing QKV Calculation in Self-Attention Mechanism with Neural Networks
In the realm of deep learning, the self-attention mechanism has substantiated its pivotal role across a myriad of tasks, encompassing natural language processing and computer vision.
Parrot Mind: Towards Explaining the Complex Task Reasoning of Pretrained Large Language Models with Template-Content Structure
Furthermore, by generalizing this structure to the hierarchical case, we demonstrate that models can achieve task composition, further reducing the space needed to learn from linear to logarithmic, thereby effectively learning on complex reasoning involving multiple steps.
On the Stability of Expressive Positional Encodings for Graphs
Despite many attempts to address non-uniqueness, most methods overlook stability, leading to poor generalization on unseen graph structures.
Molecular Property Prediction
Out-of-Distribution Generalization
+1
One for All: Towards Training One Graph Model for All Classification Tasks
For in-context learning on graphs, OFA introduces a novel graph prompting paradigm that appends prompting substructures to the input graph, which enables it to address varied tasks without fine-tuning.
Graph Contrastive Learning Meets Graph Meta Learning: A Unified Method for Few-shot Node Tasks
In our study, we first identify two crucial advantages of contrastive learning compared to meta learning, including (1) the comprehensive utilization of graph nodes and (2) the power of graph augmentations.
Distance-Restricted Folklore Weisfeiler-Leman GNNs with Provable Cycle Counting Power
Many of the proposed GNN models with provable cycle counting power are based on subgraph GNNs, i. e., extracting a bag of subgraphs from the input graph, generating representations for each subgraph, and using them to augment the representation of the input graph.
Rethinking the Power of Graph Canonization in Graph Representation Learning with Stability
We theoretically reveal the trade-off of expressivity and stability in graph-canonization-enhanced GNNs.
CktGNN: Circuit Graph Neural Network for Electronic Design Automation
The electronic design automation of analog circuits has been a longstanding challenge in the integrated circuit field due to the huge design space and complex design trade-offs among circuit specifications.
VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs
To address this issue, we propose to learn a new powerful graph representation space by directly labeling nodes' diverse local structures for GNN-to-MLP distillation.
Extending the Design Space of Graph Neural Networks by Rethinking Folklore Weisfeiler-Lehman
We theoretically prove that even if we fix the space complexity to $O(n^k)$ (for any $k\geq 2$) in $(k, t)$-FWL, we can construct an expressiveness hierarchy up to solving the graph isomorphism problem.
Ranked #2 on
Graph Regression
on ZINC-500k
Code Prompting: a Neural Symbolic Method for Complex Reasoning in Large Language Models
We also consider the ensemble of code prompting and CoT prompting to combine the strengths of both.
Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
On the whole, our analysis provides a novel perspective on the role of semantics in developing and evaluating language models' reasoning abilities.
From Relational Pooling to Subgraph GNNs: A Universal Framework for More Expressive Graph Neural Networks
Relational pooling is a framework for building more expressive and permutation-invariant graph neural networks.
Improving Graph Neural Networks on Multi-node Tasks with Labeling Tricks
When we want to learn a node-set representation involving multiple nodes, a common practice in previous works is to directly aggregate the single-node representations obtained by a GNN.
Towards Better Evaluation of GNN Expressiveness with BREC Dataset
Research on the theoretical expressiveness of Graph Neural Networks (GNNs) has developed rapidly, and many methods have been proposed to enhance the expressiveness.
Efficiently Counting Substructures by Subgraph GNNs without Running GNN on Subgraphs
Among these works, a popular way is to use subgraph GNNs, which decompose the input graph into a collection of subgraphs and enhance the representation of the graph by applying GNN to individual subgraphs.
SUREL+: Moving from Walks to Sets for Scalable Subgraph-based Graph Representation Learning
Subgraph-based graph representation learning (SGRL) has recently emerged as a powerful tool in many prediction tasks on graphs due to its advantages in model expressiveness and generalization ability.
Ranked #1 on
Link Property Prediction
on ogbl-ppa
Time Associated Meta Learning for Clinical Prediction
We view time-associated disease prediction as classification tasks at multiple time points.
Neural Common Neighbor with Completion for Link Prediction
Despite its outstanding performance in various graph tasks, vanilla Message Passing Neural Network (MPNN) usually fails in link prediction tasks, as it only uses representations of two individual target nodes and ignores the pairwise relation between them.
Ranked #1 on
Link Property Prediction
on ogbl-ddi
RulE: Neural-Symbolic Knowledge Graph Reasoning with Rule Embedding
In this paper, we propose a novel and principled framework called \textbf{RulE} (stands for {Rul}e {E}mbedding) to effectively leverage logical rules to enhance KG reasoning.
Boosting the Cycle Counting Power of Graph Neural Networks with I$^2$-GNNs
To the best of our knowledge, it is the first linear-time GNN model that can count 6-cycles with theoretical guarantees.
1st ICLR International Workshop on Privacy, Accountability, Interpretability, Robustness, Reasoning on Structured Data (PAIR^2Struct)
Recent years have seen advances on principles and guidance relating to accountable and ethical use of artificial intelligence (AI) spring up around the globe.
Geodesic Graph Neural Network for Efficient Graph Representation Learning
The GNN embeddings of nodes on the shortest paths are used to generate geodesic representations.
Rethinking Knowledge Graph Evaluation Under the Open-World Assumption
However, evaluation of knowledge graph completion (KGC) models often ignores the incompleteness -- facts in the test set are ranked against all unknown triplets which may contain a large number of missing facts not included in the KG yet.
Graph Neural Network with Local Frame for Molecular Potential Energy Surface
Projected onto a frame, equivariant features like 3D coordinates are converted to invariant features, so that we can capture geometric information with these projections and decouple the symmetry requirement from GNN design.
Two-Dimensional Weisfeiler-Lehman Graph Neural Networks for Link Prediction
As pointed out by previous works, this two-step procedure results in low discriminating power, as 1-WL-GNNs by nature learn node-level representations instead of link-level.
How Powerful are K-hop Message Passing Graph Neural Networks
Recently, researchers extended 1-hop message passing to K-hop message passing by aggregating information from K-hop neighbors of nodes simultaneously.
How Powerful are Spectral Graph Neural Networks
We also establish a connection between the expressive power of spectral GNNs and Graph Isomorphism (GI) testing, the latter of which is often used to characterize spatial GNNs' expressive power.
3DLinker: An E(3) Equivariant Variational Autoencoder for Molecular Linker Design
The main computational challenges include: 1) the generation of linkers is conditional on the two given molecules, in contrast to generating full molecules from scratch in previous works; 2) linkers heavily depend on the anchor atoms of the two molecules to be connected, which are not known beforehand; 3) 3D structures and orientations of the molecules need to be considered to avoid atom clashes, for which equivariance to E(3) group are necessary.
PACE: A Parallelizable Computation Encoder for Directed Acyclic Graphs
In this work, we propose a Parallelizable Attention-based Computation structure Encoder (PACE) that processes nodes simultaneously and encodes DAGs in parallel.
Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks
Graph neural networks (GNN) have shown great advantages in many graph-based learning tasks but often fail to predict accurately for a task-based on sets of nodes such as link/motif prediction and so on.
Algorithm and System Co-design for Efficient Subgraph-based Graph Representation Learning
Subgraph-based graph representation learning (SGRL) has been recently proposed to deal with some fundamental challenges encountered by canonical graph neural networks (GNNs), and has demonstrated advantages in many important data science applications such as link, relation and motif prediction.
Ranked #1 on
Link Property Prediction
on ogbl-citation2
Decoupling the Depth and Scope of Graph Neural Networks
We propose a design principle to decouple the depth and scope of GNNs -- to generate representation of a target entity (i. e., a node or an edge), we first extract a localized subgraph as the bounded-size scope, and then apply a GNN of arbitrary depth on top of the subgraph.
Ranked #3 on
Node Classification
on Reddit
Network In Graph Neural Network
However, wider hidden layers can easily lead to overfitting, and incrementally adding more GNN layers can potentially result in over-smoothing. In this paper, we present a model-agnostic methodology, namely Network In Graph Neural Network (NGNN ), that allows arbitrary GNN models to increase their model capacity by making the model deeper.
Ranked #1 on
Link Property Prediction
on ogbl-citation2
Nested Graph Neural Networks
The key is to make each node representation encode a subgraph around it more than a subtree.
Ranked #9 on
Graph Property Prediction
on ogbg-molpcba
GLASS: GNN with Labeling Tricks for Subgraph Representation Learning
And training a GLASS model only takes 28% time needed for a SubGNN on average.
Principled Hyperedge Prediction with Structural Spectral Features and Neural Networks
SNALS captures the joint interactions of a hyperedge by its local environment, which is retrieved by collecting the spectrum information of their connections.
Learning Two-Time-Scale Representations For Large Scale Recommendations
We propose a surprisingly simple but effective two-time-scale (2TS) model for learning user representations for recommendation.
Deep Graph Neural Networks with Shallow Subgraph Samplers
We propose a simple "deep GNN, shallow sampler" design principle to improve both the GNN accuracy and efficiency -- to generate representation of a target node, we use a deep GNN to pass messages only within a shallow, localized subgraph.
Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning
In this paper, we provide a theory of using graph neural networks (GNNs) for multi-node representation learning (where we are interested in learning a representation for a set of more than one node, such as link).
Ranked #1 on
Link Property Prediction
on ogbl-citation2
Revisiting Graph Neural Networks for Link Prediction
Graph neural networks (GNNs) have achieved great success in recent years.
Pooling Regularized Graph Neural Network for fMRI Biomarker Analysis
We propose an interpretable GNN framework with a novel salient region selection mechanism to determine neurological brain biomarkers associated with disorders.
Inductive Matrix Completion Based on Graph Neural Networks
Under the extreme setting where not any side information is available other than the matrix to complete, can we still learn an inductive matrix completion model?
Ranked #1 on
Recommendation Systems
on Flixster Monti
D-VAE: A Variational Autoencoder for Directed Acyclic Graphs
Graph structured data are abundant in the real world.
An End-to-End Deep Learning Architecture for Graph Classification
Neural networks are typically designed to deal with data in tensor forms.
Ranked #16 on
Graph Classification
on D&D
Link Prediction Based on Graph Neural Networks
The theory unifies a wide range of heuristics in a single framework, and proves that all these heuristics can be well approximated from local subgraphs.
Ranked #1 on
Link Prediction
on USAir
Weisfeiler-lehman neural machine for link prediction
Compared with traditional link prediction methods, Wlnm does not assume a particular link formation mechanism (such as common neighbors), but learns this mechanism from the graph itself.
