Search Results for author:
Found 83 papers, 56 papers with code
GNNs as Predictors of Agentic Workflow Performances
Agentic workflows invoked by Large Language Models (LLMs) have achieved remarkable success in handling complex tasks.
VACT: A Video Automatic Causal Testing System and a Benchmark
With the rapid advancement of text-conditioned Video Generation Models (VGMs), the quality of generated videos has significantly improved, bringing these models closer to functioning as ``*world simulators*'' and making real-world-level video generation more accessible and cost-effective.
LIFT: Improving Long Context Understanding of Large Language Models through Long Input Fine-Tuning
This paper presents Long Input Fine-Tuning (LIFT), a novel framework for long-context modeling that can improve the long-context performance of arbitrary (short-context) LLMs by dynamically adapting model parameters based on the long input.
Training Large Language Models to be Better Rule Followers
As a result, while LLMs can often recall rules with ease, they fail to apply these rules strictly and consistently in relevant reasoning scenarios.
TransMLA: Multi-Head Latent Attention Is All You Need
In this paper, we show that GQA can always be represented by MLA while maintaining the same KV cache overhead, but the converse does not hold.
Using Random Noise Equivariantly to Boost Graph Neural Networks Universally
Recent advances in Graph Neural Networks (GNNs) have explored the potential of random noise as an input feature to enhance expressivity across diverse tasks.
Do Graph Diffusion Models Accurately Capture and Generate Substructure Distributions?
Diffusion models have gained popularity in graph generation tasks; however, the extent of their expressivity concerning the graph distributions they can learn is not fully understood.
RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?
In this work, we explore the untapped potential of scaling Long Chain-of-Thought (Long-CoT) data to 1000k samples, pioneering the development of a slow-thinking model, RedStar.
Exact Acceleration of Subgraph Graph Neural Networks by Eliminating Computation Redundancy
Many recent studies have proposed the use of graph convolution methods over the numerous subgraphs of each graph, a concept known as subgraph graph neural networks (subgraph GNNs), to enhance GNNs' ability to distinguish non-isomorphic graphs.
LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning
Long context understanding remains challenging for large language models due to their limited context windows.
GL-Fusion: Rethinking the Combination of Graph Neural Network and Large Language model
Recent research on integrating Large Language Models (LLMs) with Graph Neural Networks (GNNs) typically follows two approaches: LLM-centered models, which convert graph data into tokens for LLM processing, and GNN-centered models, which use LLMs to encode text features into node and edge representations for GNN input.
CLOVER: Cross-Layer Orthogonal Vectors Pruning and Fine-Tuning
For instance, the perplexity of pruning 70\% of the \( Q \)-\( K \) pairs in GPT-2 XL is similar to that of pruning just 8\% with vanilla methods.
Reconsidering the Performance of GAE in Link Prediction
Various graph neural networks (GNNs) with advanced training techniques and model designs have been proposed for link prediction tasks.
Ranked #1 on
Link Property Prediction
on ogbl-ppa
Number Cookbook: Number Understanding of Language Models and How to Improve It
We also finetune practical-scale LLMs on our proposed NUPA tasks and find that 1) naive finetuning can improve NUPA a lot on many but not all tasks, and 2) surprisingly, techniques designed to enhance NUPA prove ineffective for finetuning pretrained models.
Towards Stable, Globally Expressive Graph Representations with Laplacian Eigenvectors
Graph neural networks (GNNs) have achieved remarkable success in a variety of machine learning tasks over graph data.
Mars: Situated Inductive Reasoning in an Open-World Environment
In this paper, we design Mars, an interactive environment devised for situated inductive reasoning.
Geometric Representation Condition Improves Equivariant Molecule Generation
Compared to directly generating a molecule, the relatively easy-to-generate representation in the first-stage guides the second-stage generation to reach a high-quality molecule in a more goal-oriented and much faster way.
Fine-Grained Expressive Power of Weisfeiler-Leman: A Homomorphism Counting Perspective
The ability of graph neural networks (GNNs) to count homomorphisms has recently been proposed as a practical and fine-grained measure of their expressive power.
On Lexical Invariance on Multisets and Graphs
For example, multiset {1, 2, 3, 2} is equivalent to multiset {a, b, c, b} if we specify an injective transformation that maps 1 to a, 2 to b and 3 to c. We study the sufficient and necessary conditions for a most expressive lexical invariant (and permutation invariant) function on multisets and graphs, and proves that for multisets, the function must have a form that only takes the multiset of counts of the unique elements in the original multiset as input.
GOFA: A Generative One-For-All Model for Joint Graph Language Modeling
For example, current attempts at designing general graph models either transform graph data into a language format for LLM-based prediction or still train a GNN model with LLM as an assistant.
Foundations and Frontiers of Graph Learning Theory
Recent advancements in graph learning have revolutionized the way to understand and analyze data with complex structures.
TAGLAS: An atlas of text-attributed graph datasets in the era of large graph and language models
In TAGLAS, we collect and integrate more than 23 TAG datasets with domains ranging from citation graphs to molecule graphs and tasks from node classification to graph question-answering.
Efficient Neural Common Neighbor for Temporal Graph Link Prediction
However, such methods focus on learning individual node representations, but overlook the pairwise representation learning nature of link prediction and fail to capture the important pairwise features of links such as common neighbors (CN).
Graph as Point Set
In contrast, this paper introduces a novel graph-to-set conversion method that bijectively transforms interconnected nodes into a set of independent points and then uses a set encoder to learn the graph representation.
4DBInfer: A 4D Benchmarking Toolbox for Graph-Centric Predictive Modeling on Relational DBs
Although RDBs store vast amounts of rich, informative data spread across interconnected tables, the progress of predictive machine learning models as applied to such tasks arguably falls well behind advances in other domains such as computer vision or natural language processing.
SPGNN: Recognizing Salient Subgraph Patterns via Enhanced Graph Convolution and Pooling
We propose a novel Subgraph Pattern GNN (SPGNN) architecture that incorporates these enhancements.
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models
PiSSA shares the same architecture as LoRA, but initializes the adaptor matrices $A$ and $B$ with the principal components of the original matrix $W$, and put the remaining components into a residual matrix $W^{res} \in \mathbb{R}^{m \times n}$ which is frozen during fine-tuning.
Case-Based or Rule-Based: How Do Transformers Do the Math?
Through carefully designed intervention experiments on five math tasks, we confirm that transformers are performing case-based reasoning, no matter whether scratchpad is used, which aligns with the previous observations that transformers use subgraph matching/shortcut learning to reason.
Highly Accurate Disease Diagnosis and Highly Reproducible Biomarker Identification with PathFormer
However, we found two major limitations of existing GNNs in omics data analysis, i. e., limited-prediction (diagnosis) accuracy and limited-reproducible biomarker identification capacity across multiple datasets.
On the Completeness of Invariant Geometric Deep Learning Models
We then show that GeoNGNN, the geometric counterpart of one of the simplest subgraph graph neural networks (subgraph GNNs), can effectively break these corner cases' symmetry and thus achieve E(3)-completeness.
PyTorch Geometric High Order: A Unified Library for High Order Graph Neural Network
We introduce PyTorch Geometric High Order (PyGHO), a library for High Order Graph Neural Networks (HOGNNs) that extends PyTorch Geometric (PyG).
Chain of Images for Intuitively Reasoning
The human brain is naturally equipped to comprehend and interpret visual information rapidly.
LooGLE: Can Long-Context Language Models Understand Long Contexts?
In this paper, we present LooGLE, a Long Context Generic Language Evaluation benchmark for LLMs' long context understanding.
Facilitating Graph Neural Networks with Random Walk on Simplicial Complexes
Second, on $1$-simplices or edge level, we bridge edge-level random walk and Hodge $1$-Laplacians and design corresponding edge PE respectively.
Neural Attention: Enhancing QKV Calculation in Self-Attention Mechanism with Neural Networks
In the realm of deep learning, the self-attention mechanism has substantiated its pivotal role across a myriad of tasks, encompassing natural language processing and computer vision.
Parrot Mind: Towards Explaining the Complex Task Reasoning of Pretrained Large Language Models with Template-Content Structure
Furthermore, by generalizing this structure to the hierarchical case, we demonstrate that models can achieve task composition, further reducing the space needed to learn from linear to logarithmic, thereby effectively learning on complex reasoning involving multiple steps.
On the Stability of Expressive Positional Encodings for Graphs
Despite many attempts to address non-uniqueness, most methods overlook stability, leading to poor generalization on unseen graph structures.
Molecular Property Prediction
Out-of-Distribution Generalization
+1
One for All: Towards Training One Graph Model for All Classification Tasks
For in-context learning on graphs, OFA introduces a novel graph prompting paradigm that appends prompting substructures to the input graph, which enables it to address varied tasks without fine-tuning.
Graph Contrastive Learning Meets Graph Meta Learning: A Unified Method for Few-shot Node Tasks
In our study, we first identify two crucial advantages of contrastive learning compared to meta learning, including (1) the comprehensive utilization of graph nodes and (2) the power of graph augmentations.
Distance-Restricted Folklore Weisfeiler-Leman GNNs with Provable Cycle Counting Power
Many of the proposed GNN models with provable cycle counting power are based on subgraph GNNs, i. e., extracting a bag of subgraphs from the input graph, generating representations for each subgraph, and using them to augment the representation of the input graph.
Rethinking the Power of Graph Canonization in Graph Representation Learning with Stability
We theoretically reveal the trade-off of expressivity and stability in graph-canonization-enhanced GNNs.
CktGNN: Circuit Graph Neural Network for Electronic Design Automation
The electronic design automation of analog circuits has been a longstanding challenge in the integrated circuit field due to the huge design space and complex design trade-offs among circuit specifications.
VQGraph: Rethinking Graph Representation Space for Bridging GNNs and MLPs
To address this issue, we propose to learn a new powerful graph representation space by directly labeling nodes' diverse local structures for GNN-to-MLP distillation.
Extending the Design Space of Graph Neural Networks by Rethinking Folklore Weisfeiler-Lehman
We theoretically prove that even if we fix the space complexity to $O(n^k)$ (for any $k\geq 2$) in $(k, t)$-FWL, we can construct an expressiveness hierarchy up to solving the graph isomorphism problem.
Ranked #3 on
Graph Regression
on ZINC-500k
Code Prompting: a Neural Symbolic Method for Complex Reasoning in Large Language Models
We also consider the ensemble of code prompting and CoT prompting to combine the strengths of both.
Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
On the whole, our analysis provides a novel perspective on the role of semantics in developing and evaluating language models' reasoning abilities.
From Relational Pooling to Subgraph GNNs: A Universal Framework for More Expressive Graph Neural Networks
Relational pooling is a framework for building more expressive and permutation-invariant graph neural networks.
Improving Graph Neural Networks on Multi-node Tasks with Labeling Tricks
When we want to learn a node-set representation involving multiple nodes, a common practice in previous works is to directly aggregate the single-node representations obtained by a GNN.
An Empirical Study of Realized GNN Expressiveness
To address these limitations, we study the realized expressive power that a practical model instance can achieve using a novel expressiveness dataset, BREC, which poses greater difficulty (with up to 4-WL-indistinguishable graphs), finer granularity (enabling comparison of models between 1-WL and 3-WL), a larger scale (consisting of 800 1-WL-indistinguishable graphs that are non-isomorphic to each other).
An Efficient Subgraph GNN with Provable Substructure Counting Power
We investigate the enhancement of graph neural networks' (GNNs) representation power through their ability in substructure counting.
SUREL+: Moving from Walks to Sets for Scalable Subgraph-based Graph Representation Learning
Subgraph-based graph representation learning (SGRL) has recently emerged as a powerful tool in many prediction tasks on graphs due to its advantages in model expressiveness and generalization ability.
Ranked #1 on
Link Property Prediction
on ogbl-ppa
Time Associated Meta Learning for Clinical Prediction
We view time-associated disease prediction as classification tasks at multiple time points.
Is Distance Matrix Enough for Geometric Deep Learning?
In this work, we first construct families of novel and symmetric geometric graphs that Vanilla DisGNN cannot distinguish even when considering all-pair distances, which greatly expands the existing counterexample families.
Neural Common Neighbor with Completion for Link Prediction
In this work, we propose a novel link prediction model and further boost it by studying graph incompleteness.
Ranked #1 on
Link Property Prediction
on ogbl-ddi
RulE: Knowledge Graph Reasoning with Rule Embedding
In this paper, we propose a novel and principled framework called \textbf{RulE} (stands for {Rul}e {E}mbedding) to effectively leverage logical rules to enhance KG reasoning.
Boosting the Cycle Counting Power of Graph Neural Networks with I$^2$-GNNs
To the best of our knowledge, it is the first linear-time GNN model that can count 6-cycles with theoretical guarantees.
1st ICLR International Workshop on Privacy, Accountability, Interpretability, Robustness, Reasoning on Structured Data (PAIR^2Struct)
Recent years have seen advances on principles and guidance relating to accountable and ethical use of artificial intelligence (AI) spring up around the globe.
Geodesic Graph Neural Network for Efficient Graph Representation Learning
The GNN embeddings of nodes on the shortest paths are used to generate geodesic representations.
Rethinking Knowledge Graph Evaluation Under the Open-World Assumption
However, evaluation of knowledge graph completion (KGC) models often ignores the incompleteness -- facts in the test set are ranked against all unknown triplets which may contain a large number of missing facts not included in the KG yet.
Graph Neural Network with Local Frame for Molecular Potential Energy Surface
Projected onto a frame, equivariant features like 3D coordinates are converted to invariant features, so that we can capture geometric information with these projections and decouple the symmetry requirement from GNN design.
Two-Dimensional Weisfeiler-Lehman Graph Neural Networks for Link Prediction
As pointed out by previous works, this two-step procedure results in low discriminating power, as 1-WL-GNNs by nature learn node-level representations instead of link-level.
How Powerful are K-hop Message Passing Graph Neural Networks
Recently, researchers extended 1-hop message passing to K-hop message passing by aggregating information from K-hop neighbors of nodes simultaneously.
How Powerful are Spectral Graph Neural Networks
We also establish a connection between the expressive power of spectral GNNs and Graph Isomorphism (GI) testing, the latter of which is often used to characterize spatial GNNs' expressive power.
3DLinker: An E(3) Equivariant Variational Autoencoder for Molecular Linker Design
The main computational challenges include: 1) the generation of linkers is conditional on the two given molecules, in contrast to generating full molecules from scratch in previous works; 2) linkers heavily depend on the anchor atoms of the two molecules to be connected, which are not known beforehand; 3) 3D structures and orientations of the molecules need to be considered to avoid atom clashes, for which equivariance to E(3) group are necessary.
PACE: A Parallelizable Computation Encoder for Directed Acyclic Graphs
In this work, we propose a Parallelizable Attention-based Computation structure Encoder (PACE) that processes nodes simultaneously and encodes DAGs in parallel.
Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks
Graph neural networks (GNN) have shown great advantages in many graph-based learning tasks but often fail to predict accurately for a task-based on sets of nodes such as link/motif prediction and so on.
Algorithm and System Co-design for Efficient Subgraph-based Graph Representation Learning
Subgraph-based graph representation learning (SGRL) has been recently proposed to deal with some fundamental challenges encountered by canonical graph neural networks (GNNs), and has demonstrated advantages in many important data science applications such as link, relation and motif prediction.
Ranked #1 on
Link Property Prediction
on ogbl-citation2
Decoupling the Depth and Scope of Graph Neural Networks
We propose a design principle to decouple the depth and scope of GNNs -- to generate representation of a target entity (i. e., a node or an edge), we first extract a localized subgraph as the bounded-size scope, and then apply a GNN of arbitrary depth on top of the subgraph.
Ranked #3 on
Node Classification
on Reddit
Network In Graph Neural Network
However, wider hidden layers can easily lead to overfitting, and incrementally adding more GNN layers can potentially result in over-smoothing. In this paper, we present a model-agnostic methodology, namely Network In Graph Neural Network (NGNN ), that allows arbitrary GNN models to increase their model capacity by making the model deeper.
Ranked #1 on
Link Property Prediction
on ogbl-citation2
Nested Graph Neural Networks
The key is to make each node representation encode a subgraph around it more than a subtree.
Ranked #10 on
Graph Property Prediction
on ogbg-molpcba
GLASS: GNN with Labeling Tricks for Subgraph Representation Learning
And training a GLASS model only takes 28% time needed for a SubGNN on average.
Principled Hyperedge Prediction with Structural Spectral Features and Neural Networks
SNALS captures the joint interactions of a hyperedge by its local environment, which is retrieved by collecting the spectrum information of their connections.
Learning Two-Time-Scale Representations For Large Scale Recommendations
We propose a surprisingly simple but effective two-time-scale (2TS) model for learning user representations for recommendation.
Deep Graph Neural Networks with Shallow Subgraph Samplers
We propose a simple "deep GNN, shallow sampler" design principle to improve both the GNN accuracy and efficiency -- to generate representation of a target node, we use a deep GNN to pass messages only within a shallow, localized subgraph.
Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning
In this paper, we provide a theory of using graph neural networks (GNNs) for multi-node representation learning (where we are interested in learning a representation for a set of more than one node, such as link).
Ranked #1 on
Link Property Prediction
on ogbl-citation2
Revisiting Graph Neural Networks for Link Prediction
Graph neural networks (GNNs) have achieved great success in recent years.
Pooling Regularized Graph Neural Network for fMRI Biomarker Analysis
We propose an interpretable GNN framework with a novel salient region selection mechanism to determine neurological brain biomarkers associated with disorders.
Inductive Matrix Completion Based on Graph Neural Networks
Under the extreme setting where not any side information is available other than the matrix to complete, can we still learn an inductive matrix completion model?
Ranked #1 on
Recommendation Systems
on Flixster Monti
D-VAE: A Variational Autoencoder for Directed Acyclic Graphs
Graph structured data are abundant in the real world.
An End-to-End Deep Learning Architecture for Graph Classification
Neural networks are typically designed to deal with data in tensor forms.
Ranked #18 on
Graph Classification
on D&D
Link Prediction Based on Graph Neural Networks
The theory unifies a wide range of heuristics in a single framework, and proves that all these heuristics can be well approximated from local subgraphs.
Ranked #1 on
Link Prediction
on USAir
Weisfeiler-lehman neural machine for link prediction
Compared with traditional link prediction methods, Wlnm does not assume a particular link formation mechanism (such as common neighbors), but learns this mechanism from the graph itself.
