Search Results for author:
Found 29 papers, 17 papers with code
One Model for One Graph: A New Perspective for Pretraining with Cross-domain Graphs
Graph Neural Networks (GNNs) have emerged as a powerful tool to capture intricate network patterns, achieving success across different domains.
Do Neural Scaling Laws Exist on Graph Self-Supervised Learning?
This paper examines existing SSL techniques for the feasibility of Graph SSL techniques in developing GFMs and opens a new direction for graph SSL design with the new evaluation prototype.
Intrinsic Self-correction for Enhanced Morality: An Analysis of Internal Mechanisms and the Superficial Hypothesis
Through empirical investigation with tasks of language generation and multi-choice question answering, we conclude:(i) LLMs exhibit good performance across both tasks, and self-correction instructions are particularly beneficial when the correct answer is already top-ranked; (ii) The morality levels in intermediate hidden states are strong indicators as to whether one instruction would be more effective than another; (iii) Based on our analysis of intermediate hidden states and task case studies of self-correction behaviors, we are first to propose the hypothesis that intrinsic moral self-correction is in fact superficial.
A Pure Transformer Pretraining Framework on Text-attributed Graphs
Pretraining plays a pivotal role in acquiring generalized knowledge from large-scale data, achieving remarkable successes as evidenced by large models in CV and NLP.
Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights
First, the absence of a comprehensive benchmark with unified problem settings hinders a clear understanding of the comparative effectiveness and practical value of different text-space GFMs.
On the Intrinsic Self-Correction Capability of LLMs: Uncertainty and Latent Concept
Our findings are verified in: (1) the scenario of multi-round question answering, by comprehensively demonstrating that intrinsic self-correction can progressively introduce performance gains through iterative interactions, ultimately converging to stable performance; and (2) the context of intrinsic self-correction for enhanced morality, in which we provide empirical evidence that iteratively applying instructions reduces model uncertainty towards convergence, which then leads to convergence of both the calibration error and self-correction performance, ultimately resulting in a stable state of intrinsic self-correction.
PDHG-Unrolled Learning-to-Optimize Method for Large-Scale Linear Programming
In this work, we propose an FOM-unrolled neural network (NN) called PDHG-Net, and propose a two-stage L2O method to solve large-scale LP problems.
Cross-Domain Graph Data Scaling: A Showcase with Diffusion Models
To achieve effective data scaling, we aim to develop a general model that is able to capture diverse data patterns of graphs and can be utilized to adaptively help the downstream tasks.
Graph Machine Learning in the Era of Large Language Models (LLMs)
Meanwhile, graphs, especially knowledge graphs, are rich in reliable factual knowledge, which can be utilized to enhance the reasoning capabilities of LLMs and potentially alleviate their limitations such as hallucinations and the lack of explainability.
Addressing Shortcomings in Fair Graph Learning Datasets: Towards a New Benchmark
These datasets are thoughtfully designed to include relevant graph structures and bias information crucial for the fair evaluation of models.
Universal Link Predictor By In-Context Learning on Graphs
In this work, we introduce the Universal Link Predictor (UniLP), a novel model that combines the generalizability of heuristic approaches with the pattern learning capabilities of parametric models.
Towards Neural Scaling Laws on Graphs
Yet, the neural scaling laws on graphs, i. e., how the performance of deep graph models changes with model and dataset sizes, have not been systematically investigated, casting doubts on the feasibility of achieving large graph models.
Position: Graph Foundation Models are Already Here
Graph Foundation Models (GFMs) are emerging as a significant research topic in the graph domain, aiming to develop graph models trained on extensive and diverse data to enhance their applicability across various tasks and domains.
A Survey to Recent Progress Towards Understanding In-Context Learning
In-Context Learning (ICL) empowers Large Language Models (LLMs) with the ability to learn from a few examples provided in the prompt, enabling downstream generalization without the requirement for gradient updates.
LPFormer: An Adaptive Graph Transformer for Link Prediction
These methods perform predictions by using the output of an MPNN in conjunction with a "pairwise encoding" that captures the relationship between nodes in the candidate link.
Label-free Node Classification on Graphs with Large Language Models (LLMS)
In light of these observations, this work introduces a label-free node classification on graphs with LLMs pipeline, LLM-GNN.
Revisiting Link Prediction: A Data Perspective
We recognize three fundamental factors critical to link prediction: local structural proximity, global structural proximity, and feature proximity.
Amazon-M2: A Multilingual Multi-locale Shopping Session Dataset for Recommendation and Text Generation
To test the potential of the dataset, we introduce three tasks in this work: (1) next-product recommendation, (2) next-product recommendation with domain shifts, and (3) next-product title generation.
Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs
The most popular pipeline for learning on graphs with textual node attributes primarily relies on Graph Neural Networks (GNNs), and utilizes shallow text embedding as initial node representations, which has limitations in general knowledge and profound semantic understanding.
Evaluating Graph Neural Networks for Link Prediction: Current Pitfalls and New Benchmarking
Furthermore, new and diverse datasets have also been created to better evaluate the effectiveness of these new models.
Demystifying Structural Disparity in Graph Neural Networks: Can One Size Fit All?
Recent studies on Graph Neural Networks(GNNs) provide both empirical and theoretical evidence supporting their effectiveness in capturing structural patterns on both homophilic and certain heterophilic graphs.
Company Competition Graph
Based on the output from our system, we construct a knowledge graph with more than 700 nodes and 1200 edges.
Form 10-K Itemization
With the segmented Item sections, NLP techniques can directly apply on those Item sections related to downstream tasks.
Whole Page Unbiased Learning to Rank
To address the above challenges, we propose a Bias Agnostic whole-page unbiased Learning to rank algorithm, named BAL, to automatically find the user behavior model with causal discovery and mitigate the biases induced by multiple SERP features with no specific design.
A Large Scale Search Dataset for Unbiased Learning to Rank
The unbiased learning to rank (ULTR) problem has been greatly advanced by recent deep learning techniques and well-designed debias algorithms.
Alternately Optimized Graph Neural Networks
Extensive experiments demonstrate that the proposed method can achieve comparable or better performance with state-of-the-art baselines while it has significantly better computation and memory efficiency.
Source Free Unsupervised Graph Domain Adaptation
To address the non-trivial adaptation challenges in this practical scenario, we propose a model-agnostic algorithm called SOGA for domain adaptation to fully exploit the discriminative ability of the source model while preserving the consistency of structural proximity on the target graph.
Neuron with Steady Response Leads to Better Generalization
Based on the analysis results, we articulate the Neuron Steadiness Hypothesis: the neuron with similar responses to instances of the same class leads to better generalization.
Neuron Campaign for Initialization Guided by Information Bottleneck Theory
Initialization plays a critical role in the training of deep neural networks (DNN).
