no code implementations • 13 Oct 2024 • Dan Ley, Suraj Srinivas, Shichang Zhang, Gili Rusak, Himabindu Lakkaraju
Data Attribution (DA) methods quantify the influence of individual training data points on model outputs and have broad applications such as explainability, data selection, and noisy label identification.
no code implementations • 13 Jun 2024 • Shichang Zhang, Botao Xia, Zimin Zhang, Qianli Wu, Fang Sun, Ziniu Hu, Yizhou Sun
Artificial intelligence (AI) is significantly transforming scientific research.
no code implementations • 13 Jun 2024 • Shichang Zhang, Da Zheng, Jiani Zhang, Qi Zhu, Xiang Song, Soji Adeshina, Christos Faloutsos, George Karypis, Yizhou Sun
Large Language Models (LLMs), noted for their superior text understanding abilities, offer a solution for processing the text in graphs but face integration challenges due to their limitation for encoding graph structures and their computational complexities when dealing with extensive text in large neighborhoods of interconnected nodes.
1 code implementation • 4 Jun 2024 • Min Cai, Yuchen Zhang, Shichang Zhang, Fan Yin, Dan Zhang, Difan Zou, Yisong Yue, Ziniu Hu
Given a desired behavior expressed in a natural language suffix string concatenated to the input prompt, SelfControl computes gradients of the LLM's self-evaluation of the suffix with respect to its latent representations.
no code implementations • 27 May 2024 • Junwei Deng, Ting-Wei Li, Shichang Zhang, Jiaqi Ma
Training data attribution (TDA) methods aim to quantify the influence of individual training data points on the model predictions, with broad applications in data-centric AI, such as mislabel detection, data selection, and copyright compensation.
no code implementations • 28 Apr 2024 • Qi Zhu, Da Zheng, Xiang Song, Shichang Zhang, Bowen Jin, Yizhou Sun, George Karypis
Inspired by this, we introduce Graph-aware Parameter-Efficient Fine-Tuning - GPEFT, a novel approach for efficient graph representation learning with LLMs on text-rich graphs.
1 code implementation • 8 Dec 2023 • Haoyu Li, Shichang Zhang, Longwen Tang, Mathieu Bauchy, Yizhou Sun
Metallic Glasses (MGs) are widely used materials that are stronger than steel while being shapeable as plastic.
1 code implementation • 20 Jul 2023 • Xiaoxuan Wang, Ziniu Hu, Pan Lu, Yanqiao Zhu, Jieyu Zhang, Satyen Subramaniam, Arjun R. Loomba, Shichang Zhang, Yizhou Sun, Wei Wang
Most of the existing Large Language Model (LLM) benchmarks on scientific problem reasoning focus on problems grounded in high-school subjects and are confined to elementary algebraic operations.
no code implementations • 24 Jun 2023 • Shichang Zhang, Atefeh Sohrabizadeh, Cheng Wan, Zijie Huang, Ziniu Hu, Yewen Wang, Yingyan, Lin, Jason Cong, Yizhou Sun
Graph neural networks (GNNs) are emerging for machine learning research on graph-structured data.
1 code implementation • 24 Feb 2023 • Shichang Zhang, Jiani Zhang, Xiang Song, Soji Adeshina, Da Zheng, Christos Faloutsos, Yizhou Sun
However, GNN explanation for link prediction (LP) is lacking in the literature.
no code implementations • 11 Oct 2022 • Zhichun Guo, William Shiao, Shichang Zhang, Yozen Liu, Nitesh V. Chawla, Neil Shah, Tong Zhao
In this work, to combine the advantages of GNNs and MLPs, we start with exploring direct knowledge distillation (KD) methods for link prediction, i. e., predicted logit-based matching and node representation-based matching.
1 code implementation • 28 Jan 2022 • Shichang Zhang, Yozen Liu, Neil Shah, Yizhou Sun
Explaining machine learning models is an important and increasingly popular area of research interest.
1 code implementation • ICLR 2022 • Shichang Zhang, Yozen Liu, Yizhou Sun, Neil Shah
Conversely, multi-layer perceptrons (MLPs) have no graph dependency and infer much faster than GNNs, even though they are less accurate than GNNs for node classification in general.
Ranked #5 on Node Classification on AMZ Computers
2 code implementations • ICLR 2022 • Wei Jin, Lingxiao Zhao, Shichang Zhang, Yozen Liu, Jiliang Tang, Neil Shah
Given the prevalence of large-scale graphs in real-world applications, the storage and time for training neural models have raised increasing concerns.
no code implementations • 23 Dec 2020 • Shichang Zhang, Ziniu Hu, Arjun Subramonian, Yizhou Sun
Our framework MotIf-driven Contrastive leaRning Of Graph representations (MICRO-Graph) can: 1) use GNNs to extract motifs from large graph datasets; 2) leverage learned motifs to sample informative subgraphs for contrastive learning of GNN.