Search Results for author: Zirui Liu

Found 43 papers, 21 papers with code

LiNo: Advancing Recursive Residual Decomposition of Linear and Nonlinear Patterns for Robust Time Series Forecasting

1 code implementation22 Oct 2024 Guoqi Yu, Yaoming Li, XIAOYU GUO, Dayu Wang, Zirui Liu, Shujun Wang, Tong Yang

Forecasting models are pivotal in a data-driven world with vast volumes of time series data that appear as a compound of vast Linear and Nonlinear patterns.

Time Series Time Series Forecasting

Gradient Rewiring for Editable Graph Neural Network Training

1 code implementation21 Oct 2024 Zhimeng Jiang, Zirui Liu, Xiaotian Han, Qizhang Feng, Hongye Jin, Qiaoyu Tan, Kaixiong Zhou, Na Zou, Xia Hu

In this paper, we first observe the gradient of cross-entropy loss for the target node and training nodes with significant inconsistency, which indicates that directly fine-tuning the base model using the loss on the target node deteriorates the performance on training nodes.

Graph Neural Network Model Editing

Robust Network Learning via Inverse Scale Variational Sparsification

no code implementations27 Sep 2024 Zhiling Zhou, Zirui Liu, Chengming Xu, Yanwei Fu, Xinwei Sun

While neural networks have made significant strides in many AI tasks, they remain vulnerable to a range of noise types, including natural corruptions, adversarial noise, and low-resolution artifacts.

INT-FlashAttention: Enabling Flash Attention for INT8 Quantization

1 code implementation25 Sep 2024 Shimao Chen, Zirui Liu, Zhiying Wu, Ce Zheng, Peizhuang Cong, Zihan Jiang, Yuhan Wu, Lei Su, Tong Yang

As the foundation of large language models (LLMs), self-attention module faces the challenge of quadratic time and memory complexity with respect to sequence length.

Quantization

From Commands to Prompts: LLM-based Semantic File System for AIOS

no code implementations23 Sep 2024 Zeru Shi, Kai Mei, Mingyu Jin, Yongye Su, Chaoji Zuo, Wenyue Hua, Wujiang Xu, Yujie Ren, Zirui Liu, Mengnan Du, Dong Deng, Yongfeng Zhang

Large language models (LLMs) have demonstrated significant potential in the development of intelligent applications and systems such as LLM-based agents and agent operating systems (AIOS).

Management Navigate

Assessing and Enhancing Large Language Models in Rare Disease Question-answering

no code implementations15 Aug 2024 Guanchu Wang, Junhao Ran, Ruixiang Tang, Chia-Yuan Chang, Yu-Neng Chuang, Zirui Liu, Vladimir Braverman, Zhandong Liu, Xia Hu

In this work, we introduce a rare disease question-answering (ReDis-QA) dataset to evaluate the performance of LLMs in diagnosing rare diseases.

Question Answering

Research on Tibetan Tourism Viewpoints information generation system based on LLM

no code implementations18 Jul 2024 Jinhu Qi, Shuai Yan, Wentao Zhang, Yibo Zhang, Zirui Liu, Ke Wang

Tibet, ensconced within China's territorial expanse, is distinguished by its labyrinthine and heterogeneous topography, a testament to its profound historical heritage, and the cradle of a unique religious ethos.

Language Modelling Large Language Model

Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity

no code implementations5 Jun 2024 Wentao Guo, Jikai Long, Yimeng Zeng, Zirui Liu, Xinyu Yang, Yide Ran, Jacob R. Gardner, Osbert Bastani, Christopher De Sa, Xiaodong Yu, Beidi Chen, Zhaozhuo Xu

Zeroth-order optimization (ZO) is a memory-efficient strategy for fine-tuning Large Language Models using only forward passes.

Quantization

Quantifying Multilingual Performance of Large Language Models Across Languages

no code implementations17 Apr 2024 Zihao Li, Yucheng Shi, Zirui Liu, Fan Yang, Ali Payani, Ninghao Liu, Mengnan Du

Besides, the experiments show that there is a strong correlation between the LLM's performance in different languages and the proportion of those languages in its pre-training corpus.

Survey of Computerized Adaptive Testing: A Machine Learning Perspective

1 code implementation31 Mar 2024 Qi Liu, Yan Zhuang, Haoyang Bi, Zhenya Huang, Weizhe Huang, Jiatong Li, Junhao Yu, Zirui Liu, Zirui Hu, Yuting Hong, Zachary A. Pardos, Haiping Ma, Mengxiao Zhu, Shijin Wang, Enhong Chen

Computerized Adaptive Testing (CAT) provides an efficient and tailored method for assessing the proficiency of examinees, by dynamically adjusting test questions based on their performance.

cognitive diagnosis Question Selection +2

LoRA-as-an-Attack! Piercing LLM Safety Under The Share-and-Play Scenario

no code implementations29 Feb 2024 Hongyi Liu, Zirui Liu, Ruixiang Tang, Jiayi Yuan, Shaochen Zhong, Yu-Neng Chuang, Li Li, Rui Chen, Xia Hu

Our aim is to raise awareness of the potential risks under the emerging share-and-play scenario, so as to proactively prevent potential consequences caused by LoRA-as-an-Attack.

Learning to Compress Prompt in Natural Language Formats

no code implementations28 Feb 2024 Yu-Neng Chuang, Tianwei Xing, Chia-Yuan Chang, Zirui Liu, Xun Chen, Xia Hu

In this work, we propose a Natural Language Prompt Encapsulation (Nano-Capsulator) framework compressing original prompts into NL formatted Capsule Prompt while maintaining the prompt utility and transferability.

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

1 code implementation5 Feb 2024 Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, Xia Hu

However, there is a lack of in-depth studies that explore the element distribution of KV cache to understand the hardness and limitation of KV cache quantization.

Quantization

LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

2 code implementations2 Jan 2024 Hongye Jin, Xiaotian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia-Yuan Chang, Huiyuan Chen, Xia Hu

To achieve this goal, we propose SelfExtend to extend the context window of LLMs by constructing bi-level attention information: the grouped attention and the neighbor attention.

TVE: Learning Meta-attribution for Transferable Vision Explainer

1 code implementation23 Dec 2023 Guanchu Wang, Yu-Neng Chuang, Fan Yang, Mengnan Du, Chia-Yuan Chang, Shaochen Zhong, Zirui Liu, Zhaozhuo Xu, Kaixiong Zhou, Xuanting Cai, Xia Hu

This meta-attribution leverages the versatility of generic backbone encoders to comprehensively encode the attribution knowledge for the input instance, which enables TVE to seamlessly transfer to explain various downstream tasks, without the need for training on task-specific data.

CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models

1 code implementation6 Dec 2023 Hailin Zhang, Zirui Liu, Boxuan Chen, Yikai Zhao, Tong Zhao, Tong Yang, Bin Cui

Guided by our design philosophy, we further propose a multi-level hash embedding framework to optimize the embedding tables of non-hot features.

Feature Importance Philosophy

Experimental Analysis of Large-scale Learnable Vector Storage Compression

1 code implementation27 Nov 2023 Hailin Zhang, Penghao Zhao, Xupeng Miao, Yingxia Shao, Zirui Liu, Tong Yang, Bin Cui

Learnable embedding vector is one of the most important applications in machine learning, and is widely used in various database-related domains.

Benchmarking

Efficient GNN Explanation via Learning Removal-based Attribution

no code implementations9 Jun 2023 Yao Rong, Guanchu Wang, Qizhang Feng, Ninghao Liu, Zirui Liu, Enkelejda Kasneci, Xia Hu

A strategy of subgraph sampling is designed in LARA to improve the scalability of the training process.

Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model

1 code implementation NeurIPS 2023 Zirui Liu, Guanchu Wang, Shaochen Zhong, Zhaozhuo Xu, Daochen Zha, Ruixiang Tang, Zhimeng Jiang, Kaixiong Zhou, Vipin Chaudhary, Shuai Xu, Xia Hu

While the model parameters do contribute to memory usage, the primary memory bottleneck during training arises from storing feature maps, also known as activations, as they are crucial for gradient calculation.

Language Modelling Stochastic Optimization

Editable Graph Neural Network for Node Classifications

no code implementations24 May 2023 Zirui Liu, Zhimeng Jiang, Shaochen Zhong, Kaixiong Zhou, Li Li, Rui Chen, Soo-Hyun Choi, Xia Hu

However, model editing for graph neural networks (GNNs) is rarely explored, despite GNNs' widespread applicability.

Fake News Detection Graph Neural Network +1

Efficient XAI Techniques: A Taxonomic Survey

no code implementations7 Feb 2023 Yu-Neng Chuang, Guanchu Wang, Fan Yang, Zirui Liu, Xuanting Cai, Mengnan Du, Xia Hu

Finally, we summarize the challenges of deploying XAI acceleration methods to real-world scenarios, overcoming the trade-off between faithfulness and efficiency, and the selection of different acceleration methods.

Explainable artificial intelligence Explainable Artificial Intelligence (XAI) +1

Retiring $Δ$DP: New Distribution-Level Metrics for Demographic Parity

1 code implementation31 Jan 2023 Xiaotian Han, Zhimeng Jiang, Hongye Jin, Zirui Liu, Na Zou, Qifan Wang, Xia Hu

Unfortunately, in this paper, we reveal that the fairness metric $\Delta DP$ can not precisely measure the violation of demographic parity, because it inherently has the following drawbacks: i) zero-value $\Delta DP$ does not guarantee zero violation of demographic parity, ii) $\Delta DP$ values can vary with different classification thresholds.

Fairness

RSC: Accelerating Graph Neural Networks Training via Randomized Sparse Computations

no code implementations19 Oct 2022 Zirui Liu, Shengyuan Chen, Kaixiong Zhou, Daochen Zha, Xiao Huang, Xia Hu

To this end, we propose Randomized Sparse Computation, which for the first time demonstrate the potential of training GNNs with approximated operations.

DreamShard: Generalizable Embedding Table Placement for Recommender Systems

1 code implementation5 Oct 2022 Daochen Zha, Louis Feng, Qiaoyu Tan, Zirui Liu, Kwei-Herng Lai, Bhargav Bhushanam, Yuandong Tian, Arun Kejariwal, Xia Hu

Although prior work has explored learning-based approaches for the device placement of computational graphs, embedding table placement remains to be a challenging problem because of 1) the operation fusion of embedding tables, and 2) the generalizability requirement on unseen placement tasks with different numbers of tables and/or devices.

Recommendation Systems Reinforcement Learning (RL)

DIVISION: Memory Efficient Training via Dual Activation Precision

1 code implementation5 Aug 2022 Guanchu Wang, Zirui Liu, Zhimeng Jiang, Ninghao Liu, Na Zou, Xia Hu

Activation compressed training provides a solution towards reducing the memory cost of training deep neural networks~(DNNs).

Quantization

FMP: Toward Fair Graph Message Passing against Topology Bias

no code implementations8 Feb 2022 Zhimeng Jiang, Xiaotian Han, Chao Fan, Zirui Liu, Na Zou, Ali Mostafavi, Xia Hu

Despite recent advances in achieving fair representations and predictions through regularization, adversarial debiasing, and contrastive learning in graph neural networks (GNNs), the working mechanism (i. e., message passing) behind GNNs inducing unfairness issue remains unknown.

Contrastive Learning Fairness +1

An Information Fusion Approach to Learning with Instance-Dependent Label Noise

no code implementations ICLR 2022 Zhimeng Jiang, Kaixiong Zhou, Zirui Liu, Li Li, Rui Chen, Soo-Hyun Choi, Xia Hu

Instance-dependent label noise (IDN) widely exists in real-world datasets and usually misleads the training of deep neural networks.

EXACT: Scalable Graph Neural Networks Training via Extreme Activation Compression

no code implementations ICLR 2022 Zirui Liu, Kaixiong Zhou, Fan Yang, Li Li, Rui Chen, Xia Hu

Based on the implementation, we propose a memory-efficient framework called ``EXACT'', which for the first time demonstrate the potential and evaluate the feasibility of training GNNs with compressed activations.

Graph Learning

Adaptive Label Smoothing To Regularize Large-Scale Graph Training

no code implementations30 Aug 2021 Kaixiong Zhou, Ninghao Liu, Fan Yang, Zirui Liu, Rui Chen, Li Li, Soo-Hyun Choi, Xia Hu

Graph neural networks (GNNs), which learn the node representations by recursively aggregating information from its neighbors, have become a predominant computational tool in many domains.

Node Clustering

DivAug: Plug-in Automated Data Augmentation with Explicit Diversity Maximization

1 code implementation ICCV 2021 Zirui Liu, Haifeng Jin, Ting-Hsiang Wang, Kaixiong Zhou, Xia Hu

We validate in experiments that the relative gain from automated data augmentation in test accuracy is highly correlated to Variance Diversity.

Data Augmentation Diversity

Detecting Interactions from Neural Networks via Topological Analysis

no code implementations NeurIPS 2020 Zirui Liu, Qingquan Song, Kaixiong Zhou, Ting-Hsiang Wang, Ying Shan, Xia Hu

Motivated by the observation, in this paper, we propose to investigate the interaction detection problem from a novel topological perspective by analyzing the connectivity in neural networks.

Towards Interaction Detection Using Topological Analysis on Neural Networks

no code implementations25 Oct 2020 Zirui Liu, Qingquan Song, Kaixiong Zhou, Ting Hsiang Wang, Ying Shan, Xia Hu

Detecting statistical interactions between input features is a crucial and challenging task.

AutoRec: An Automated Recommender System

1 code implementation26 Jun 2020 Ting-Hsiang Wang, Qingquan Song, Xiaotian Han, Zirui Liu, Haifeng Jin, Xia Hu

To address the need, we present AutoRec, an open-source automated machine learning (AutoML) platform extended from the TensorFlow ecosystem and, to our knowledge, the first framework to leverage AutoML for model search and hyperparameter tuning in deep recommendation models.

AutoML Click-Through Rate Prediction +1

Mitigating Gender Bias in Captioning Systems

1 code implementation15 Jun 2020 Ruixiang Tang, Mengnan Du, Yuening Li, Zirui Liu, Na Zou, Xia Hu

Image captioning has made substantial progress with huge supporting image collections sourced from the web.

Benchmarking Gender Prediction +1

Cannot find the paper you are looking for? You can Submit a new open access paper.