Search Results for author: Jiahao Liu

Found 25 papers, 8 papers with code

VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation

1 code implementation ACL 2021 Fuli Luo, Wei Wang, Jiahao Liu, Yijia Liu, Bin Bi, Songfang Huang, Fei Huang, Luo Si

Existing work in multilingual pretraining has demonstrated the potential of cross-lingual transferability by training a unified Transformer encoder for multiple languages.

Language Modelling Question Answering +4

RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank

1 code implementation26 May 2023 Jiduan Liu, Jiahao Liu, Qifan Wang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Kai Chen, Rui Yan

In this paper, we propose a novel approach, RankCSE, for unsupervised sentence representation learning, which incorporates ranking consistency and ranking distillation with contrastive learning into a unified framework.

Contrastive Learning Learning-To-Rank +4

GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model

1 code implementation11 Jun 2023 Shicheng Tan, Weng Lam Tam, Yuanchun Wang, Wenwen Gong, Yang Yang, Hongyin Tang, Keqing He, Jiahao Liu, Jingang Wang, Shu Zhao, Peng Zhang, Jie Tang

Currently, the reduction in the parameter scale of large-scale pre-trained language models (PLMs) through knowledge distillation has greatly facilitated their widespread deployment on various devices.

General Knowledge Knowledge Distillation +1

Lifting the Curse of Capacity Gap in Distilling Language Models

1 code implementation20 May 2023 Chen Zhang, Yang Yang, Jiahao Liu, Jingang Wang, Yunsen Xian, Benyou Wang, Dawei Song

However, when the capacity gap between the teacher and the student is large, a curse of capacity gap appears, invoking a deficiency in distilling LMs.

Knowledge Distillation

Parameter-free Dynamic Graph Embedding for Link Prediction

1 code implementation15 Oct 2022 Jiahao Liu, Dongsheng Li, Hansu Gu, Tun Lu, Peng Zhang, Ning Gu

Dynamic interaction graphs have been widely adopted to model the evolution of user-item interactions over time.

Attribute Dynamic graph embedding +1

FedDWA: Personalized Federated Learning with Dynamic Weight Adjustment

1 code implementation10 May 2023 Jiahao Liu, Jiang Wu, Jinyu Chen, Miao Hu, Yipeng Zhou, Di wu

In this paper, we propose a new PFL algorithm called \emph{FedDWA (Federated Learning with Dynamic Weight Adjustment)} to address the above problem, which leverages the parameter server (PS) to compute personalized aggregation weights based on collected models from clients.

Personalized Federated Learning

AutoSeqRec: Autoencoder for Efficient Sequential Recommendation

1 code implementation14 Aug 2023 Sijia Liu, Jiahao Liu, Hansu Gu, Dongsheng Li, Tun Lu, Peng Zhang, Ning Gu

Sequential recommendation demonstrates the capability to recommend items by modeling the sequential behavior of users.

Collaborative Filtering Computational Efficiency +1

MiniDisc: Minimal Distillation Schedule for Language Model Compression

1 code implementation29 May 2022 Chen Zhang, Yang Yang, Qifan Wang, Jiahao Liu, Jingang Wang, Wei Wu, Dawei Song

In particular, motivated by the finding that the performance of the student is positively correlated to the scale-performance tradeoff of the teacher assistant, MiniDisc is designed with a $\lambda$-tradeoff to measure the optimality of the teacher assistant without trial distillation to the student.

Knowledge Distillation Language Modelling +2

A Planning based Framework for Essay Generation

no code implementations18 Dec 2015 Bing Qin, Duyu Tang, Xinwei Geng, Dandan Ning, Jiahao Liu, Ting Liu

Generating an article automatically with computer program is a challenging task in artificial intelligence and natural language processing.

Sentence

Community-preserving Graph Convolutions for Structural and Functional Joint Embedding of Brain Networks

no code implementations8 Nov 2019 Jiahao Liu, Guixiang Ma, Fei Jiang, Chun-Ta Lu, Philip S. Yu, Ann B. Ragin

Specifically, we use graph convolutions to learn the structural and functional joint embedding, where the graph structure is defined with structural connectivity and node features are from the functional connectivity.

MULTI-VIEW LEARNING

VECO: Variable Encoder-decoder Pre-training for Cross-lingual Understanding and Generation

no code implementations28 Sep 2020 Fuli Luo, Wei Wang, Jiahao Liu, Yijia Liu, Bin Bi, Songfang Huang, Fei Huang, Luo Si

Recent studies about learning multilingual representations have achieved significant performance gains across a wide range of downstream cross-lingual tasks.

Language Modelling Masked Language Modeling +5

GNN-encoder: Learning a Dual-encoder Architecture via Graph Neural Networks for Dense Passage Retrieval

no code implementations18 Apr 2022 Jiduan Liu, Jiahao Liu, Yang Yang, Jingang Wang, Wei Wu, Dongyan Zhao, Rui Yan

To enhance the performance of dense retrieval models without loss of efficiency, we propose a GNN-encoder model in which query (passage) information is fused into passage (query) representations via graph neural networks that are constructed by queries and their top retrieved passages.

Natural Questions Passage Retrieval +2

Personalized Graph Signal Processing for Collaborative Filtering

no code implementations4 Feb 2023 Jiahao Liu, Dongsheng Li, Hansu Gu, Tun Lu, Peng Zhang, Li Shang, Ning Gu

However, the interaction signal may not be sufficient to accurately characterize user interests and the low-pass filters may ignore the useful information contained in the high-frequency component of the observed signals, resulting in suboptimal accuracy.

Collaborative Filtering

An Error-Surface-Based Fractional Motion Estimation Algorithm and Hardware Implementation for VVC

no code implementations13 Feb 2023 Shushi Chen, Leilei Huang, Jiahao Liu, Chao Liu, Yibo Fan

In this context, this paper proposes an error-surface-based FME algorithm and the corresponding hardware implementation.

Motion Estimation

Triple Structural Information Modelling for Accurate, Explainable and Interactive Recommendation

no code implementations23 Apr 2023 Jiahao Liu, Dongsheng Li, Hansu Gu, Tun Lu, Peng Zhang, Li Shang, Ning Gu

Specifically, TriSIM4Rec consists of 1) a dynamic ideal low-pass graph filter to dynamically mine co-occurrence information in user-item interactions, which is implemented by incremental singular value decomposition (SVD); 2) a parameter-free attention module to capture sequential information of user interactions effectively and efficiently; and 3) an item transition matrix to store the transition probabilities of item pairs.

Collaborative Filtering

PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models

no code implementations30 May 2023 Zhuocheng Gong, Jiahao Liu, Qifan Wang, Yang Yang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Rui Yan

While transformer-based pre-trained language models (PLMs) have dominated a number of NLP applications, these models are heavy to deploy and expensive to use.

Quantization

Recommendation Unlearning via Matrix Correction

no code implementations29 Jul 2023 Jiahao Liu, Dongsheng Li, Hansu Gu, Tun Lu, Jiongran Wu, Peng Zhang, Li Shang, Ning Gu

We conducted comprehensive experiments to validate the effectiveness of IMCorrect and the results demonstrate that IMCorrect is superior in completeness, utility, and efficiency, and is applicable in many recommendation unlearning scenarios.

Collaborative Filtering Recommendation Systems

mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learning

no code implementations17 Aug 2023 Ying Mo, Jian Yang, Jiahao Liu, Qifan Wang, Ruoyu Chen, Jingang Wang, Zhoujun Li

A multi-view contrastive learning framework is introduced to encompass semantic contrasts between source, codeswitched, and target sentences, as well as contrasts among token-to-token relations.

Contrastive Learning named-entity-recognition +2

Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression

no code implementations24 Oct 2023 Jiduan Liu, Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai, Dongyan Zhao, Ran Lucien Wang, Rui Yan

In particular, our approach extracts knowledge from LLMs to construct a knowledge store, from which the small-scale model can retrieve relevant information and leverage it for effective inference.

Language Modelling Large Language Model +3

Improving Input-label Mapping with Demonstration Replay for In-context Learning

no code implementations30 Oct 2023 Zhuocheng Gong, Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai, Dongyan Zhao, Rui Yan

The effectiveness of ICL can be attributed to the strong language modeling capabilities of large language models (LLMs), which enable them to learn the mapping between input and labels based on in-context demonstrations.

In-Context Learning Language Modelling

View Distribution Alignment with Progressive Adversarial Learning for UAV Visual Geo-Localization

no code implementations3 Jan 2024 Cuiwei Liu, Jiahao Liu, Huaijun Qiu, Zhaokui Li, Xiangbin Shi

Previous works map images captured by UAVs and satellites to a shared feature space and employ a classification framework to learn location-dependent features while neglecting the overall distribution shift between the UAV view and the satellite view.

C-ICL: Contrastive In-context Learning for Information Extraction

no code implementations17 Feb 2024 Ying Mo, Jian Yang, Jiahao Liu, Shun Zhang, Jingang Wang, Zhoujun Li

Recently, there has been increasing interest in exploring the capabilities of advanced large language models (LLMs) in the field of information extraction (IE), specifically focusing on tasks related to named entity recognition (NER) and relation extraction (RE).

In-Context Learning Miscellaneous +4

What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbation

no code implementations11 Mar 2024 Zhuocheng Gong, Jiahao Liu, Jingang Wang, Xunliang Cai, Dongyan Zhao, Rui Yan

Our findings reveal several connections between the properties of perturbations and LLM performance, providing insights into the failure cases of uniform quantization and suggesting potential solutions to improve the robustness of LLM quantization.

Computational Efficiency Quantization

Cannot find the paper you are looking for? You can Submit a new open access paper.