Search Results for author: Weijie Liu

Found 20 papers, 11 papers with code

Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System

no code implementations NAACL (ACL) 2022 Tao Zhu, Zhe Zhao, Weijie Liu, Jiachi Liu, Yiren Chen, Weiquan Mao, Haoyan Liu, Kunbo Ding, Yudong Li, Xuefeng Yang

Catastrophic forgetting is a challenge for model deployment in industrial real-time systems, which requires the model to quickly master a new task without forgetting the old one.

Continual Learning text-classification +1

TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities

1 code implementation13 Dec 2022 Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Guo, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan

The proposed pre-training models of different modalities are showing a rising trend of homogeneity in their model structures, which brings the opportunity to implement different pre-training models within a uniform framework.

A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning

1 code implementation COLING 2022 Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen

Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries, which are expensive and impractical for low-resource languages.

text-classification Text Classification +3

SAMP: A Toolkit for Model Inference with Self-Adaptive Mixed-Precision

no code implementations19 Sep 2022 Rong Tian, Zijing Zhao, Weijie Liu, Haoyan Liu, Weiquan Mao, Zhe Zhao, Kimmo Yan

The latest industrial inference engines, such as FasterTransformer1 and TurboTransformers, have verified that half-precision floating point (FP16) and 8-bit integer (INT8) quantization can greatly improve model inference speed.


Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching

1 code implementation Findings (NAACL) 2022 Kunbo Ding, Weijie Liu, Yuejian Fang, Zhe Zhao, Qi Ju, Xuefeng Yang

Previous studies have proved that cross-lingual knowledge distillation can significantly improve the performance of pre-trained models for cross-lingual similarity matching tasks.

Contrastive Learning Knowledge Distillation +3

Merak: An Efficient Distributed DNN Training Framework with Automated 3D Parallelism for Giant Foundation Models

1 code implementation10 Jun 2022 Zhiquan Lai, Shengwei Li, Xudong Tang, Keshi Ge, Weijie Liu, Yabo Duan, Linbo Qiao, Dongsheng Li

These features make it necessary to apply 3D parallelism, which integrates data parallelism, pipeline model parallelism and tensor model parallelism, to achieve high training efficiency.

Semantic Matching from Different Perspectives

1 code implementation14 Feb 2022 Weijie Liu, Tao Zhu, Weiquan Mao, Zhe Zhao, Weigang Guo, Xuefeng Yang, Qi Ju

In this paper, we pay attention to the issue which is usually overlooked, i. e., \textit{similarity should be determined from different perspectives}.

Text Matching text similarity

SIGMA: A Structural Inconsistency Reducing Graph Matching Algorithm

no code implementations6 Feb 2022 Weijie Liu, Chao Zhang, Nenggan Zheng, Hui Qian

In this paper, we propose a novel criterion to measure the graph matching accuracy, structural inconsistency (SI), which is defined based on the network topological structure.

Graph Matching

Approximating Optimal Transport via Low-rank and Sparse Factorization

no code implementations12 Nov 2021 Weijie Liu, Chao Zhang, Nenggan Zheng, Hui Qian

Optimal transport (OT) naturally arises in a wide range of machine learning applications but may often become the computational bottleneck.

Dynamic Relevance Learning for Few-Shot Object Detection

1 code implementation4 Aug 2021 Weijie Liu, Chong Wang, Haohe Li, Shenghao Yu, Jiangbo Qian, Jun Wang, Jiafei Wu

By adjusting the prediction distribution of the base detector using the output of this GCN, the proposed model serves as a hard auxiliary classification task, which guides the detector to improve the class representation implicitly.

Few-Shot Object Detection Meta-Learning +1

Efficient Cross-Device Federated Learning Algorithms for Minimax Problems

no code implementations29 May 2021 Jiahao Xie, Chao Zhang, Zebang Shen, Weijie Liu, Hui Qian

In many machine learning applications where massive and privacy-sensitive data are generated on numerous mobile or IoT devices, collecting data in a centralized location may be prohibitive.

Federated Learning

Whitening Sentence Representations for Better Semantics and Faster Retrieval

3 code implementations29 Mar 2021 Jianlin Su, Jiarun Cao, Weijie Liu, Yangyiwen Ou

Therefore, some attempts of boosting the isotropy of sentence distribution, such as flow-based model, have been applied to sentence representations and achieved some improvement.


Partial Gromov-Wasserstein Learning for Partial Graph Matching

no code implementations2 Dec 2020 Weijie Liu, Chao Zhang, Jiahao Xie, Zebang Shen, Hui Qian, Nenggan Zheng

Graph matching finds the correspondence of nodes across two graphs and is a basic task in graph-based machine learning.

Graph Matching

A Bidirectional Tree Tagging Scheme for Joint Medical Relation Extraction

no code implementations31 Aug 2020 Xukun Luo, Weijie Liu, Meng Ma, Ping Wang

In this paper, inspired by the tree-like relation structures in the medical text, we propose a novel scheme called Bidirectional Tree Tagging (BiTT) to form the medical relation triples into two two binary trees and convert the trees into a word-level tags sequence.

Medical Relation Extraction Relation Extraction

A Decentralized Proximal Point-type Method for Saddle Point Problems

no code implementations31 Oct 2019 Weijie Liu, Aryan Mokhtari, Asuman Ozdaglar, Sarath Pattathil, Zebang Shen, Nenggan Zheng

In this paper, we focus on solving a class of constrained non-convex non-concave saddle point problems in a decentralized manner by a group of nodes in a network.

K-BERT: Enabling Language Representation with Knowledge Graph

2 code implementations arXiv 2019 Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, Ping Wang

For machines to achieve this capability, we propose a knowledge-enabled language representation model (K-BERT) with knowledge graphs (KGs), in which triples are injected into the sentences as domain knowledge.

Knowledge Graphs

Cannot find the paper you are looking for? You can Submit a new open access paper.