Search Results for author: Weijie Liu

Found 23 papers, 12 papers with code

TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities

3 code implementations13 Dec 2022 Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Guo, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan

The proposed pre-training models of different modalities are showing a rising trend of homogeneity in their model structures, which brings the opportunity to implement different pre-training models within a uniform framework.

K-BERT: Enabling Language Representation with Knowledge Graph

2 code implementations arXiv 2019 Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, Ping Wang

For machines to achieve this capability, we propose a knowledge-enabled language representation model (K-BERT) with knowledge graphs (KGs), in which triples are injected into the sentences as domain knowledge.

Knowledge Graphs Sentence

Whitening Sentence Representations for Better Semantics and Faster Retrieval

3 code implementations29 Mar 2021 Jianlin Su, Jiarun Cao, Weijie Liu, Yangyiwen Ou

Therefore, some attempts of boosting the isotropy of sentence distribution, such as flow-based model, have been applied to sentence representations and achieved some improvement.

Retrieval Sentence

Merak: An Efficient Distributed DNN Training Framework with Automated 3D Parallelism for Giant Foundation Models

1 code implementation10 Jun 2022 Zhiquan Lai, Shengwei Li, Xudong Tang, Keshi Ge, Weijie Liu, Yabo Duan, Linbo Qiao, Dongsheng Li

These features make it necessary to apply 3D parallelism, which integrates data parallelism, pipeline model parallelism and tensor model parallelism, to achieve high training efficiency.

Dynamic Relevance Learning for Few-Shot Object Detection

1 code implementation4 Aug 2021 Weijie Liu, Chong Wang, Haohe Li, Shenghao Yu, Jiafei Wu

By adjusting the prediction distribution of the base detector using the output of this GCN, the proposed model serves as a hard auxiliary classification task, which guides the detector to improve the class representation implicitly.

Few-Shot Object Detection Meta-Learning +2

Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching

1 code implementation Findings (NAACL) 2022 Kunbo Ding, Weijie Liu, Yuejian Fang, Zhe Zhao, Qi Ju, Xuefeng Yang

Previous studies have proved that cross-lingual knowledge distillation can significantly improve the performance of pre-trained models for cross-lingual similarity matching tasks.

Contrastive Learning Knowledge Distillation +3

Semantic Matching from Different Perspectives

1 code implementation14 Feb 2022 Weijie Liu, Tao Zhu, Weiquan Mao, Zhe Zhao, Weigang Guo, Xuefeng Yang, Qi Ju

In this paper, we pay attention to the issue which is usually overlooked, i. e., \textit{similarity should be determined from different perspectives}.

Sentence Text Matching +1

A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning

1 code implementation COLING 2022 Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, Yiren Chen

Existing zero-shot cross-lingual transfer methods rely on parallel corpora or bilingual dictionaries, which are expensive and impractical for low-resource languages.

text-classification Text Classification +3

CDMA: A Practical Cross-Device Federated Learning Algorithm for General Minimax Problems

1 code implementation29 May 2021 Jiahao Xie, Chao Zhang, Zebang Shen, Weijie Liu, Hui Qian

We establish theoretical guarantees of CDMA under different choices of hyperparameters and conduct experiments on AUC maximization, robust adversarial network training, and GAN training tasks.

Federated Learning Generative Adversarial Network

A Decentralized Proximal Point-type Method for Saddle Point Problems

no code implementations31 Oct 2019 Weijie Liu, Aryan Mokhtari, Asuman Ozdaglar, Sarath Pattathil, Zebang Shen, Nenggan Zheng

In this paper, we focus on solving a class of constrained non-convex non-concave saddle point problems in a decentralized manner by a group of nodes in a network.

Vocal Bursts Type Prediction

A Bidirectional Tree Tagging Scheme for Joint Medical Relation Extraction

no code implementations31 Aug 2020 Xukun Luo, Weijie Liu, Meng Ma, Ping Wang

In this paper, inspired by the tree-like relation structures in the medical text, we propose a novel scheme called Bidirectional Tree Tagging (BiTT) to form the medical relation triples into two two binary trees and convert the trees into a word-level tags sequence.

Medical Relation Extraction Relation +1

Partial Gromov-Wasserstein Learning for Partial Graph Matching

no code implementations2 Dec 2020 Weijie Liu, Chao Zhang, Jiahao Xie, Zebang Shen, Hui Qian, Nenggan Zheng

Graph matching finds the correspondence of nodes across two graphs and is a basic task in graph-based machine learning.

Graph Matching

Approximating Optimal Transport via Low-rank and Sparse Factorization

no code implementations12 Nov 2021 Weijie Liu, Chao Zhang, Nenggan Zheng, Hui Qian

Optimal transport (OT) naturally arises in a wide range of machine learning applications but may often become the computational bottleneck.

SIGMA: A Structural Inconsistency Reducing Graph Matching Algorithm

no code implementations6 Feb 2022 Weijie Liu, Chao Zhang, Nenggan Zheng, Hui Qian

In this paper, we propose a novel criterion to measure the graph matching accuracy, structural inconsistency (SI), which is defined based on the network topological structure.

Graph Matching

Parameter-efficient Continual Learning Framework in Industrial Real-time Text Classification System

no code implementations NAACL (ACL) 2022 Tao Zhu, Zhe Zhao, Weijie Liu, Jiachi Liu, Yiren Chen, Weiquan Mao, Haoyan Liu, Kunbo Ding, Yudong Li, Xuefeng Yang

Catastrophic forgetting is a challenge for model deployment in industrial real-time systems, which requires the model to quickly master a new task without forgetting the old one.

Continual Learning text-classification +1

SAMP: A Model Inference Toolkit of Post-Training Quantization for Text Processing via Self-Adaptive Mixed-Precision

no code implementations19 Sep 2022 Rong Tian, Zijing Zhao, Weijie Liu, Haoyan Liu, Weiquan Mao, Zhe Zhao, Kan Zhou

The latest industrial inference engines, such as FasterTransformer and TurboTransformers, have verified that half-precision floating point (FP16) and 8-bit integer (INT8) quantization can greatly improve model inference speed.

Quantization

Recouple Event Field via Probabilistic Bias for Event Extraction

no code implementations19 May 2023 Xingyu Bai, Taiqiang Wu, Han Guo, Zhe Zhao, Xuefeng Yang, Jiayi Li, Weijie Liu, Qi Ju, Weigang Guo, Yujiu Yang

Event Extraction (EE), aiming to identify and classify event triggers and arguments from event mentions, has benefited from pre-trained language models (PLMs).

Event Extraction

Towards Optimal Randomized Strategies in Adversarial Example Game

no code implementations29 Jun 2023 Jiahao Xie, Chao Zhang, Weijie Liu, Wensong Bai, Hui Qian

The vulnerability of deep neural network models to adversarial example attacks is a practical challenge in many artificial intelligence applications.

DYNAMITE: Dynamic Interplay of Mini-Batch Size and Aggregation Frequency for Federated Learning with Static and Streaming Dataset

no code implementations20 Oct 2023 Weijie Liu, Xiaoxi Zhang, Jingpu Duan, Carlee Joe-Wong, Zhi Zhou, Xu Chen

Federated Learning (FL) is a distributed learning paradigm that can coordinate heterogeneous edge devices to perform model training without sharing private data.

Federated Learning Navigate

Cannot find the paper you are looking for? You can Submit a new open access paper.