Search Results for author: Canwen Xu

Found 15 papers, 11 papers with code

Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression

1 code implementation7 Sep 2021 Canwen Xu, Wangchunshu Zhou, Tao Ge, Ke Xu, Julian McAuley, Furu Wei

Recent studies on compression of pretrained language models (e. g., BERT) usually use preserved accuracy as the metric for evaluation.

Knowledge Distillation Quantization

Meta Learning for Knowledge Distillation

1 code implementation8 Jun 2021 Wangchunshu Zhou, Canwen Xu, Julian McAuley

We present Meta Learning for Knowledge Distillation (MetaDistil), a simple yet effective alternative to traditional knowledge distillation (KD) methods where the teacher model is fixed during training.

Knowledge Distillation Meta-Learning

Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting

no code implementations2 Jan 2021 Wangchunshu Zhou, Tao Ge, Canwen Xu, Ke Xu, Furu Wei

In this paper, we generalize text infilling (e. g., masked language models) by proposing Sequence Span Rewriting (SSR) as a self-supervised sequence-to-sequence (seq2seq) pre-training objective.

Text Infilling

BERT Loses Patience: Fast and Robust Inference with Early Exit

1 code implementation NeurIPS 2020 Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, Furu Wei

In this paper, we propose Patience-based Early Exit, a straightforward yet effective inference method that can be used as a plug-and-play technique to simultaneously improve the efficiency and robustness of a pretrained language model (PLM).

Language Modelling

Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders

1 code implementation ACL 2020 Yu Duan, Canwen Xu, Jiaxin Pei, Jialong Han, Chenliang Li

Conditional Text Generation has drawn much attention as a topic of Natural Language Generation (NLG) which provides the possibility for humans to control the properties of generated contents.

Conditional Text Generation

Exploiting Multiple Embeddings for Chinese Named Entity Recognition

1 code implementation28 Aug 2019 Canwen Xu, Feiyang Wang, Jialong Han, Chenliang Li

Identifying the named entities mentioned in text would enrich many semantic applications at the downstream level.

Chinese Named Entity Recognition NER

Obj-GloVe: Scene-Based Contextual Object Embedding

no code implementations2 Jul 2019 Canwen Xu, Zhenzhong Chen, Chenliang Li

Recently, with the prevalence of large-scale image dataset, the co-occurrence information among classes becomes rich, calling for a new way to exploit it to facilitate inference.

Dimensionality Reduction Image Generation +1

DLocRL: A Deep Learning Pipeline for Fine-Grained Location Recognition and Linking in Tweets

no code implementations21 Jan 2019 Canwen Xu, Jing Li, Xiangyang Luo, Jiaxin Pei, Chenliang Li, Donghong Ji

Recognizing and linking such fine-grained location mentions to well-defined location profiles are beneficial for retrieval and recommendation systems.

Recommendation Systems Representation Learning +1

Cannot find the paper you are looking for? You can Submit a new open access paper.