1 code implementation • ICML 2020 • chengyu dong, Liyuan Liu, Zichao Li, Jingbo Shang
Serving as a crucial factor, the depth of residual networks balances model capacity, performance, and training efficiency.
no code implementations • 14 Jun 2022 • chengyu dong, Liyuan Liu, Jingbo Shang
To fill this gap, we propose a novel student-oriented teacher network training framework SoTeacher, inspired by recent findings that student performance hinges on teacher's capability to approximate the true label distribution of training samples.
no code implementations • 15 Feb 2022 • Sha Li, Liyuan Liu, Yiqing Xie, Heng Ji, Jiawei Han
Our framework decomposes event detection into an identification task and a localization task.
no code implementations • 7 Oct 2021 • chengyu dong, Liyuan Liu, Jingbo Shang
We show that label noise exists in adversarial training.
no code implementations • 29 Sep 2021 • Zichao Li, Liyuan Liu, chengyu dong, Jingbo Shang
While this phenomenon is commonly explained as overfitting, we observe that it is a twin process: not only does the model catastrophic overfits to one type of perturbation, but also the perturbation deteriorates into random noise.
1 code implementation • 21 Jun 2021 • Tao Chen, Haochen Shi, Liyuan Liu, Siliang Tang, Jian Shao, Zhigang Chen, Yueting Zhuang
In this paper, we propose collaborative adversarial training to improve the data utilization, which coordinates virtual adversarial training (VAT) and adversarial training (AT) at different levels.
1 code implementation • 17 Jun 2021 • Liyuan Liu, Jialu Liu, Jiawei Han
Multi-head attention plays a crucial role in the recent success of Transformer models, which leads to consistent performance improvements over conventional attention in various applications.
2 code implementations • 28 May 2021 • Xiaotao Gu, Zihan Wang, Zhenyu Bi, Yu Meng, Liyuan Liu, Jiawei Han, Jingbo Shang
Training a conventional neural tagger based on silver labels usually faces the risk of overfitting phrase surface names.
Ranked #1 on Phrase Tagging on KPTimes
1 code implementation • 15 Feb 2021 • chengyu dong, Liyuan Liu, Jingbo Shang
Specifically, we first propose a strategy to measure the data quality based on the learning behaviors of the data during adversarial training and find that low-quality data may not be useful and even detrimental to the adversarial robustness.
no code implementations • NAACL 2021 • Xiaotao Gu, Liyuan Liu, Hongkun Yu, Jing Li, Chen Chen, Jiawei Han
Due to the excessive cost of large-scale language model pre-training, considerable efforts have been made to train BERT progressively -- start from an inferior but low-cost model and gradually grow the model to increase the computational complexity.
2 code implementations • 15 Oct 2020 • Zichao Li, Liyuan Liu, chengyu dong, Jingbo Shang
Our goal is to understand why the robustness drops after conducting adversarial training for too long.
4 code implementations • 18 Aug 2020 • Xiaodong Liu, Kevin Duh, Liyuan Liu, Jianfeng Gao
We explore the application of very deep Transformer models for Neural Machine Translation (NMT).
Ranked #1 on Machine Translation on WMT2014 English-French (using extra training data)
no code implementations • 1 May 2020 • Shi Zhi, Liyuan Liu, Yu Zhang, Shiyin Wang, Qi Li, Chao Zhang, Jiawei Han
While typical named entity recognition (NER) models require the training set to be annotated with all target types, each available datasets may only cover a part of them.
2 code implementations • EMNLP 2020 • Liyuan Liu, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Jiawei Han
Transformers have proved effective in many NLP tasks.
Ranked #5 on Machine Translation on WMT2014 English-French
no code implementations • 27 Dec 2019 • Mingxin Zhao, Li Cheng, Xu Yang, Peng Feng, Liyuan Liu, Nanjian Wu
Meanwhile, we propose a joint loss function and a training method.
1 code implementation • ACL 2020 • Ouyu Lan, Xiao Huang, Bill Yuchen Lin, He Jiang, Liyuan Liu, Xiang Ren
Its performance is largely influenced by the annotation quality and quantity in supervised learning scenarios, and obtaining ground truth labels is often costly.
1 code implementation • IJCNLP 2019 • Zihan Wang, Jingbo Shang, Liyuan Liu, Lihao Lu, Jiacheng Liu, Jiawei Han
Therefore, we manually correct these label mistakes and form a cleaner test set.
Ranked #3 on Named Entity Recognition (NER) on CoNLL++ (using extra training data)
1 code implementation • ACL 2020 • Yuning Mao, Liyuan Liu, Qi Zhu, Xiang Ren, Jiawei Han
In this paper, we present a facet-aware evaluation setup for better assessment of the information coverage in extracted summaries.
1 code implementation • 14 Aug 2019 • Liyuan Liu, Zihan Wang, Jingbo Shang, Dandong Yin, Heng Ji, Xiang Ren, Shaowen Wang, Jiawei Han
Our model neither requires the conversion from character sequences to word sequences, nor assumes tokenizer can correctly detect all word boundaries.
19 code implementations • ICLR 2020 • Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, Jiawei Han
The learning rate warmup heuristic achieves remarkable success in stabilizing training, accelerating convergence and improving generalization for adaptive stochastic optimization algorithms like RMSprop and Adam.
no code implementations • WS 2019 • Liyuan Liu, Jingbo Shang, Jiawei Han
This paper presents the winning solution to the Arabic Named Entity Recognition challenge run by Topcoder. com.
1 code implementation • ACL 2019 • Ying Lin, Liyuan Liu, Heng Ji, Dong Yu, Jiawei Han
We design a set of word frequency-based reliability signals to indicate the quality of each word embedding.
1 code implementation • IJCNLP 2019 • Qinyuan Ye, Liyuan Liu, Maosen Zhang, Xiang Ren
In this paper, we study the problem what limits the performance of DS-trained neural models, conduct thorough analyses, and identify a factor that can influence the performance greatly, shifted label distribution.
1 code implementation • 27 Dec 2018 • Yujin Yuan, Liyuan Liu, Siliang Tang, Zhongfei Zhang, Yueting Zhuang, ShiLiang Pu, Fei Wu, Xiang Ren
Distant supervision leverages knowledge bases to automatically label instances, thus allowing us to train relation extractor without human annotations.
1 code implementation • EMNLP 2018 • Jingbo Shang, Liyuan Liu, Xiang Ren, Xiaotao Gu, Teng Ren, Jiawei Han
Recent advances in deep neural models allow us to build reliable named entity recognition (NER) systems without handcrafting features.
1 code implementation • EMNLP 2018 • Liyuan Liu, Xiang Ren, Jingbo Shang, Jian Peng, Jiawei Han
Many efforts have been made to facilitate natural language processing tasks with pre-trained language models (LMs), and brought significant improvements to various applications.
Ranked #45 on Named Entity Recognition (NER) on CoNLL 2003 (English)
no code implementations • 9 Mar 2018 • Huan Gui, Qi Zhu, Liyuan Liu, Aston Zhang, Jiawei Han
We study the task of expert finding in heterogeneous bibliographical networks based on two aspects: textual content analysis and authority ranking.
1 code implementation • 21 Dec 2017 • Carl Yang, Mengxiong Liu, Zongyi Wang, Liyuan Liu, Jiawei Han
Unlike most existing embedding methods that are task-agnostic, we simultaneously solve for the underlying node representations and the optimal clustering assignments in an end-to-end manner.
Social and Information Networks Physics and Society
3 code implementations • 13 Sep 2017 • Liyuan Liu, Jingbo Shang, Frank F. Xu, Xiang Ren, Huan Gui, Jian Peng, Jiawei Han
In this study, we develop a novel neural framework to extract abundant knowledge hidden in raw texts to empower the sequence labeling task.
Ranked #13 on Part-Of-Speech Tagging on Penn Treebank
1 code implementation • EMNLP 2017 • Liyuan Liu, Xiang Ren, Qi Zhu, Shi Zhi, Huan Gui, Heng Ji, Jiawei Han
These annotations, referred as heterogeneous supervision, often conflict with each other, which brings a new challenge to the original relation extraction task: how to infer the true label from noisy labels for a given instance.