no code implementations • IJCNLP 2015 • Fandong Meng, Zhengdong Lu, Mingxuan Wang, Hang Li, Wenbin Jiang, Qun Liu
The recently proposed neural network joint model (NNJM) (Devlin et al., 2014) augments the n-gram target language model with a heuristically chosen source context window, achieving state-of-the-art performance in SMT.
no code implementations • 9 Mar 2015 • Mingxuan Wang, Zhengdong Lu, Hang Li, Qun Liu
Many tasks in natural language processing, ranging from machine translation to question answering, can be reduced to the problem of matching two sentences or more generally two short texts.
no code implementations • 17 Mar 2015 • Mingxuan Wang, Zhengdong Lu, Hang Li, Wenbin Jiang, Qun Liu
Different from previous work on neural network-based language modeling and generation (e. g., RNN or LSTM), we choose not to greedily summarize the history of words as a fixed length vector.
no code implementations • 22 Jun 2015 • Fandong Meng, Zhengdong Lu, Zhaopeng Tu, Hang Li, Qun Liu
We propose DEEPMEMORY, a novel deep architecture for sequence-to-sequence learning, which performs the task through a series of nonlinear transformations from the representation of the input sequence (e. g., a Chinese sentence) to the final output sequence (e. g., translation to English).
no code implementations • 9 Aug 2015 • Hui Yu, Xiaofeng Wu, Wenbin Jiang, Qun Liu, ShouXun Lin
To avoid these problems, we propose a novel automatic evaluation metric based on dependency parsing model, with no need to define sub-structures by human.
no code implementations • 10 Aug 2015 • Hui Yu, Xiaofeng Wu, Wenbin Jiang, Qun Liu, ShouXun Lin
The widely-used automatic evaluation metrics cannot adequately reflect the fluency of the translations.
1 code implementation • EMNLP 2016 • Biao Zhang, Deyi Xiong, Jinsong Su, Qun Liu, Rongrong Ji, Hong Duan, Min Zhang
In order to perform efficient inference and learning, we introduce neural discourse relation models to approximate the prior and posterior distributions of the latent variable, and employ these approximated distributions to optimize a reparameterized variational lower bound.
no code implementations • NAACL 2016 • Long-Yue Wang, Zhaopeng Tu, Xiaojun Zhang, Hang Li, Andy Way, Qun Liu
Finally, we integrate the above outputs into our translation system to recall missing pronouns by both extracting rules from the DP-labelled training data and translating the DP-generated input sentences.
no code implementations • LREC 2016 • Xiaofeng Wu, Jinhua Du, Qun Liu, Andy Way
This paper presents ProphetMT, a tree-based SMT-driven Controlled Language (CL) authoring and post-editing tool.
no code implementations • LREC 2016 • Long-Yue Wang, Xiaojun Zhang, Zhaopeng Tu, Andy Way, Qun Liu
Then tags such as speaker and discourse boundary from the script data are projected to its subtitle data via an information retrieval approach in order to map monolingual discourse to bilingual texts.
no code implementations • EMNLP 2016 • Mingxuan Wang, Zhengdong Lu, Hang Li, Qun Liu
We propose to enhance the RNN decoder in a neural machine translator (NMT) with external memory, as a natural but powerful extension to the state in the decoding RNN.
no code implementations • COLING 2016 • Fandong Meng, Zhengdong Lu, Hang Li, Qun Liu
Conventional attention-based Neural Machine Translation (NMT) conducts dynamic alignment in generating the target sentence.
no code implementations • COLING 2016 • Peyman Passban, Qun Liu, Andy Way
PBSMT engines by default provide four probability scores in phrase tables which are considered as the main set of bilingual features.
no code implementations • COLING 2016 • Jian Zhang, Xiaofeng Wu, Andy Way, Qun Liu
We show that the neural LM perplexity can be reduced by 7. 395 and 12. 011 using the proposed domain adaptation mechanism on the Penn Treebank and News data, respectively.
no code implementations • COLING 2016 • Jian Zhang, Liangyou Li, Andy Way, Qun Liu
In recent years, neural machine translation (NMT) has demonstrated state-of-the-art machine translation (MT) performance.
no code implementations • COLING 2016 • Qiuye Zhao, Qun Liu
For Chinese, the most notable increase is as high as 3. 63 (UAS) when the proposed framework is applied to first-order parsing models.
no code implementations • 23 Jan 2017 • Iacer Calixto, Qun Liu, Nick Campbell
We introduce multi-modal, attention-based neural machine translation (NMT) models which incorporate visual features into different parts of both the encoder and the decoder.
Ranked #10 on Multimodal Machine Translation on Multi30K
no code implementations • 3 Feb 2017 • Iacer Calixto, Qun Liu, Nick Campbell
We propose a novel discriminative model that learns embeddings from multilingual and multi-modal data, meaning that our model can take advantage of images and descriptions in multiple languages to improve embedding quality.
no code implementations • ACL 2017 • Iacer Calixto, Qun Liu, Nick Campbell
We introduce a Multi-modal Neural Machine Translation model in which a doubly-attentive decoder naturally incorporates spatial visual features obtained using pre-trained convolutional neural networks, bridging the gap between image description and translation.
Ranked #11 on Multimodal Machine Translation on Multi30K
no code implementations • WS 2017 • Alfredo Maldonado, Lifeng Han, Erwan Moreau, Ashjan Alsulaimani, Koel Dutta Chowdhury, Carl Vogel, Qun Liu
A description of a system for identifying Verbal Multi-Word Expressions (VMWEs) in running text is presented.
no code implementations • EACL 2017 • Yvette Graham, Qingsong Ma, Timothy Baldwin, Qun Liu, Carla Parra, Carolina Scarton
Meaningful conclusions about the relative performance of NLP systems are only possible if the gold standard employed in a given evaluation is both valid and reliable.
no code implementations • EACL 2017 • Santanu Pal, Sudip Kumar Naskar, Mihaela Vela, Qun Liu, Josef van Genabith
APE translations produced by our system show statistically significant improvements over the first-stage MT, phrase-based APE and the best reported score on the WMT 2016 APE dataset by a previous neural APE system.
no code implementations • EACL 2017 • Liangyou Li, Andy Way, Qun Liu
In this paper, we present an improved graph-based translation model which segments an input graph into node-induced subgraphs by taking source context into consideration.
no code implementations • EACL 2017 • Dasha Bogdanova, Jennifer Foster, Daria Dzendzik, Qun Liu
We show that a neural approach to the task of non-factoid answer reranking can benefit from the inclusion of tried-and-tested handcrafted features.
1 code implementation • EMNLP 2017 • Long-Yue Wang, Zhaopeng Tu, Andy Way, Qun Liu
In translation, considering the document as a whole can help to resolve ambiguities and inconsistencies.
1 code implementation • ACL 2017 • Chris Hokamp, Qun Liu
Lexical constraints take the form of phrases or words that must be present in the output sequence.
no code implementations • ACL 2017 • Mingxuan Wang, Zhengdong Lu, Jie zhou, Qun Liu
Deep Neural Networks (DNNs) have provably enhanced the state-of-the-art Neural Machine Translation (NMT) with their capability in modeling complex functions and capturing complex linguistic structures.
no code implementations • ACL 2017 • Jinchao Zhang, Mingxuan Wang, Qun Liu, Jie zhou
This paper proposes three distortion models to explicitly incorporate the word reordering knowledge into attention-based Neural Machine Translation (NMT) for further improving translation performance.
no code implementations • RANLP 2017 • Iacer Calixto, Qun Liu
We propose a novel discriminative ranking model that learns embeddings from multilingual and multi-modal data, meaning that our model can take advantage of images and descriptions in multiple languages to improve embedding quality.
no code implementations • WS 2017 • Ond{\v{r}}ej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shu-Jian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, Marco Turchi
1 code implementation • EMNLP 2017 • Qingsong Ma, Yvette Graham, Timothy Baldwin, Qun Liu
Monolingual evaluation of Machine Translation (MT) aims to simplify human assessment by requiring assessors to compare the meaning of the MT output with a reference translation, opening up the task to a much larger pool of genuinely qualified evaluators.
no code implementations • EMNLP 2017 • Iacer Calixto, Qun Liu
We introduce multi-modal, attention-based neural machine translation (NMT) models which incorporate visual features into different parts of both the encoder and the decoder.
no code implementations • 6 Sep 2017 • Wen Zhang, Jiawei Hu, Yang Feng, Qun Liu
Even though sequence-to-sequence neural machine translation (NMT) model have achieved state-of-art performance in the recent fewer years, but it is widely concerned that the recurrent neural network (RNN) units are very hard to capture the long-distance state information, which means RNN can hardly find the feature with long term dependency as the sequence becomes longer.
no code implementations • 12 Sep 2017 • Wen Zhang, Jiawei Hu, Yang Feng, Qun Liu
Although neural machine translation (NMT) with the encoder-decoder framework has achieved great success in recent times, it still suffers from some drawbacks: RNNs tend to forget old information which is often useful and the encoder only operates through words without considering word relationship.
1 code implementation • 17 Oct 2017 • Jiawei Hu, Qun Liu
We participated in the MLWS 2017 on Tibetan word segmentation task, our system is trained in a unrestricted way, by introducing a baseline system and 76w tibetan segmented sentences of ours.
no code implementations • IJCNLP 2017 • Long-Yue Wang, Jinhua Du, Liangyou Li, Zhaopeng Tu, Andy Way, Qun Liu
We showcase TODAY, a semantics-enhanced task-oriented dialogue translation system, whose novelties are: (i) task-oriented named entity (NE) definition and a hybrid strategy for NE recognition and translation; and (ii) a novel grounded semantic method for dialogue understanding and task-order management.
no code implementations • IJCNLP 2017 • Daria Dzendzik, Alberto Poncelas, Carl Vogel, Qun Liu
We describe the work of a team from the ADAPT Centre in Ireland in addressing automatic answer selection for the Multi-choice Question Answering in Examinations shared task.
1 code implementation • 10 Jan 2018 • Long-Yue Wang, Zhaopeng Tu, Shuming Shi, Tong Zhang, Yvette Graham, Qun Liu
Next, the annotated source sentence is reconstructed from hidden representations in the NMT model.
no code implementations • NAACL 2018 • Peyman Passban, Qun Liu, Andy Way
Recently, neural machine translation (NMT) has emerged as a powerful alternative to conventional statistical approaches.
no code implementations • 2 May 2018 • Qun Liu, Supratik Mukhopadhyay
In this paper, we present a new architecture and an approach for unsupervised object recognition that addresses the above mentioned problem with fine tuning associated with pretrained CNN-based supervised deep learning approaches while allowing automated feature extraction.
Ranked #1 on Fine-Grained Image Classification on Caltech-101 (Accuracy metric)
Few-Shot Image Classification Fine-Grained Image Classification +2
no code implementations • 3 May 2018 • Qun Liu, Suman Kumar, Vijay Mago
This paper proposes SafeRNet, a safe route computation framework which takes advantage of these technologies to analyze streaming traffic data and historical data to effectively infer safe routes and deliver them back to users in real time.
no code implementations • COLING 2018 • Wen Zhang, Jiawei Hu, Yang Feng, Qun Liu
Although neural machine translation with the encoder-decoder framework has achieved great success recently, it still suffers drawbacks of forgetting distant information, which is an inherent disadvantage of recurrent neural network structure, and disregarding relationship between source words during encoding step.
no code implementations • 5 Jun 2018 • Chao-Hong Liu, Declan Groves, Akira Hayakawa, Alberto Poncelas, Qun Liu
Understanding and being able to react to customer feedback is the most fundamental task in providing good customer service.
no code implementations • 21 Jun 2018 • Manohar Karki, Qun Liu, Robert DiBiano, Saikat Basu, Supratik Mukhopadhyay
Classification techniques for images of handwritten characters are susceptible to noise.
Ranked #1 on Document Image Classification on n-MNIST
1 code implementation • ACL 2018 • Shuman Liu, Hongshen Chen, Zhaochun Ren, Yang Feng, Qun Liu, Dawei Yin
Our empirical study on a real-world dataset prove that our model is capable of generating meaningful, diverse and natural responses for both factoid-questions and knowledge grounded chi-chats.
no code implementations • WS 2018 • Koel Dutta Chowdhury, Mohammed Hasanuzzaman, Qun Liu
In this paper, we investigate the effectiveness of training a multimodal neural machine translation (MNMT) system with image features for a low-resource language pair, Hindi and English, using synthetic data.
no code implementations • COLING 2018 • Peyman Passban, Andy Way, Qun Liu
A morphologically complex word (MCW) is a hierarchical constituent with meaning-preserving subunits, so word-based models which rely on surface forms might not be powerful enough to translate such structures.
no code implementations • EMNLP 2018 • Wen Zhang, Liang Huang, Yang Feng, Lei Shen, Qun Liu
Although neural machine translation has achieved promising results, it suffers from slow translation speed.
no code implementations • EMNLP 2018 • Long-Yue Wang, Zhaopeng Tu, Andy Way, Qun Liu
Pronouns are frequently omitted in pro-drop languages, such as Chinese, generally leading to significant challenges with respect to the production of complete translations.
no code implementations • WS 2018 • Henry Elder, Sebastian Gehrmann, Alex O{'}Connor, er, Qun Liu
In natural language generation (NLG), the task is to generate utterances from a more abstract input, such as structured data.
no code implementations • 2 Nov 2018 • Xiang Li, Haiyang Xue, Wei Chen, Yang Liu, Yang Feng, Qun Liu
Although neural machine translation (NMT) has achieved impressive progress recently, it is usually trained on the clean parallel data set and hence cannot work well when the input sentence is the production of the automatic speech recognition (ASR) system due to the enormous errors in the source.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +5
no code implementations • 27 Mar 2019 • Qun Liu, Supratik Mukhopadhyay, Yimin Zhu, Ravindra Gudishala, Sanaz Saeidi, Alimire Nabijiang
High fidelity route choice models are required to predict traffic levels with higher accuracy.
no code implementations • NAACL 2019 • Shuhao Gu, Yang Feng, Qun Liu
Besides, we add a discriminator to the shared encoder and employ adversarial training for the whole model to reinforce the performance of information separation and machine translation simultaneously.
no code implementations • WS 2019 • Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar, Qun Liu, Mehdi Rezagholizadeh
Latent space based GAN methods and attention based sequence to sequence models have achieved impressive results in text generation and unsupervised machine translation respectively.
2 code implementations • ACL 2019 • Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, Qun Liu
Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP tasks.
Ranked #1 on Entity Linking on FIGER
no code implementations • ACL 2019 • Wen Zhang, Yang Feng, Fandong Meng, Di You, Qun Liu
Neural Machine Translation (NMT) generates target words sequentially in the way of predicting the next word conditioned on the context words.
no code implementations • ACL 2019 • Zichao Li, Xin Jiang, Lifeng Shang, Qun Liu
Paraphrasing exists at different granularity levels, such as lexical level, phrasal level and sentential level.
2 code implementations • 29 Jun 2019 • Yi Liao, Yasheng Wang, Qun Liu, Xin Jiang
We present a simple yet effective method for generating high quality classical Chinese poetry with Generative Pre-trained Language Model (GPT).
1 code implementation • ACL 2019 • Fanchao Qi, Jun-Jie Huang, Chenghao Yang, Zhiyuan Liu, Xiao Chen, Qun Liu, Maosong Sun
In this paper, we verify the effectiveness of sememes, the minimum semantic units of human languages, in modeling SC by a confirmatory experiment.
multi-word expression embedding multi-word expression sememe prediction
no code implementations • WS 2019 • Wei Peng, Jianfeng Liu, Liangyou Li, Qun Liu
This paper describes Huawei{'}s neural machine translation systems for the WMT 2019 biomedical translation shared task.
no code implementations • 11 Aug 2019 • Qun Liu, Edward Collier, Supratik Mukhopadhyay
We show that by learning the features at each resolution independently a trained model is able to accurately classify characters even in the presence of noise.
Ranked #1 on Image Classification on Noisy MNIST (AWGN)
no code implementations • 21 Aug 2019 • Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu
Neural dialog state trackers are generally limited due to the lack of quantity and diversity of annotated training data.
10 code implementations • 31 Aug 2019 • Junqiu Wei, Xiaozhe Ren, Xiaoguang Li, Wenyong Huang, Yi Liao, Yasheng Wang, Jiashu Lin, Xin Jiang, Xiao Chen, Qun Liu
The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized information in text by pre-training on large-scale corpora.
7 code implementations • Findings of the Association for Computational Linguistics 2020 • Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu
To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based models.
Ranked #1 on Natural Language Inference on MultiNLI Dev
1 code implementation • 20 Oct 2019 • Yujia Qin, Fanchao Qi, Sicong Ouyang, Zhiyuan Liu, Cheng Yang, Yasheng Wang, Qun Liu, Maosong Sun
Sememes, the minimum semantic units of human languages, have been successfully utilized in various natural language processing applications.
1 code implementation • ACL 2020 • Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, Maosong Sun
Also, further experiments show our model has higher transferability and can bring more robustness enhancement to victim models by adversarial training.
no code implementations • Asian Chapter of the Association for Computational Linguistics 2020 • Yun Chen, Liangyou Li, Xin Jiang, Xiao Chen, Qun Liu
Despite the success of neural machine translation (NMT), simultaneous neural machine translation (SNMT), the task of translating in real time before a full sentence has been observed, remains challenging due to the syntactic structure difference and simultaneity requirements.
no code implementations • 8 Nov 2019 • Liangyou Li, Xin Jiang, Qun Liu
Previous work on document-level NMT usually focuses on limited contexts because of degraded performance on larger contexts.
no code implementations • 9 Nov 2019 • Yinpeng Guo, Yi Liao, Xin Jiang, Qing Zhang, Yibo Zhang, Qun Liu
Leveraging multilingual parallel texts to automatically generate paraphrases has drawn much attention as size of high-quality paraphrase corpus is limited.
1 code implementation • 15 Nov 2019 • Qun Liu, Saikat Basu, Sangram Ganguly, Supratik Mukhopadhyay, Robert DiBiano, Manohar Karki, Ramakrishna Nemani
Satellite image classification is a challenging problem that lies at the crossroads of remote sensing, computer vision, and machine learning.
Ranked #1 on Satellite Image Classification on SAT-4
no code implementations • 20 Nov 2019 • Qun Liu, Lihua Fu, Meng Zhang
Synthetic and field data were tested to assess the performance of the proposed algorithm (DSPRecon algorithm); the advantages of using our method were evaluated by comparing it with the singular spectrum analysis (SSA) method for irregular data reconstruction and de-aliased Cadzow method for regular data reconstruction.
no code implementations • 30 Nov 2019 • Bin He, Di Zhou, Jinghui Xiao, Xin Jiang, Qun Liu, Nicholas Jing Yuan, Tong Xu
Complex node interactions are common in knowledge graphs, and these interactions also contain rich knowledge information.
no code implementations • 5 Dec 2019 • Gang Chen, Yang Liu, Huanbo Luan, Meng Zhang, Qun Liu, Maosong Sun
While the use of neural networks has proven effective in improving story generation, how to learn to generate an explainable high-level plot still remains a major challenge.
no code implementations • 8 Dec 2019 • Qun Liu, Subhashis Hazarika, John M. Patchett, James Paul Ahrens, Ayan Biswas
Data modeling and reduction for in situ is important.
1 code implementation • 18 Dec 2019 • Lei Zhang, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Qun Liu, Maosong Sun
A reverse dictionary takes the description of a target word as input and outputs the target word together with other words that match the description.
no code implementations • 7 Jan 2020 • Supratik Mukhopadhyay, Qun Liu, Edward Collier, Yimin Zhu, Ravindra Gudishala, Chanachok Chokwitthaya, Robert DiBiano, Alimire Nabijiang, Sanaz Saeidi, Subhajit Sidhanta, Arnab Ganguly
The impacts of context factors driving human system interaction are challenging and are difficult to capture and replicate in existing design models.
no code implementations • 23 Jan 2020 • Qun Liu, Supratik Mukhopadhyay, Maria Ximena Bastidas Rodriguez, Xing Fu, Sushant Sahu, David Burk, Manas Gartia
Myocardial infarction (MI) is a scientific term that refers to heart attack.
no code implementations • 6 Apr 2020 • Wei Peng, Chongxuan Huang, Tian-Hao Li, Yun Chen, Qun Liu
Existing data augmentation approaches for neural machine translation (NMT) have predominantly relied on back-translating in-domain (IND) monolingual corpora.
3 code implementations • NeurIPS 2020 • Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu
The pre-trained language models like BERT, though powerful in many natural language processing tasks, are both computation and memory expensive.
3 code implementations • ACL 2020 • Yi Liao, Xin Jiang, Qun Liu
Masked language model and autoregressive language model are two types of language models.
1 code implementation • EMNLP 2020 • Yun Chen, Yang Liu, Guanhua Chen, Xin Jiang, Qun Liu
Shift-Att is an interpretation method that induces alignments from the attention weights of Transformer and does not require parameter update or architecture change.
1 code implementation • ACL 2020 • Zhiyong Wu, Yun Chen, Ben Kao, Qun Liu
However, this approach of evaluating a language model is undermined by the uncertainty of the amount of knowledge that is learned by the probe itself.
no code implementations • 8 May 2020 • Meng Zhang, Xin Jiang, Yang Liu, Qun Liu
In this work, we put machine translation in a cross-lingual pipeline and introduce downstream tasks to define task-specific acceptability of machine translations.
no code implementations • 28 Jul 2020 • Shuai Zhang, Peng Zhang, Xindian Ma, Junqiu Wei, Ningning Wang, Qun Liu
Transformer has been widely-used in many Natural Language Processing (NLP) tasks and the scaled dot-product attention between tokens is a core module of Transformer.
5 code implementations • EMNLP 2020 • Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, Qun Liu
Transformer-based pre-training models like BERT have achieved remarkable performance in many natural language processing tasks. However, these models are both computation and memory expensive, hindering their deployment to resource-constrained devices.
no code implementations • 2 Oct 2020 • Yang Bai, Xiaoguang Li, Gang Wang, Chaoliang Zhang, Lifeng Shang, Jun Xu, Zhaowei Wang, Fangshan Wang, Qun Liu
Term-based sparse representations dominate the first-stage text retrieval in industrial applications, due to its advantage in efficiency, interpretability, and exact term matching.
2 code implementations • EMNLP 2020 • Yimeng Wu, Peyman Passban, Mehdi Rezagholizade, Qun Liu
With the growth of computing power neural machine translation (NMT) models also grow accordingly and become better.
no code implementations • 13 Oct 2020 • Qun Liu, Matthew Shreve, Raja Bala
Although data is abundant, data labeling is expensive.
3 code implementations • Findings of the Association for Computational Linguistics 2020 • Yudong Zhu, Di Zhou, Jinghui Xiao, Xin Jiang, Xiao Chen, Qun Liu
Natural language data exhibit tree-like hierarchical structures such as the hypernym-hyponym relations in WordNet.
1 code implementation • Findings of the Association for Computational Linguistics 2020 • Jie He, Tao Wang, Deyi Xiong, Qun Liu
Our experiments and analyses demonstrate that neural machine translation performs poorly on commonsense reasoning of the three ambiguity types in terms of both reasoning accuracy ( 6 60. 1{\%}) and reasoning consistency (6 31{\%}).
no code implementations • Findings of the Association for Computational Linguistics 2020 • Bin He, Di Zhou, Jinghui Xiao, Xin Jiang, Qun Liu, Nicholas Jing Yuan, Tong Xu
Complex node interactions are common in knowledge graphs (KGs), and these interactions can be considered as contextualized knowledge exists in the topological structure of KGs.
no code implementations • 7 Nov 2020 • Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Qun Liu, Maosong Sun
To measure the informativeness of attention heads, we train our Single-Shot Meta-Pruner (SMP) with a meta-learning paradigm aiming to maintain the distribution of text representations after pruning.
no code implementations • 10 Nov 2020 • Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar, Qun Liu, Mehdi Rezagholizadeh
B-GAN is able to generate a distributed latent space representation which can be paired with an attention based decoder to generate fluent sentences.
no code implementations • 7 Dec 2020 • Bin He, Di Zhou, Jing Xie, Jinghui Xiao, Xin Jiang, Qun Liu
Entities may have complex interactions in a knowledge graph (KG), such as multi-step relationships, which can be viewed as graph contextual information of the entities.
no code implementations • EMNLP 2021 • Mingzhou Xu, Liangyou Li, Derek. F. Wong, Qun Liu, Lidia S. Chao
Previous works have shown that contextual information can improve the performance of neural machine translation (NMT).
no code implementations • 7 Dec 2020 • Bin He, Xin Jiang, Jinghui Xiao, Qun Liu
Recent studies on pre-trained language models have demonstrated their ability to capture factual knowledge and applications in knowledge-aware downstream tasks.
no code implementations • 11 Dec 2020 • Xiaoqi Jiao, Huating Chang, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu
Comprehensive experiments on the evaluation benchmarks demonstrate that 1) layer mapping strategy has a significant effect on task-agnostic BERT distillation and different layer mappings can result in quite different performances; 2) the optimal layer mapping strategy from the proposed search process consistently outperforms the other heuristic ones; 3) with the optimal layer mapping, our student model achieves state-of-the-art performance on the GLUE tasks.
no code implementations • 27 Dec 2020 • Peyman Passban, Yimeng Wu, Mehdi Rezagholizadeh, Qun Liu
Knowledge distillation is considered as a training and compression strategy in which two neural networks, namely a teacher and a student, are coupled together during training.
no code implementations • 31 Dec 2020 • Shaobo Li, Xiaoguang Li, Lifeng Shang, Xin Jiang, Qun Liu, Chengjie Sun, Zhenzhou Ji, Bingquan Liu
In this paper, we propose a new retrieval target, hop, to collect the hidden reasoning evidence from Wikipedia for complex question answering.
Ranked #5 on Question Answering on HotpotQA
no code implementations • Findings (EMNLP) 2021 • Peyman Passban, Puneeth S. M. Saladi, Qun Liu
There is a large body of work in the NMT literature on analyzing the behavior of conventional models for the problem of noise but Transformers are relatively understudied in this context.
1 code implementation • ACL 2021 • Haoli Bai, Wei zhang, Lu Hou, Lifeng Shang, Jing Jin, Xin Jiang, Qun Liu, Michael Lyu, Irwin King
In this paper, we propose BinaryBERT, which pushes BERT quantization to the limit by weight binarization.
1 code implementation • 31 Dec 2020 • Chenglei Si, Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Qun Liu, Maosong Sun
In this work, we propose a simple and effective method to cover a much larger proportion of the attack search space, called Adversarial and Mixup Data Augmentation (AMDA).
no code implementations • ICLR 2021 • Benyou Wang, Lifeng Shang, Christina Lioma, Xin Jiang, Hao Yang, Qun Liu, Jakob Grue Simonsen
Various Position Embeddings (PEs) have been proposed in Transformer based architectures~(e. g. BERT) to model word order.
1 code implementation • 23 Jan 2021 • Junqiu Wei, Qun Liu, Yinpeng Guo, Xin Jiang
The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized information in text by pre-training on large-scale corpora.
no code implementations • 11 Mar 2021 • Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, Qun Liu
The multilingual pre-trained language models (e. g, mBERT, XLM and XLM-R) have shown impressive performance on cross-lingual natural language understanding tasks.
1 code implementation • ICLR 2021 • Mingyang Yi, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, Zhi-Ming Ma
Inspired by adversarial training, we minimize this maximal expected loss (MMEL) and obtain a simple and interpretable closed-form solution: more attention should be paid to augmented samples with large loss values (i. e., harder examples).
no code implementations • 20 Mar 2021 • Liangyou Li, Andy Way, Qun Liu
We present graph-based translation models which translate source graphs into target strings.
no code implementations • 25 Mar 2021 • Tong Cui, Jinghui Xiao, Liangyou Li, Xin Jiang, Qun Liu
Speech-enabled systems typically first convert audio to text through an automatic speech recognition (ASR) model and then feed the text to downstream natural language processing (NLP) modules.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +5
no code implementations • 18 Apr 2021 • Krtin Kumar, Peyman Passban, Mehdi Rezagholizadeh, Yiu Sing Lau, Qun Liu
Embedding matrices are key components in neural natural language processing (NLP) models that are responsible to provide numerical representations of input tokens.\footnote{In this paper words and subwords are referred to as \textit{tokens} and the term \textit{embedding} only refers to embeddings of inputs.}
no code implementations • 24 Apr 2021 • Cheng Chen, Yichun Yin, Lifeng Shang, Zhi Wang, Xin Jiang, Xiao Chen, Qun Liu
Task-agnostic knowledge distillation, a teacher-student framework, has been proved effective for BERT compression.
4 code implementations • 26 Apr 2021 • Wei Zeng, Xiaozhe Ren, Teng Su, Hui Wang, Yi Liao, Zhiwei Wang, Xin Jiang, ZhenZhang Yang, Kaisheng Wang, Xiaoda Zhang, Chen Li, Ziyan Gong, Yifan Yao, Xinjing Huang, Jun Wang, Jianfeng Yu, Qi Guo, Yue Yu, Yan Zhang, Jin Wang, Hengtao Tao, Dasen Yan, Zexuan Yi, Fang Peng, Fangqing Jiang, Han Zhang, Lingfeng Deng, Yehong Zhang, Zhe Lin, Chao Zhang, Shaojie Zhang, Mingyue Guo, Shanzhi Gu, Gaojun Fan, YaoWei Wang, Xuefeng Jin, Qun Liu, Yonghong Tian
To enhance the generalization ability of PanGu-$\alpha$, we collect 1. 1TB high-quality Chinese data from a wide range of domains to pretrain the model.
Ranked #1 on Reading Comprehension (One-Shot) on DuReader
Cloze (multi-choices) (Few-Shot) Cloze (multi-choices) (One-Shot) +19
1 code implementation • Findings (ACL) 2021 • Silin Gao, Ryuichi Takanobu, Wei Peng, Qun Liu, Minlie Huang
To address this task, we propose a TOD system with hybrid knowledge management, HyKnow.
1 code implementation • 14 May 2021 • Zhixing Tan, Zeyuan Yang, Meng Zhang, Qun Liu, Maosong Sun, Yang Liu
With the rapid development of artificial intelligence (AI), there is a trend in moving AI applications, such as neural machine translation (NMT), from cloud to mobile devices.
no code implementations • 24 May 2021 • Mingyang Yi, Lu Hou, Jiacheng Sun, Lifeng Shang, Xin Jiang, Qun Liu, Zhi-Ming Ma
In this paper, after defining OOD generalization via Wasserstein distance, we theoretically show that a model robust to input perturbation generalizes well on OOD data.
no code implementations • 1 Jun 2021 • Xingshan Zeng, Liangyou Li, Qun Liu
We use a unified transformer architecture for our MultiST model, so that the data from different modalities (i. e., speech and text) and different tasks (i. e., Speech Recognition, Machine Translation, and Speech Translation) can be exploited to enhance the model's ability.
2 code implementations • 1 Jun 2021 • Chenglei Si, Zhengyan Zhang, Yingfa Chen, Fanchao Qi, Xiaozhi Wang, Zhiyuan Liu, Yasheng Wang, Qun Liu, Maosong Sun
2) Pronunciation-based SubChar tokenizers can encode Chinese homophones into the same transliteration sequences and produce the same tokenization output, hence being robust to homophone typos.
no code implementations • 9 Jun 2021 • Yinpeng Guo, Liangyou Li, Xin Jiang, Qun Liu
Recently, pre-training multilingual language models has shown great potential in learning multilingual representation, a crucial topic of natural language processing.
no code implementations • Findings (ACL) 2021 • Xingshan Zeng, Liangyou Li, Qun Liu
To bridge the modality gap between speech and text, RealTranS gradually downsamples the input speech with interleaved convolution and unidirectional Transformer layers for acoustic modeling, and then maps speech features into text space with a weighted-shrinking operation and a semantic encoder.
1 code implementation • ACL 2021 • Zhihong Shao, Lifeng Shang, Qun Liu, Minlie Huang
This setting gives rise to the spurious solution problem: there may exist many spurious solutions that coincidentally derive the correct answer, but training on such solutions can hurt model performance (e. g., producing wrong solutions or answers).
1 code implementation • ACL 2021 • Yichun Yin, Cheng Chen, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu
Specifically, we carefully design the techniques of one-shot learning and the search space to provide an adaptive and efficient development way of tiny PLMs for various latency constraints.
no code implementations • ACL 2021 • Jie He, Bo Peng, Yi Liao, Qun Liu, Deyi Xiong
Each error is hence manually labeled with comprehensive annotations, including the span of the error, the associated span, minimal correction to the error, the type of the error, and rationale behind the error.
no code implementations • ACL 2021 • Zhiqi Huang, Lu Hou, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu
Transformer-based pre-trained language models like BERT, though powerful in many tasks, are expensive in both memory and computation, due to their large number of parameters.
no code implementations • EMNLP 2021 • Minghao Wu, Yitong Li, Meng Zhang, Liangyou Li, Gholamreza Haffari, Qun Liu
In this work, we propose an approach, MultiUAT, that dynamically adjusts the training data usage based on the model's uncertainty on a small set of trusted clean data for multi-corpus machine translation.
no code implementations • Findings (EMNLP) 2021 • Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, Qun Liu
Math word problem (MWP) is a challenging and critical task in natural language processing.
Ranked #2 on Math Word Problem Solving on Math23K
no code implementations • 7 Sep 2021 • Zhihua Jin, Xin Jiang, Xingbo Wang, Qun Liu, Yong Wang, Xiaozhe Ren, Huamin Qu
However, those models do not consider the numerical properties of numbers and cannot perform robustly on numerical reasoning tasks (e. g., math word problems and measurement estimation).
no code implementations • 7 Sep 2021 • Shaobo Li, Qun Liu, Xin Jiang, Yichun Yin, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Lifeng Shang
Human-designed rules are widely used to build industry applications.
1 code implementation • 9 Sep 2021 • Yinquan Lu, Haonan Lu, Guirong Fu, Qun Liu
Incorporating factual knowledge into pre-trained language models (PLM) such as BERT is an emerging trend in recent NLP studies.
Ranked #11 on Common Sense Reasoning on ReCoRD
no code implementations • 10 Sep 2021 • Fei Mi, Yitong Li, Yasheng Wang, Xin Jiang, Qun Liu
As labeling cost for different modules in task-oriented dialog (ToD) systems is high, a major challenge in practice is to learn different tasks with the least amount of labeled data.
no code implementations • 13 Sep 2021 • Zhengkun Zhang, Xiaojun Meng, Yasheng Wang, Xin Jiang, Qun Liu, Zhenglu Yang
Specially, we adopt knowledge distillation from a vision-language pretrained model to improve image selection, which avoids any requirement on the existence and quality of image captions.
no code implementations • EMNLP 2021 • Chenyang Lyu, Lifeng Shang, Yvette Graham, Jennifer Foster, Xin Jiang, Qun Liu
Template-based QG uses linguistically-informed heuristics to transform declarative sentences into interrogatives, whereas supervised QG uses existing Question Answering (QA) datasets to train a system to generate a question given a passage and an answer.
1 code implementation • EMNLP 2021 • Baojun Wang, Zhao Zhang, Kun Xu, Guang-Yuan Hao, Yuyang Zhang, Lifeng Shang, Linlin Li, Xiao Chen, Xin Jiang, Qun Liu
Incorporating lexical knowledge into deep learning models has been proved to be very effective for sequence labeling tasks.
no code implementations • 28 Sep 2021 • Qianmengke Zhao, Ye Wang, Qun Liu
Although deep learning models are powerful among various applications, most deep learning models are still a black box, lacking verifiability and interpretability, which means the decision-making process that human beings cannot understand.
no code implementations • 29 Sep 2021 • Chao Xing, Dong Wang, LiRong Dai, Qun Liu, Anderson Avila
Overparameterized transformer-based architectures have shown remarkable performance in recent years, achieving state-of-the-art results in speech processing tasks such as speech recognition, speech synthesis, keyword spotting, and speech enhancement et al.
no code implementations • ACL 2022 • Cheng Chen, Yichun Yin, Lifeng Shang, Xin Jiang, Yujia Qin, Fengyu Wang, Zhi Wang, Xiao Chen, Zhiyuan Liu, Qun Liu
However, large language model pre-training costs intensive computational resources and most of the models are trained from scratch without reusing the existing pre-trained models, which is wasteful.
no code implementations • dialdoc (ACL) 2022 • Xinyan Zhao, Bin He, Yasheng Wang, Yitong Li, Fei Mi, Yajiao Liu, Xin Jiang, Qun Liu, Huanhuan Chen
With the advances in deep learning, tremendous progress has been made with chit-chat dialogue systems and task-oriented dialogue systems.
no code implementations • 16 Nov 2021 • Nianzu Zheng, Liqun Deng, Wenyong Huang, Yu Ting Yeung, Baohua Xu, Yuanyuan Guo, Yasheng Wang, Xiao Chen, Xin Jiang, Qun Liu
We utilize conv-transformer structure to encode input speech in a streaming manner.
1 code implementation • 22 Nov 2021 • Zihan Yan, Li Liu, Xin Li, William K. Cheung, Youmin Zhang, Qun Liu, Guoyin Wang
Social network alignment aims at aligning person identities across social networks.
1 code implementation • 8 Dec 2021 • Abbas Ghaddar, Yimeng Wu, Ahmad Rashid, Khalil Bibi, Mehdi Rezagholizadeh, Chao Xing, Yasheng Wang, Duan Xinyu, Zhefeng Wang, Baoxing Huai, Xin Jiang, Qun Liu, Philippe Langlais
Language-specific pre-trained models have proven to be more accurate than multilingual ones in a monolingual evaluation setting, Arabic is no exception.
no code implementations • Findings (NAACL) 2022 • Mengjie Zhao, Fei Mi, Yasheng Wang, Minglei Li, Xin Jiang, Qun Liu, Hinrich Schütze
We propose LMTurk, a novel approach that treats few-shot learners as crowdsourcing workers.
1 code implementation • ICLR 2022 • Wenyong Huang, Zhenhe Zhang, Yu Ting Yeung, Xin Jiang, Qun Liu
The student network is trained to output representation resembling that of the teacher.