Search Results for author: Deyi Xiong

Found 88 papers, 15 papers with code

Re-embedding Difficult Samples via Mutual Information Constrained Semantically Oversampling for Imbalanced Text Classification

no code implementations EMNLP 2021 Jiachen Tian, Shizhan Chen, Xiaowang Zhang, Zhiyong Feng, Deyi Xiong, Shaojuan Wu, Chunliu Dou

Difficult samples of the minority class in imbalanced text classification are usually hard to be classified as they are embedded into an overlapping semantic region with the majority class.

Text Classification

TED-CDB: A Large-Scale Chinese Discourse Relation Dataset on TED Talks

no code implementations EMNLP 2020 Wanqiu Long, Bonnie Webber, Deyi Xiong

As different genres are known to differ in their communicative properties and as previously, for Chinese, discourse relations have only been annotated over news text, we have created the TED-CDB dataset.

Transfer Learning

KaFSP: Knowledge-Aware Fuzzy Semantic Parsing for Conversational Question Answering over a Large-Scale Knowledge Base

no code implementations ACL 2022 Junzhuo Li, Deyi Xiong

In this paper, we study two issues of semantic parsing approaches to conversational question answering over a large-scale knowledge base: (1) The actions defined in grammar are not sufficient to handle uncertain reasoning common in real-world scenarios.

Conversational Question Answering Entity Disambiguation +2

Adaptive Differential Privacy for Language Model Training

no code implementations FL4NLP (ACL) 2022 Xinwei Wu, Li Gong, Deyi Xiong

Although differential privacy (DP) can protect language models from leaking privacy, its indiscriminative protection on all data points reduces its practical utility.

Language Modelling

A Test Suite for Evaluating Discourse Phenomena in Document-level Neural Machine Translation

no code implementations AACL (iwdp) 2020 Xinyi Cai, Deyi Xiong

The need to evaluate the ability of context-aware neural machine translation (NMT) models in dealing with specific discourse phenomena arises in document-level NMT.

Machine Translation Translation

Chinese WPLC: A Chinese Dataset for Evaluating Pretrained Language Models on Word Prediction Given Long-Range Context

no code implementations EMNLP 2021 Huibin Ge, Chenxi Sun, Deyi Xiong, Qun Liu

Experiment results show that the Chinese pretrained language model PanGu-\alpha is 45 points behind human in terms of top-1 word prediction accuracy, indicating that Chinese WPLC is a challenging dataset.

Language Modelling Pretrained Language Models

GEMv2: Multilingual NLG Benchmarking in a Single Line of Code

no code implementations22 Jun 2022 Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina McMillan-Major, Anna Shvets, Ashish Upadhyay, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez-Beltrachini, Leonardo F. R. Ribeiro, Lewis Tunstall, Li Zhang, Mahima Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou

This problem is especially pertinent in natural language generation which requires ever-improving suites of datasets, metrics, and human evaluation to make definitive claims.

Text Generation

Unsupervised and Few-shot Parsing from Pretrained Language Models

no code implementations10 Jun 2022 Zhiyuan Zeng, Deyi Xiong

We therefore extend the unsupervised models to few-shot parsing models (FPOA, FPIO) that use a few annotated trees to learn better linear projection matrices for parsing.

Language Modelling Pretrained Language Models

Efficient Cluster-Based k-Nearest-Neighbor Machine Translation

1 code implementation ACL 2022 Dexin Wang, Kai Fan, Boxing Chen, Deyi Xiong

k-Nearest-Neighbor Machine Translation (kNN-MT) has been recently proposed as a non-parametric solution for domain adaptation in neural machine translation (NMT).

Contrastive Learning Domain Adaptation +2

Bridging between Cognitive Processing Signals and Linguistic Features via a Unified Attentional Network

no code implementations16 Dec 2021 Yuqi Ren, Deyi Xiong

The proposed framework only requires cognitive processing signals recorded under natural reading as inputs, and can be used to detect a wide range of linguistic features with a single cognitive dataset.

Natural Language Processing

Secoco: Self-Correcting Encoding for Neural Machine Translation

no code implementations Findings (EMNLP) 2021 Tao Wang, Chengqi Zhao, Mingxuan Wang, Lei LI, Hang Li, Deyi Xiong

This paper presents Self-correcting Encoding (Secoco), a framework that effectively deals with input noise for robust neural machine translation by introducing self-correcting predictors.

Machine Translation Translation

TGEA: An Error-Annotated Dataset and Benchmark Tasks for TextGeneration from Pretrained Language Models

no code implementations ACL 2021 Jie He, Bo Peng, Yi Liao, Qun Liu, Deyi Xiong

Each error is hence manually labeled with comprehensive annotations, including the span of the error, the associated span, minimal correction to the error, the type of the error, and rationale behind the error.

Common Sense Reasoning Pretrained Language Models +1

Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation

no code implementations ACL 2021 Hongfei Xu, Qiuhui Liu, Josef van Genabith, Deyi Xiong, Meng Zhang

This has to be computed n times for a sequence of length n. The linear transformations involved in the LSTM gate and state computations are the major cost factors in this.

Machine Translation Translation

An Empirical Study on Adversarial Attack on NMT: Languages and Positions Matter

no code implementations ACL 2021 Zhiyuan Zeng, Deyi Xiong

For autoregressive NMT models that generate target words from left to right, we observe that adversarial attack on the source language is more effective than on the target language, and that attacking front positions of target sentences or positions of source sentences aligned to the front positions of corresponding target sentences is more effective than attacking other positions.

Adversarial Attack

CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signals

1 code implementation ACL 2021 Yuqi Ren, Deyi Xiong

Most previous studies integrate cognitive language processing signals (e. g., eye-tracking or EEG data) into neural models of natural language processing (NLP) just by directly concatenating word embeddings with cognitive features, ignoring the gap between the two modalities (i. e., textual vs. cognitive) and noise in cognitive features.

EEG named-entity-recognition +5

Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learning

1 code implementation5 Mar 2021 Jinsong Su, Jialong Tang, Hui Jiang, Ziyao Lu, Yubin Ge, Linfeng Song, Deyi Xiong, Le Sun, Jiebo Luo

In aspect-based sentiment analysis (ABSA), many neural models are equipped with an attention mechanism to quantify the contribution of each context word to sentiment prediction.

Aspect-Based Sentiment Analysis

Efficient Object-Level Visual Context Modeling for Multimodal Machine Translation: Masking Irrelevant Objects Helps Grounding

no code implementations18 Dec 2020 Dexin Wang, Deyi Xiong

In this paper, we propose an object-level visual context modeling framework (OVC) to efficiently capture and explore visual information for multimodal machine translation.

Multimodal Machine Translation Translation

Balanced Joint Adversarial Training for Robust Intent Detection and Slot Filling

no code implementations COLING 2020 Xu Cao, Deyi Xiong, Chongyang Shi, Chao Wang, Yao Meng, Changjian Hu

Joint intent detection and slot filling has recently achieved tremendous success in advancing the performance of utterance understanding.

Intent Detection Slot Filling

A Learning-Exploring Method to Generate Diverse Paraphrases with Multi-Objective Deep Reinforcement Learning

no code implementations COLING 2020 Mingtong Liu, Erguang Yang, Deyi Xiong, Yujie Zhang, Yao Meng, Changjian Hu, Jinan Xu, Yufeng Chen

We propose a learning-exploring method to generate sentences as learning objectives from the learned data distribution, and employ reinforcement learning to combine these new learning objectives for model training.

Natural Language Processing Paraphrase Generation +1

The Box is in the Pen: Evaluating Commonsense Reasoning in Neural Machine Translation

1 code implementation Findings of the Association for Computational Linguistics 2020 Jie He, Tao Wang, Deyi Xiong, Qun Liu

Our experiments and analyses demonstrate that neural machine translation performs poorly on commonsense reasoning of the three ambiguity types in terms of both reasoning accuracy ( 6 60. 1{\%}) and reasoning consistency (6 31{\%}).

Common Sense Reasoning Machine Translation +1

RiSAWOZ: A Large-Scale Multi-Domain Wizard-of-Oz Dataset with Rich Semantic Annotations for Task-Oriented Dialogue Modeling

1 code implementation EMNLP 2020 Jun Quan, Shian Zhang, Qian Cao, Zizhong Li, Deyi Xiong

In order to alleviate the shortage of multi-domain data and to capture discourse phenomena for task-oriented dialogue modeling, we propose RiSAWOZ, a large-scale multi-domain Chinese Wizard-of-Oz dataset with Rich Semantic Annotations.

Dialogue State Tracking Intent Detection +3

Transformer with Depth-Wise LSTM

no code implementations13 Jul 2020 Hongfei Xu, Qiuhui Liu, Deyi Xiong, Josef van Genabith

In this paper, we suggest that the residual connection has its drawbacks, and propose to train Transformers with the depth-wise LSTM which regards outputs of layers as steps in time series instead of residual connections, under the motivation that the vanishing gradient problem suffered by deep networks is the same as recurrent networks applied to long sequences, while LSTM (Hochreiter and Schmidhuber, 1997) has been proven of good capability in capturing long-distance relationship, and its design may alleviate some drawbacks of residual connections while ensuring the convergence.

Time Series

Learning Source Phrase Representations for Neural Machine Translation

no code implementations ACL 2020 Hongfei Xu, Josef van Genabith, Deyi Xiong, Qiuhui Liu, Jingyi Zhang

Considering that modeling phrases instead of words has significantly improved the Statistical Machine Translation (SMT) approach through the use of larger translation blocks ("phrases") and its reordering ability, modeling NMT at phrase level is an intuitive proposal to help the model capture long-distance relationships.

Machine Translation Translation

Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change

no code implementations ACL 2020 Hongfei Xu, Josef van Genabith, Deyi Xiong, Qiuhui Liu

We propose to automatically and dynamically determine batch sizes by accumulating gradients of mini-batches and performing an optimization step at just the time when the direction of gradients starts to fluctuate.

Modeling Long Context for Task-Oriented Dialogue State Generation

no code implementations ACL 2020 Jun Quan, Deyi Xiong

Based on the recently proposed transferable dialogue state generator (TRADE) that predicts dialogue states from utterance-concatenated dialogue context, we propose a multi-task learning model with a simple yet effective utterance tagging technique and a bidirectional language model as an auxiliary task for task-oriented dialogue state generation.

Language Modelling Multi-Task Learning

Learning Contextualized Sentence Representations for Document-Level Neural Machine Translation

no code implementations30 Mar 2020 Pei Zhang, Xu Zhang, Wei Chen, Jian Yu, Yan-Feng Wang, Deyi Xiong

In this paper, we propose a new framework to model cross-sentence dependencies by training neural machine translation (NMT) to predict both the target translation and surrounding sentences of a source sentence.

Document Level Machine Translation Machine Translation +2

Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers

no code implementations NAACL 2021 Hongfei Xu, Josef van Genabith, Qiuhui Liu, Deyi Xiong

Due to its effectiveness and performance, the Transformer translation model has attracted wide attention, most recently in terms of probing-based approaches.

Translation Word Translation

Shallow Discourse Annotation for Chinese TED Talks

1 code implementation LREC 2020 Wanqiu Long, Xinyi Cai, James E. M. Reid, Bonnie Webber, Deyi Xiong

Text corpora annotated with language-related properties are an important resource for the development of Language Technology.


Merging External Bilingual Pairs into Neural Machine Translation

no code implementations2 Dec 2019 Tao Wang, Shaohui Kuang, Deyi Xiong, António Branco

As neural machine translation (NMT) is not easily amenable to explicit correction of errors, incorporating pre-specified translations into NMT is widely regarded as a non-trivial challenge.

Machine Translation Translation

Learning to Reuse Translations: Guiding Neural Machine Translation with Examples

no code implementations25 Nov 2019 Qian Cao, Shaohui Kuang, Deyi Xiong

In this paper, we study the problem of enabling neural machine translation (NMT) to reuse previous translations from similar examples in target prediction.

Machine Translation Translation

Lipschitz Constrained Parameter Initialization for Deep Transformers

no code implementations ACL 2020 Hongfei Xu, Qiuhui Liu, Josef van Genabith, Deyi Xiong, Jingyi Zhang

In this paper, we first empirically demonstrate that a simple modification made in the official implementation, which changes the computation order of residual connection and layer normalization, can significantly ease the optimization of deep Transformers.


BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels

1 code implementation IJCNLP 2019 Yimin Jing, Deyi Xiong, Yan Zhen

We analyze BiPaR in depth and find that BiPaR offers good diversification in prefixes of questions, answer types and relationships between questions and passages.

Coreference Resolution Machine Reading Comprehension

Generating Highly Relevant Questions

no code implementations IJCNLP 2019 Jiazuo Qiu, Deyi Xiong

The neural seq2seq based question generation (QG) is prone to generating generic and undiversified questions that are poorly relevant to the given passage and target answer.

Question Generation

Towards Linear Time Neural Machine Translation with Capsule Networks

no code implementations IJCNLP 2019 Mingxuan Wang, Jun Xie, Zhixing Tan, Jinsong Su, Deyi Xiong, Lei LI

In this study, we first investigate a novel capsule network with dynamic routing for linear time Neural Machine Translation (NMT), referred as \textsc{CapsNMT}.

Machine Translation Translation

Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks

3 code implementations EMNLP 2018 Biao Zhang, Deyi Xiong, Jinsong Su, Qian Lin, Huiji Zhang

Experiments on WMT14 translation tasks demonstrate that ATR-based neural machine translation can yield competitive performance on English- German and English-French language pairs in terms of both translation quality and speed.

Chinese Word Segmentation Machine Translation +3

Sentence Weighting for Neural Machine Translation Domain Adaptation

no code implementations COLING 2018 Shiqi Zhang, Deyi Xiong

In this paper, we propose a new sentence weighting method for the domain adaptation of neural machine translation.

Domain Adaptation Language Modelling +2

Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Model

no code implementations COLING 2018 Shaohui Kuang, Deyi Xiong

Neural machine translation (NMT) systems are usually trained on a large amount of bilingual sentence pairs and translate one sentence at a time, ignoring inter-sentence information.

Machine Translation Translation

Accelerating Neural Transformer via an Average Attention Network

1 code implementation ACL 2018 Biao Zhang, Deyi Xiong, Jinsong Su

To alleviate this issue, we propose an average attention network as an alternative to the self-attention network in the decoder of the neural Transformer.


Variational Recurrent Neural Machine Translation

no code implementations16 Jan 2018 Jinsong Su, Shan Wu, Deyi Xiong, Yaojie Lu, Xianpei Han, Biao Zhang

Partially inspired by successful applications of variational recurrent neural networks, we propose a novel variational recurrent neural machine translation (VRNMT) model in this paper.

Machine Translation Translation

Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings

no code implementations ACL 2018 Shaohui Kuang, Junhui Li, António Branco, Weihua Luo, Deyi Xiong

In neural machine translation, a source sequence of words is encoded into a vector from which a target sequence is generated in the decoding phase.

Machine Translation Translation +1

Modeling Source Syntax for Neural Machine Translation

no code implementations ACL 2017 Junhui Li, Deyi Xiong, Zhaopeng Tu, Muhua Zhu, Min Zhang, Guodong Zhou

Even though a linguistics-free sequence to sequence model in neural machine translation (NMT) has certain capability of implicitly learning syntactic information of source sentences, this paper shows that source syntax can be explicitly incorporated into NMT effectively to provide further improvements.

Machine Translation Translation

A GRU-Gated Attention Model for Neural Machine Translation

no code implementations27 Apr 2017 Biao Zhang, Deyi Xiong, Jinsong Su

In this paper, we propose a novel GRU-gated attention model (GAtt) for NMT which enhances the degree of discrimination of context vectors by enabling source representations to be sensitive to the partial translation generated by the decoder.

Machine Translation Translation

Improving Translation Selection with Supersenses

no code implementations COLING 2016 Haiqing Tang, Deyi Xiong, Oier Lopez de Lacalle, Eneko Agirre

Selecting appropriate translations for source words with multiple meanings still remains a challenge for statistical machine translation (SMT).

Machine Translation Translation +1

Improving Statistical Machine Translation with Selectional Preferences

no code implementations COLING 2016 Haiqing Tang, Deyi Xiong, Min Zhang, ZhengXian Gong

In this paper, we study semantic dependencies between verbs and their arguments by modeling selectional preferences in the context of machine translation.

Machine Translation Semantic Role Labeling +2

Learning Event Expressions via Bilingual Structure Projection

no code implementations COLING 2016 Fangyuan Li, Ruihong Huang, Deyi Xiong, Min Zhang

Aiming to resolve high complexities of event descriptions, previous work (Huang and Riloff, 2013) proposes multi-faceted event recognition and a bootstrapping method to automatically acquire both event facet phrases and event expressions from unannotated texts.

Neural Machine Translation Advised by Statistical Machine Translation

no code implementations17 Oct 2016 Xing Wang, Zhengdong Lu, Zhaopeng Tu, Hang Li, Deyi Xiong, Min Zhang

Neural Machine Translation (NMT) is a new approach to machine translation that has made great progress in recent years.

Machine Translation Translation

Lattice-Based Recurrent Neural Network Encoders for Neural Machine Translation

no code implementations25 Sep 2016 Jinsong Su, Zhixing Tan, Deyi Xiong, Rongrong Ji, Xiaodong Shi, Yang Liu

Neural machine translation (NMT) heavily relies on word-level modelling to learn semantic representations of input sentences.

Machine Translation Translation

Cseq2seq: Cyclic Sequence-to-Sequence Learning

no code implementations29 Jul 2016 Biao Zhang, Deyi Xiong, Jinsong Su

The vanilla sequence-to-sequence learning (seq2seq) reads and encodes a source sequence into a fixed-length vector only once, suffering from its insufficiency in modeling structural correspondence between the source and target sequence.

Machine Translation Translation +1

Variational Neural Machine Translation

1 code implementation EMNLP 2016 Biao Zhang, Deyi Xiong, Jinsong Su, Hong Duan, Min Zhang

Models of neural machine translation are often from a discriminative family of encoderdecoders that learn a conditional distribution of a target sentence given a source sentence.

Machine Translation Translation

BattRAE: Bidimensional Attention-Based Recursive Autoencoders for Learning Bilingual Phrase Embeddings

1 code implementation25 May 2016 Biao Zhang, Deyi Xiong, Jinsong Su

In this paper, we propose a bidimensional attention based recursive autoencoder (BattRAE) to integrate clues and sourcetarget interactions at multiple levels of granularity into bilingual phrase representations.

Semantic Similarity Semantic Textual Similarity

Variational Neural Discourse Relation Recognizer

1 code implementation EMNLP 2016 Biao Zhang, Deyi Xiong, Jinsong Su, Qun Liu, Rongrong Ji, Hong Duan, Min Zhang

In order to perform efficient inference and learning, we introduce neural discourse relation models to approximate the prior and posterior distributions of the latent variable, and employ these approximated distributions to optimize a reparameterized variational lower bound.

Neural Discourse Relation Recognition with Semantic Memory

no code implementations12 Mar 2016 Biao Zhang, Deyi Xiong, Jinsong Su

Inspired by this, we propose a neural recognizer for implicit discourse relation analysis, which builds upon a semantic memory that stores knowledge in a distributed fashion.

Word Embeddings

Cannot find the paper you are looking for? You can Submit a new open access paper.