Search Results for author:
Found 215 papers, 75 papers with code
Syntax in End-to-End Natural Language Processing
This tutorial surveys the latest technical progress of syntactic parsing and the role of syntax in end-to-end natural language processing (NLP) tasks, in which semantic role labeling (SRL) and machine translation (MT) are the representative NLP tasks that have always been beneficial from informative syntactic clues since a long time ago, though the advance from end-to-end deep learning models shows new results.
BiBL: AMR Parsing and Generation with Bidirectional Bayesian Learning
Abstract Meaning Representation (AMR) offers a unified semantic representation for natural language sentences.
Ranked #7 on
AMR Parsing
on LDC2020T02
Nested Named Entity Recognition as Corpus Aware Holistic Structure Parsing
Thus in order to address this mismatch, this work models the full nested NEs in a sentence as a holistic structure, then we propose a holistic structure parsing algorithm to disclose the entire NEs once for all.
Aspect-based Sentiment Analysis as Machine Reading Comprehension
Existing studies typically handle aspect-based sentiment analysis by stacking multiple neural modules, which inevitably result in severe error propagation.
Aspect-Based Sentiment Analysis (ABSA)
Machine Reading Comprehension
ArT: All-round Thinker for Unsupervised Commonsense Question Answering
ArT is totally unsupervised and KBs-free.
Modeling Hierarchical Reasoning Chains by Linking Discourse Units and Key Phrases for Reading Comprehension
Machine reading comprehension (MRC) poses new challenges to logical reasoning, which aims to understand the implicit logical relations entailed in the given contexts and perform inference over them.
Restricted or Not: A General Training Framework for Neural Machine Translation
Although this can satisfy the requirements overall, it usually requires a larger beam size and far longer decoding time than unrestricted translation, which limits the concurrent processing ability of the translation model in deployment, and thus its practicality.
What Works and Doesn’t Work, A Deep Decoder for Neural Machine Translation
Inspired by this discovery, we then propose approaches to improving it, with respect to model structure and model training, to make the deep decoder practical in NMT.
NICT’s Neural Machine Translation Systems for the WAT21 Restricted Translation Task
This paper describes our system (Team ID: nictrb) for participating in the WAT’21 restricted machine translation task.
What If Sentence-hood is Hard to Define: A Case Study in Chinese Reading Comprehension
Machine reading comprehension (MRC) is a challenging NLP task for it requires to carefully deal with all linguistic granularities from word, sentence to passage.
MiSS@WMT21: Contrastive Learning-reinforced Domain Adaptation in Neural Machine Translation
In this paper, we describe our MiSS system that participated in the WMT21 news translation task.
Unsupervised Neural Machine Translation with Universal Grammar
Machine translation usually relies on parallel corpora to provide parallel signals for training.
MiSS: An Assistant for Multi-Style Simultaneous Translation
With this system, we aim to provide a complete translation experience for machine translation users.
SJTU-NICT’s Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task
In this paper, we introduced our joint team SJTU-NICT ‘s participation in the WMT 2020 machine translation shared task.
Multimodal Chain-of-Thought Reasoning in Language Models
Large language models (LLMs) have shown impressive performance on complex reasoning by leveraging chain-of-thought (CoT) prompting to generate intermediate reasoning chains as the rationale to infer the answer.
Ranked #1 on
Science Question Answering
on ScienceQA
Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension
We decouple the contextualized word representations by masking mechanisms in Transformer-based PrLM, making each word only focus on the words in current utterance, other utterances, and two speaker roles (i. e., utterances of sender and utterances of the receiver), respectively.
Universal Multimodal Representation for Language Understanding
Representation learning is the foundation of natural language processing (NLP).
Self-Prompting Large Language Models for Open-Domain QA
Open-Domain Question Answering (ODQA) requires models to answer factoid questions with no context given.
Language Model Pre-training on True Negatives
Discriminative pre-trained language models (PLMs) learn to predict original texts from intentionally corrupted ones.
Forging Multiple Training Objectives for Pre-trained Language Models via Meta-Learning
Multiple pre-training objectives fill the vacancy of the understanding capability of single-objective language modeling, which serves the ultimate purpose of pre-trained language models (PrLMs), generalizing well on a mass of scenarios.
Sentence Representation Learning with Generative Objective rather than Contrastive Objective
Though offering amazing contextualized token-level representations, current pre-trained language models take less attention on accurately acquiring sentence-level representation during their self-supervised pre-training.
Towards End-to-End Open Conversational Machine Reading
In open-retrieval conversational machine reading (OR-CMR) task, machines are required to do multi-turn question answering given dialogue history and a textual knowledge base.
Task Compass: Scaling Multi-task Pre-training with Task Prefix
Leveraging task-aware annotated data as supervised signals to assist with self-supervised learning on large-scale unlabeled data has become a new trend in pre-training language models.
Instance Regularization for Discriminative Language Model Pre-training
They treat training instances equally throughout the training process, with little attention on the individual contribution of those instances.
Semantic-Preserving Adversarial Code Comprehension
Based on the tremendous success of pre-trained language models (PrLMs) for source code comprehension tasks, current literature studies either ways to further improve the performance (generalization) of PrLMs, or their robustness against adversarial attacks.
Learning Better Masking for Better Language Model Pre-training
Masked Language Modeling (MLM) has been widely used as the denoising objective in pre-training language models (PrLMs).
Evaluate Confidence Instead of Perplexity for Zero-shot Commonsense Reasoning
In detail, it works on PLMs according to the Replaced Token Detection (RTD) pre-training objective in ELECTRA, in which the corruption detection objective reflects the confidence on contextual integrity that is more relevant to commonsense reasoning than existing probability.
Rethinking Textual Adversarial Defense for Pre-trained Language Models
However, we find that most existing textual adversarial examples are unnatural, which can be easily distinguished by both human and machine.
Adversarial Self-Attention for Language Understanding
Deep neural models (e. g. Transformer) naturally learn spurious features, which create a ``shortcut'' between the labels and inputs, thus impairing the generalization and robustness.
Ranked #1 on
Machine Reading Comprehension
on DREAM
Machine Reading Comprehension
Named Entity Recognition (NER)
+4
Solution of DeBERTaV3 on CommonsenseQA
We report the performance of DeBERTaV3 on CommonsenseQA in this report.
Generative or Contrastive? Phrase Reconstruction for Better Sentence Representation Learning
If self-supervised learning can be distinguished into two subcategories, generative and contrastive, then most existing studies show that sentence representation learning may more benefit from the contrastive methods but not the generative methods.
Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling
Multi-turn dialogue modeling as a challenging branch of natural language understanding (NLU), aims to build representations for machines to understand human dialogues, which provides a solid foundation for multiple downstream tasks.
Nested Named Entity Recognition as Holistic Structure Parsing
Thus in order to address this mismatch, this work models the full nested NEs in a sentence as a holistic structure, then we propose a holistic structure parsing algorithm to disclose the entire NEs once for all.
Lite Unified Modeling for Discriminative Reading Comprehension
As a broad and major category in machine reading comprehension (MRC), the generalized goal of discriminative MRC is answer prediction from the given materials.
Distinguishing Non-natural from Natural Adversarial Samples for More Robust Pre-trained Language Model
We question the validity of current evaluation of robustness of PrLMs based on these non-natural adversarial samples and propose an anomaly detector to evaluate the robustness of PrLMs with more natural adversarial samples.
Semantics-Preserved Distortion for Personal Privacy Protection in Information Management
As the model training on information from users is likely to invade personal privacy, many methods have been proposed to block the learning and memorizing of the sensitive data in raw texts.
ArT: All-round Thinker for Unsupervised Commonsense Question-Answering
In detail, our model first focuses on key parts in the given context, and then generates highly related knowledge on such a basis in an association way like human thinking.
Multilingual Pre-training with Universal Dependency Learning
The pre-trained language model (PrLM) demonstrates domination in downstream natural language processing tasks, in which multilingual PrLM takes advantage of language universality to alleviate the issue of limited resources for low-resource languages.
Seeking Common but Distinguishing Difference, A Joint Aspect-based Sentiment Analysis Model
Therefore, we propose a joint ABSA model, which not only enjoys the benefits of encoder sharing but also focuses on the difference to improve the effectiveness of the model.
Unsupervised Full Constituency Parsing with Neighboring Distribution Divergence
Unsupervised constituency parsing has been explored much but is still far from being solved.
Tracing Origins: Coreference-aware Machine Reading Comprehension
In this paper, we imitate the human reading process in connecting the anaphoric expressions and explicitly leverage the coreference information of the entities to enhance the word embeddings from the pre-trained language model, in order to highlight the coreference mentions of the entities that must be identified for coreference-intensive question answering in QUOREF, a relatively new dataset that is specifically designed to evaluate the coreference-related performance of a model.
Structural Characterization for Dialogue Disentanglement
Tangled multi-party dialogue contexts lead to challenges for dialogue reading comprehension, where multiple dialogue threads flow simultaneously within a common dialogue record, increasing difficulties in understanding the dialogue history for both human and machine.
Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval
In detail, we introduce an in-passage negative sampling strategy to encourage a diverse generation of sentence representations within the same passage.
Advances in Multi-turn Dialogue Comprehension: A Survey
In this paper, we review the previous methods from the technical perspective of dialogue modeling for the dialogue comprehension task.
Multi-tasking Dialogue Comprehension with Discourse Parsing
Multi-party dialogue machine reading comprehension (MRC) raises an even more challenging understanding goal on dialogue with more than two involved speakers, compared with the traditional plain passage style MRC.
Ranked #1 on
Discourse Parsing
on Molweni
A Novel Metric for Evaluating Semantics Preservation
By exploiting the property of NDD, we implement a unsupervised and even training-free algorithm for extractive sentence compression.
Logic Pre-Training of Language Models
Pre-trained language models (PrLMs) have been shown useful for enhancing a broad range of natural language understanding (NLU) tasks.
Sparse Fuzzy Attention for Structured Sentiment Analysis
Attention scorers have achieved success in parsing tasks like semantic and syntactic dependency parsing.
Enhanced Speaker-aware Multi-party Multi-turn Dialogue Comprehension
Multi-party multi-turn dialogue comprehension brings unprecedented challenges on handling the complicated scenarios from multiple speakers and criss-crossed discourse relationship among speaker-aware utterances.
Ranked #1 on
Question Answering
on Molweni
Self- and Pseudo-self-supervised Prediction of Speaker and Key-utterance for Multi-party Dialogue Reading Comprehension
Multi-party dialogue machine reading comprehension (MRC) brings tremendous challenge since it involves multiple speakers at one dialogue, resulting in intricate speaker information flows and noisy dialogue contexts.
Ranked #2 on
Question Answering
on FriendsQA
Unsupervised Open-Domain Question Answering
Open-domain Question Answering (ODQA) has achieved significant results in terms of supervised learning manner.
Machine Reading Comprehension
Open-Domain Question Answering
Span Fine-tuning for Pre-trained Language Models
Pre-trained language models (PrLM) have to carefully manage input units when training on a very large text with a vocabulary consisting of millions of words.
Smoothing Dialogue States for Open Conversational Machine Reading
In this work, we propose an effective gating strategy by smoothing the two dialogue states in only one decoder and bridge decision making and question generation to provide a richer dialogue state reference.
Cross-lingual Transferring of Pre-trained Contextualized Language Models
Though the pre-trained contextualized language model (PrLM) has made a significant impact on NLP, training PrLMs in languages other than English can be impractical for two reasons: other languages often lack corpora sufficient for training powerful PrLMs, and because of the commonalities among human languages, computationally expensive PrLM training for different languages is somewhat redundant.
Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy
Multi-hop reading comprehension (MHRC) requires not only to predict the correct answer span in the given passage, but also to provide a chain of supporting evidences for reasoning interpretability.
Dialogue-oriented Pre-training
Pre-trained language models (PrLM) has been shown powerful in enhancing a broad range of downstream tasks including various dialogue related ones.
Pre-training Universal Language Representation
Despite the well-developed cut-edge representation learning for language, most language representation models usually focus on specific levels of linguistic units.
Defending Pre-trained Language Models from Adversarial Word Substitutions Without Performance Sacrifice
In detail, we design an auxiliary anomaly detection classifier and adopt a multi-task learning procedure, by which PrLMs are able to distinguish adversarial input samples.
Grammatical Error Correction as GAN-like Sequence Labeling
In Grammatical Error Correction (GEC), sequence labeling models enjoy fast inference compared to sequence-to-sequence models; however, inference in sequence labeling GEC models is an iterative process, as sentences are passed to the model for multiple rounds of correction, which exposes the model to sentences with progressively fewer errors at each round.
Structural Pre-training for Dialogue Comprehension
Pre-trained language models (PrLMs) have demonstrated superior performance due to their strong ability to learn universal language representations from self-supervised pre-training.
Fact-driven Logical Reasoning
Therefore, we argue that the natural logic units would be the group of backbone constituents of the sentence such as the subject-verb-object formed "facts", covering both global and local knowledge pieces that are necessary as the basis for logical reasoning.
Ranked #24 on
Reading Comprehension
on ReClor
Head-driven Phrase Structure Parsing in O($n^3$) Time Complexity
Constituent and dependency parsing, the two classic forms of syntactic parsing, have been found to benefit from joint training and decoding under a uniform formalism, Head-driven Phrase Structure Grammar (HPSG).
Neural Unsupervised Semantic Role Labeling
To decompose the task as two argument related subtasks, identification and clustering, we propose a pipeline that correspondingly consists of two neural modules.
Not All Attention Is All You Need
Beyond the success story of pre-trained language models (PrLMs) in recent natural language processing, they are susceptible to over-fitting due to unusual large model size.
Advances and Challenges in Unsupervised Neural Machine Translation
Unsupervised cross-lingual language representation initialization methods, together with mechanisms such as denoising and back-translation, have advanced unsupervised neural machine translation (UNMT), which has achieved impressive results.
Advances in Multi-turn Dialogue Comprehension: A Survey
In this paper, we review the previous methods from the technical perspective of dialogue modeling for the dialogue comprehension task.
Text Compression-aided Transformer Encoding
In this paper, we propose explicit and implicit text compression approaches to enhance the Transformer encoding and evaluate models using this approach on several typical downstream tasks that rely on the encoding heavily.
Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge
Experimental results on four dialogue comprehension benchmark tasks show that our proposed model achieves great improvements on baselines.
To Understand Representation of Layer-aware Sequence Encoders as Multi-order-graph
We also propose a revisited multigraph called Multi-order-Graph (MoG) based on our explanation to model the graph structures in the SAN-based model as subgraphs in MoG and convert the encoding of SAN-based model to the generation of MoG.
Cross-lingual Transfer Learning for Pre-trained Contextualized Language Models
Though the pre-trained contextualized language model (PrLM) has made a significant impact on NLP, training PrLMs in languages other than English can be impractical for two reasons: other languages often lack corpora sufficient for training powerful PrLMs, and because of the commonalities among human languages, computationally expensive PrLM training for different languages is somewhat redundant.
Switching-Aligned-Words Data Augmentation for Neural Machine Translation
In neural machine translation (NMT), data augmentation methods such as back-translation make it possible to use extra monolingual data to help improve translation performance, while it needs extra training data and the in-domain monolingual data is not always available.
Later Span Adaptation for Language Understanding
Instead of too early fixing the linguistic unit input as nearly all previous work did, we propose a novel method that combines span-level information into the representations generated by PrLMs during fine-tuning phase for better flexibility.
Efficient Neural Machine Translation with Prior Word Alignment
In this paper, we propose a novel method that infuses prior word alignment information into neural machine translation (NMT) to provide hints or guidelines for the target sentence at running time.
Enhancing Pre-trained Language Model with Lexical Simplification
By substituting complex words with simple alternatives, lexical simplification (LS) is a recognized method to reduce such lexical diversity, and therefore to improve the understandability of sentences.
Accurate Word Representations with Universal Visual Guidance
Word representation is a fundamental component in neural language understanding models.
Code Summarization with Structure-induced Transformer
Code summarization (CS) is becoming a promising area in recent language understanding, which aims to generate sensible human language automatically for programming language in the format of source code, serving in the most convenience of programmer developing.
Dialogue Graph Modeling for Conversational Machine Reading
Conversational Machine Reading (CMR) aims at answering questions in a complicated manner.
BURT: BERT-inspired Universal Representation from Learning Meaningful Segment
We present a universal representation model, BURT (BERT-inspired Universal Representation from learning meaningful segmenT), to encode different levels of linguistic unit into the same vector space.
SG-Net: Syntax Guided Transformer for Language Representation
In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention.
Adaptive Convolution for Semantic Role Labeling
Semantic role labeling (SRL) aims at elaborating the meaning of a sentence by forming a predicate-argument structure.
Cross-lingual Universal Dependency Parsing Only from One Monolingual Treebank
As it is unlikely to obtain a treebank for every human language, in this work, we propose an effective cross-lingual UD parsing framework for transferring parser from only one source monolingual treebank to any other target languages without treebank available.
Reference Knowledgeable Network for Machine Reading Comprehension
Thus we propose a novel reference-based knowledge enhancement model called Reference Knowledgeable Network (RekNet), which simulates human reading strategies to refine critical information from the passage and quote explicit knowledge in necessity.
LIMIT-BERT : Linguistics Informed Multi-Task BERT
Besides, LIMIT-BERT takes a semi-supervised learning strategy to offer the same large amount of linguistics task data as that for the language model training.
SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task
In this paper, we introduced our joint team SJTU-NICT 's participation in the WMT 2020 machine translation shared task.
High-order Semantic Role Labeling
Semantic role labeling is primarily used to identify predicates, arguments, and their semantic relationships.
Topic-Aware Multi-turn Dialogue Modeling
In the retrieval-based multi-turn dialogue modeling, it remains a challenge to select the most appropriate response according to extracting salient features in context utterances.
Document-level Neural Machine Translation with Document Embeddings
Standard neural machine translation (NMT) is on the assumption of document-level context independent.
Graph-to-Sequence Neural Machine Translation
In the light of the current NMT models more or less capture graph information among the sequence in a latent way, we present a graph-to-sequence model facilitating explicit graph information capturing.
Multi-span Style Extraction for Generative Reading Comprehension
Generative machine reading comprehension (MRC) requires a model to generate well-formed answers.
Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue
A multi-turn dialogue is composed of multiple utterances from two or more different speaker roles.
Composing Answer from Multi-spans for Reading Comprehension
This paper presents a novel method to generate answers for non-extraction machine reading comprehension (MRC) tasks whose answers cannot be simply extracted as one span from the given passages.
Syntax Role for Neural Semantic Role Labeling
Semantic role labeling (SRL) is dedicated to recognizing the semantic predicate-argument structure of a sentence.
Learning Universal Representations from Word to Sentence
Despite the well-developed cut-edge representation learning for language, most language representation models usually focus on specific level of linguistic unit, which cause great inconvenience when being confronted with handling multiple layers of linguistic objects in a unified way.
Dialogue-adaptive Language Model Pre-training From Quality Estimation
Pre-trained language models (PrLMs) have achieved great success on a wide range of natural language processing tasks by virtue of the universal language representation ability obtained by self-supervised learning on a large corpus.
Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
In this survey, we provide a comprehensive and comparative review on MRC covering overall research topics about 1) the origin and development of MRC and CLM, with a particular focus on the role of CLMs; 2) the impact of MRC and CLM to the NLP community; 3) the definition, datasets, and evaluation of MRC; 4) general MRC architecture and technical methods in the view of two-stage Encoder-Decoder solving architecture from the insights of the cognitive process of humans; 5) previous highlights, emerging topics, and our empirical analysis, among which we especially focus on what works in different periods of MRC researches.
Bipartite Flat-Graph Network for Nested Named Entity Recognition
In this paper, we propose a novel bipartite flat-graph network (BiFlaG) for nested named entity recognition (NER), which contains two subgraph modules: a flat NER module for outermost entities and a graph module for all the entities located in inner layers.
Ranked #6 on
Nested Mention Recognition
on ACE 2005
Neural Machine Translation with Universal Visual Representation
Though visual information has been introduced for enhancing neural machine translation (NMT), its effectiveness strongly relies on the availability of large amounts of bilingual parallel sentence pairs with manual image annotations.
Data-dependent Gaussian Prior Objective for Language Generation
However, MLE focuses on once-to-all matching between the predicted sequence and gold-standard, consequently treating all incorrect predictions as being equally incorrect.
Capsule-Transformer for Neural Machine Translation
In this paper, we thus propose the capsule-Transformer, which extends the linear transformation into a more general capsule routing algorithm by taking SAN as a special case of capsule network.
Syntax-aware Data Augmentation for Neural Machine Translation
Data augmentation is an effective performance enhancement in neural machine translation (NMT) by generating additional bilingual data.
Knowledgeable Dialogue Reading Comprehension on Key Turns
In this paper, the relevance of each turn to the question are calculated to choose key turns.
BURT: BERT-inspired Universal Representation from Twin Structure
Pre-trained contextualized language models such as BERT have shown great effectiveness in a wide range of downstream Natural Language Processing (NLP) tasks.
Semantics-Aware Inferential Network for Natural Language Understanding
Taking explicit contextualized semantics as a complementary input, the inferential module of SAIN enables a series of reasoning steps over semantic clues through an attention mechanism.
Cross-lingual Supervision Improves Unsupervised Neural Machine Translation
Neural machine translation~(NMT) is ineffective for zero-resource languages.
Reference Language based Unsupervised Neural Machine Translation
Further enriching the idea of pivot translation by extending the use of parallel corpora beyond the source-target paradigm, we propose a new reference language-based framework for UNMT, RUNMT, in which the reference language only shares a parallel corpus with the source, but this corpus still indicates a signal clear enough to help the reconstruction training of UNMT through a proposed reference agreement mechanism.
Retrospective Reader for Machine Reading Comprehension
Inspired by how humans solve reading comprehension questions, we proposed a retrospective reader (Retro-Reader) that integrates two stages of reading and verification strategies: 1) sketchy reading that briefly investigates the overall interactions of passage and question, and yield an initial judgment; 2) intensive reading that verifies the answer and gives the final prediction.
Ranked #7 on
Question Answering
on SQuAD2.0
DUMA: Reading Comprehension with Transposition Thinking
Multi-choice Machine Reading Comprehension (MRC) requires model to decide the correct answer from a set of answer options when given a passage and a question.
Ranked #3 on
Reading Comprehension
on RACE
Dual Multi-head Co-attention for Multi-choice Reading Comprehension
Multi-choice Machine Reading Comprehension (MRC) requires model to decide the correct answer from a set of answer options when given a passage and a question.
Explicit Sentence Compression for Neural Machine Translation
In this paper, we propose an explicit sentence compression method to enhance the source sentence representation for NMT.
Korean-to-Chinese Machine Translation using Chinese Character as Pivot Clue
Korean-Chinese is a low resource language pair, but Korean and Chinese have a lot in common in terms of vocabulary.
Global Greedy Dependency Parsing
Most syntactic dependency parsing models may fall into one of two categories: transition- and graph-based models.
Dependency and Span, Cross-Style Semantic Role Labeling on PropBank and NomBank
In this paper, we define a new cross-style semantic role label convention and propose a new cross-style joint optimization model designed around the most basic linguistic meaning of a semantic role, providing a solution to make the results of the two styles more comparable and allowing both formalisms of SRL to benefit from their natural connections in both linguistics and computation.
Probing Contextualized Sentence Representations with Visual Awareness
We present a universal framework to model contextualized sentence representations with visual awareness that is motivated to overcome the shortcomings of the multimodal parallel data with manual annotations.
Hierarchical Contextualized Representation for Named Entity Recognition
In this paper, we address these two deficiencies and propose a model augmented with hierarchical contextualized representation: sentence-level representation and document-level representation.
Ranked #12 on
Named Entity Recognition (NER)
on Ontonotes v5 (English)
(using extra training data)
Deepening Hidden Representations from Pre-trained Language Models
Transformer-based pre-trained language models have proven to be effective for learning contextualized language representation.
SJTU-NICT at MRP 2019: Multi-Task Learning for End-to-End Uniform Semantic Graph Parsing
This paper describes our SJTU-NICT{'}s system for participating in the shared task on Cross-Framework Meaning Representation Parsing (MRP) at the 2019 Conference for Computational Language Learning (CoNLL).
SJTU at MRP 2019: A Transition-Based Multi-Task Parser for Cross-Framework Meaning Representation Parsing
This paper describes the system of our team SJTU for our participation in the CoNLL 2019 Shared Task: Cross-Framework Meaning Representation Parsing.
Attention Is All You Need for Chinese Word Segmentation
Taking greedy decoding algorithm as it should be, this work focuses on further strengthening the model itself for Chinese word segmentation (CWS), which results in an even more fast and more accurate CWS model.
Document-level Neural Machine Translation with Associated Memory Network
Most existing document-level NMT approaches are satisfied with a smattering sense of global document-level information, while this work focuses on exploiting detailed document-level context in terms of a memory network.
LIMIT-BERT : Linguistic Informed Multi-Task BERT
In this paper, we present a Linguistic Informed Multi-Task BERT (LIMIT-BERT) for learning language representations across multiple linguistic tasks by Multi-Task Learning (MTL).
Subword ELMo
Embedding from Language Models (ELMo) has shown to be effective for improving many natural language processing (NLP) tasks, and ELMo takes character information to compose word representation to train language models. However, the character is an insufficient and unnatural linguistic unit for word representation. Thus we introduce Embedding from Subword-aware Language Models (ESuLMo) which learns word representation from subwords using unsupervised segmentation over words. We show that ESuLMo can enhance four benchmark NLP tasks more effectively than ELMo, including syntactic dependency parsing, semantic role labeling, implicit discourse relation recognition and textual entailment, which brings a meaningful improvement over ELMo.
Semantics-aware BERT for Language Understanding
The latest work on language representations carefully integrates contextualized features into language model training, which enables a series of success especially in various machine reading comprehension and natural language inference tasks.
Ranked #6 on
Natural Language Inference
on SNLI
Modeling Named Entity Embedding Distribution into Hypersphere
This work models named entity distribution from a way of visualizing topological structure of embedding space, so that we make an assumption that most, if not all, named entities (NEs) for a language tend to aggregate together to be accommodated by a specific hypersphere in embedding space.
A Smart Sliding Chinese Pinyin Input Method Editor on Touchscreen
This paper presents a smart sliding Chinese pinyin Input Method Editor (IME) for touchscreen devices which allows user finger sliding from one key to another on the touchscreen instead of tapping keys one by one, while the target Chinese character sequence will be predicted during the sliding process to help user input Chinese characters efficiently.
Syntax-aware Multilingual Semantic Role Labeling
Recently, semantic role labeling (SRL) has earned a series of success with even higher performance improvements, which can be mainly attributed to syntactic integration and enhanced word representation.
Named Entity Recognition Only from Word Embeddings
Deep neural network models have helped named entity (NE) recognition achieve amazing performance without handcrafting features.
Open Named Entity Modeling from Embedding Distribution
For monolingual cases, the proposed named entity model gives an open description of diverse named entity types and different languages.
DCMN+: Dual Co-Matching Network for Multi-choice Reading Comprehension
Multi-choice reading comprehension is a challenging task to select an answer from a set of candidate options when given passage and question.
Parsing All: Syntax and Semantics, Dependencies and Spans
Both syntactic and semantic structures are key linguistic contextual clues, in which parsing the latter has been well shown beneficial from parsing the former.
Memorizing All for Implicit Discourse Relation Recognition
As implicit discourse relation recognizer has to carefully tackle the semantic similarity of the given sentence pairs and the severe data sparsity issue exists in the meantime, it is supposed to be beneficial from mastering the entire training data.
Controllable Dual Skew Divergence Loss for Neural Machine Translation
Our experiments indicate that switching to the DSD loss after the convergence of ML training helps models escape local optima and stimulates stable performance improvements.
Concurrent Parsing of Constituency and Dependency
Constituent and dependency representation for syntactic structure share a lot of linguistic and computational characteristics, this paper thus makes the first attempt by introducing a new model that is capable of parsing constituent and dependency at the same time, so that lets either of the parsers enhance each other.
SG-Net: Syntax-Guided Machine Reading Comprehension
In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention.
Ranked #5 on
Question Answering
on SQuAD2.0 dev
Semantic Role Labeling with Associated Memory Network
Semantic role labeling (SRL) is a task to recognize all the predicate-argument pairs of a sentence, which has been in a performance improvement bottleneck after a series of latest works were presented.
Head-Driven Phrase Structure Grammar Parsing on Penn Treebank
In details, we report 96. 33 F1 of constituent parsing and 97. 20\% UAS of dependency parsing on PTB.
Ranked #4 on
Constituency Parsing
on Penn Treebank
Lattice-Based Transformer Encoder for Neural Machine Translation
To integrate different segmentations with the state-of-the-art NMT model, Transformer, we propose lattice-based encoders to explore effective word or subword representation in an automatic way during training.
GAN Driven Semi-distant Supervision for Relation Extraction
Distant supervision has been widely used in relation extraction tasks without hand-labeled datasets recently.
Minimum Divergence vs. Maximum Margin: an Empirical Comparison on Seq2Seq Models
Sequence to sequence (seq2seq) models have become a popular framework for neural sequence prediction.
Judging Chemical Reaction Practicality From Positive Sample only Learning
Chemical reaction practicality is the core task among all symbol intelligence based chemical information processing, for example, it provides indispensable clue for further automatic synthesis route inference.
Span Model for Open Information Extraction on Accurate Corpus
Open information extraction (Open IE) is a challenging task especially due to its brittle data basis.
Ranked #5 on
Open Information Extraction
on OIE2016
Dual Co-Matching Network for Multi-choice Reading Comprehension
Multi-choice reading comprehension is a challenging task that requires complex reasoning procedure.
Ranked #3 on
Question Answering
on RACE
Chemical Names Standardization using Neural Sequence to Sequence Model
Chemical information extraction is to convert chemical knowledge in text into true chemical database, which is a text processing task heavily relying on chemical compound name identification and standardization.
Chinese Word Segmentation: Another Decade Review (2007-2017)
This paper reviews the development of Chinese word segmentation (CWS) in the most recent decade, 2007-2017.
Dependency or Span, End-to-End Uniform Semantic Role Labeling
Semantic role labeling (SRL) aims to discover the predicateargument structure of a sentence.
Ranked #9 on
Semantic Role Labeling
on CoNLL 2005
Open Vocabulary Learning for Neural Chinese Pinyin IME
Pinyin-to-character (P2C) conversion is the core component of pinyin-based Chinese input method engine (IME).
Multi-labeled Relation Extraction with Attentive Capsule Network
To disclose overlapped multiple relations from a sentence still keeps challenging.
Effective Representation for Easy-First Dependency Parsing
Easy-first parsing relies on subtree re-ranking to build the complete parse tree.
Effective Subword Segmentation for Text Comprehension
Representation learning is the foundation of machine reading comprehension and inference.
Fast Neural Chinese Word Segmentation for Long Sentences
Rapidly developed neural models have achieved competitive performance in Chinese word segmentation (CWS) as their traditional counterparts.
Joint Learning of POS and Dependencies for Multilingual Universal Dependency Parsing
This paper describes the system of team LeisureX in the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies.
Multilingual Universal Dependency Parsing from Raw Text with Low-Resource Language Enhancement
This paper describes the system of our team Phoenix for participating CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies.
A Unified Syntax-aware Framework for Semantic Role Labeling
Semantic role labeling (SRL) aims to recognize the predicate-argument structure of a sentence.
Attentive Semantic Role Labeling with Boundary Indicator
The goal of semantic role labeling (SRL) is to discover the predicate-argument structure of a sentence, which plays a critical role in deep processing of natural language.
Explicit Contextual Semantics for Text Comprehension
Who did what to whom is a major focus in natural language understanding, which is right the aim of semantic role labeling (SRL) task.
Ranked #9 on
Natural Language Inference
on SNLI
Machine Reading Comprehension
Natural Language Understanding
+1
Chinese Pinyin Aided IME, Input What You Have Not Keystroked Yet
Chinese pinyin input method engine (IME) converts pinyin into character so that Chinese characters can be conveniently inputted into computer through common keyboard.
Exploring Recombination for Efficient Decoding of Neural Machine Translation
In Neural Machine Translation (NMT), the decoder can capture the features of the entire prediction history with neural connections and representations.
A Full End-to-End Semantic Role Labeler, Syntax-agnostic Over Syntax-aware?
Semantic role labeling (SRL) is to recognize the predicate-argument structure of a sentence, including subtasks of predicate disambiguation and argument labeling.
Lingke: A Fine-grained Multi-turn Chatbot for Customer Service
Traditional chatbots usually need a mass of human dialogue data, especially when using supervised machine learning method.
Effective Character-augmented Word Embedding for Machine Reading Comprehension
Machine reading comprehension is a task to model relationship between passage and query.
Seq2seq Dependency Parsing
This paper presents a sequence to sequence (seq2seq) dependency parser by directly predicting the relative position of head for each given word, which therefore results in a truly end-to-end seq2seq dependency parser for the first time.
A Full End-to-End Semantic Role Labeler, Syntactic-agnostic Over Syntactic-aware?
Semantic role labeling (SRL) is to recognize the predicate-argument structure of a sentence, including subtasks of predicate disambiguation and argument labeling.
Finding Better Subword Segmentation for Neural Machine Translation
For different language pairs, word-level neural machine translation (NMT) models with a fixed-size vocabulary suffer from the same problem of representing out-of-vocabulary (OOV) words.
Deep Enhanced Representation for Implicit Discourse Relation Recognition
Implicit discourse relation recognition is a challenging task as the relation prediction without explicit connectives in discourse parsing needs understanding of text spans and cannot be easily derived from surface features from the input sentence pairs.
Syntax for Semantic Role Labeling, To Be, Or Not To Be
Semantic role labeling (SRL) is dedicated to recognizing the predicate-argument structure of a sentence.
Moon IME: Neural-based Chinese Pinyin Aided Input Method with Customizable Association
Chinese pinyin input method engine (IME) lets user conveniently input Chinese into a computer by typing pinyin through the common keyboard.
One-shot Learning for Question-Answering in Gaokao History Challenge
Answering questions from university admission exams (Gaokao in Chinese) is a challenging AI task since it requires effective representation to capture complicated semantic relations between questions and answers.
Subword-augmented Embedding for Cloze Reading Comprehension
Representation learning is the foundation of machine reading comprehension.
Modeling Multi-turn Conversation with Deep Utterance Aggregation
In this paper, we formulate previous utterances into context using a proposed deep utterance aggregation model to form a fine-grained context representation.
Ranked #11 on
Conversational Response Selection
on E-commerce
SJTU-NLP at SemEval-2018 Task 9: Neural Hypernym Discovery with Term Embeddings
This paper describes a hypernym discovery system for our participation in the SemEval-2018 Task 9, which aims to discover the best (set of) candidate hypernyms for input concepts or entities, given the search space of a pre-defined vocabulary.
Ranked #5 on
Hypernym Discovery
on Music domain
Automatic Article Commenting: the Task and Dataset
Comments of online articles provide extended views and improve user engagement.