Search Results for author:
Found 81 papers, 34 papers with code
Modeling Hierarchical Reasoning Chains by Linking Discourse Units and Key Phrases for Reading Comprehension
Machine reading comprehension (MRC) poses new challenges to logical reasoning, which aims to understand the implicit logical relations entailed in the given contexts and perform inference over them.
Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
Spurred by advancements in scale, large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot -- i. e., without adaptation on downstream data.
Multimodal Chain-of-Thought Reasoning in Language Models
Large language models (LLMs) have shown impressive performance on complex reasoning by leveraging chain-of-thought (CoT) prompting to generate intermediate reasoning chains as the rationale to infer the answer.
Ranked #1 on
Science Question Answering
on ScienceQA
Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension
We decouple the contextualized word representations by masking mechanisms in Transformer-based PrLM, making each word only focus on the words in current utterance, other utterances, and two speaker roles (i. e., utterances of sender and utterances of the receiver), respectively.
Universal Multimodal Representation for Language Understanding
Representation learning is the foundation of natural language processing (NLP).
Self-Prompting Large Language Models for Open-Domain QA
Open-Domain Question Answering (ODQA) requires models to answer factoid questions with no context given.
Language Model Pre-training on True Negatives
Discriminative pre-trained language models (PLMs) learn to predict original texts from intentionally corrupted ones.
Retrieval Augmentation for Commonsense Reasoning: A Unified Approach
However, applying such methods to commonsense reasoning tasks faces two unique challenges, i. e., the lack of a general large-scale corpus for retrieval and a corresponding effective commonsense retriever.
Towards End-to-End Open Conversational Machine Reading
In open-retrieval conversational machine reading (OR-CMR) task, machines are required to do multi-turn question answering given dialogue history and a textual knowledge base.
Task Compass: Scaling Multi-task Pre-training with Task Prefix
Leveraging task-aware annotated data as supervised signals to assist with self-supervised learning on large-scale unlabeled data has become a new trend in pre-training language models.
Instance Regularization for Discriminative Language Model Pre-training
They treat training instances equally throughout the training process, with little attention on the individual contribution of those instances.
Automatic Chain of Thought Prompting in Large Language Models
Providing these steps for prompting demonstrations is called chain-of-thought (CoT) prompting.
Learning Better Masking for Better Language Model Pre-training
Masked Language Modeling (MLM) has been widely used as the denoising objective in pre-training language models (PrLMs).
Rethinking Textual Adversarial Defense for Pre-trained Language Models
However, we find that most existing textual adversarial examples are unnatural, which can be easily distinguished by both human and machine.
Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling
Multi-turn dialogue modeling as a challenging branch of natural language understanding (NLU), aims to build representations for machines to understand human dialogues, which provides a solid foundation for multiple downstream tasks.
Distinguishing Non-natural from Natural Adversarial Samples for More Robust Pre-trained Language Model
We question the validity of current evaluation of robustness of PrLMs based on these non-natural adversarial samples and propose an anomaly detector to evaluate the robustness of PrLMs with more natural adversarial samples.
IF equation: a feature extractor for high-concentration time-frequency representation of mixed signals
By the analysis of the properties of the IF equation, we prove that a good IF equation can unify the well-known IF and group delay estimators and provides an effective way to characterize the mixture of time-varying and frequency-varying signals.
Tracing Origins: Coreference-aware Machine Reading Comprehension
In this paper, we imitate the human reading process in connecting the anaphoric expressions and explicitly leverage the coreference information of the entities to enhance the word embeddings from the pre-trained language model, in order to highlight the coreference mentions of the entities that must be identified for coreference-intensive question answering in QUOREF, a relatively new dataset that is specifically designed to evaluate the coreference-related performance of a model.
Structural Characterization for Dialogue Disentanglement
Tangled multi-party dialogue contexts lead to challenges for dialogue reading comprehension, where multiple dialogue threads flow simultaneously within a common dialogue record, increasing difficulties in understanding the dialogue history for both human and machine.
Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval
In detail, we introduce an in-passage negative sampling strategy to encourage a diverse generation of sentence representations within the same passage.
Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese
Although pre-trained models (PLMs) have achieved remarkable improvements in a wide range of NLP tasks, they are expensive in terms of time and resources.
Advances in Multi-turn Dialogue Comprehension: A Survey
In this paper, we review the previous methods from the technical perspective of dialogue modeling for the dialogue comprehension task.
Multi-tasking Dialogue Comprehension with Discourse Parsing
Multi-party dialogue machine reading comprehension (MRC) raises an even more challenging understanding goal on dialogue with more than two involved speakers, compared with the traditional plain passage style MRC.
Ranked #1 on
Discourse Parsing
on Molweni
Logic Pre-Training of Language Models
Pre-trained language models (PrLMs) have been shown useful for enhancing a broad range of natural language understanding (NLU) tasks.
Enhanced Speaker-aware Multi-party Multi-turn Dialogue Comprehension
Multi-party multi-turn dialogue comprehension brings unprecedented challenges on handling the complicated scenarios from multiple speakers and criss-crossed discourse relationship among speaker-aware utterances.
Ranked #1 on
Question Answering
on Molweni
Span Fine-tuning for Pre-trained Language Models
Pre-trained language models (PrLM) have to carefully manage input units when training on a very large text with a vocabulary consisting of millions of words.
Smoothing Dialogue States for Open Conversational Machine Reading
In this work, we propose an effective gating strategy by smoothing the two dialogue states in only one decoder and bridge decision making and question generation to provide a richer dialogue state reference.
High-resolution chirplet transform: from parameters analysis to parameters combination
The standard chirplet transform (CT) with a chirp-modulated Gaussian window provides a valuable tool for analyzing linear chirp signals.
Cross-lingual Transferring of Pre-trained Contextualized Language Models
Though the pre-trained contextualized language model (PrLM) has made a significant impact on NLP, training PrLMs in languages other than English can be impractical for two reasons: other languages often lack corpora sufficient for training powerful PrLMs, and because of the commonalities among human languages, computationally expensive PrLM training for different languages is somewhat redundant.
Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy
Multi-hop reading comprehension (MHRC) requires not only to predict the correct answer span in the given passage, but also to provide a chain of supporting evidences for reasoning interpretability.
BO-DBA: Query-Efficient Decision-Based Adversarial Attacks via Bayesian Optimization
However, extending BO to the setting of DBA is nontrivial because in DBA only output labels instead of real-valued scores, as needed by BO, are available to attackers.
Structural Pre-training for Dialogue Comprehension
Pre-trained language models (PrLMs) have demonstrated superior performance due to their strong ability to learn universal language representations from self-supervised pre-training.
Fact-driven Logical Reasoning
Therefore, we argue that the natural logic units would be the group of backbone constituents of the sentence such as the subject-verb-object formed "facts", covering both global and local knowledge pieces that are necessary as the basis for logical reasoning.
Ranked #24 on
Reading Comprehension
on ReClor
Advances in Multi-turn Dialogue Comprehension: A Survey
In this paper, we review the previous methods from the technical perspective of dialogue modeling for the dialogue comprehension task.
Text Compression-aided Transformer Encoding
In this paper, we propose explicit and implicit text compression approaches to enhance the Transformer encoding and evaluate models using this approach on several typical downstream tasks that rely on the encoding heavily.
Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge
Experimental results on four dialogue comprehension benchmark tasks show that our proposed model achieves great improvements on baselines.
SAFELearning: Enable Backdoor Detectability In Federated Learning With Secure Aggregation
For model privacy, local model parameters in federated learning shall be obfuscated before sent to the remote aggregator.
Later Span Adaptation for Language Understanding
Instead of too early fixing the linguistic unit input as nearly all previous work did, we propose a novel method that combines span-level information into the representations generated by PrLMs during fine-tuning phase for better flexibility.
Cross-lingual Transfer Learning for Pre-trained Contextualized Language Models
Though the pre-trained contextualized language model (PrLM) has made a significant impact on NLP, training PrLMs in languages other than English can be impractical for two reasons: other languages often lack corpora sufficient for training powerful PrLMs, and because of the commonalities among human languages, computationally expensive PrLM training for different languages is somewhat redundant.
Enhancing Pre-trained Language Model with Lexical Simplification
By substituting complex words with simple alternatives, lexical simplification (LS) is a recognized method to reduce such lexical diversity, and therefore to improve the understandability of sentences.
Accurate Word Representations with Universal Visual Guidance
Word representation is a fundamental component in neural language understanding models.
Dialogue Graph Modeling for Conversational Machine Reading
Conversational Machine Reading (CMR) aims at answering questions in a complicated manner.
SG-Net: Syntax Guided Transformer for Language Representation
In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention.
Reference Knowledgeable Network for Machine Reading Comprehension
Thus we propose a novel reference-based knowledge enhancement model called Reference Knowledgeable Network (RekNet), which simulates human reading strategies to refine critical information from the passage and quote explicit knowledge in necessity.
LIMIT-BERT : Linguistics Informed Multi-Task BERT
Besides, LIMIT-BERT takes a semi-supervised learning strategy to offer the same large amount of linguistics task data as that for the language model training.
Topic-Aware Multi-turn Dialogue Modeling
In the retrieval-based multi-turn dialogue modeling, it remains a challenge to select the most appropriate response according to extracting salient features in context utterances.
Multi-span Style Extraction for Generative Reading Comprehension
Generative machine reading comprehension (MRC) requires a model to generate well-formed answers.
Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue
A multi-turn dialogue is composed of multiple utterances from two or more different speaker roles.
Composing Answer from Multi-spans for Reading Comprehension
This paper presents a novel method to generate answers for non-extraction machine reading comprehension (MRC) tasks whose answers cannot be simply extracted as one span from the given passages.
Dialogue-adaptive Language Model Pre-training From Quality Estimation
Pre-trained language models (PrLMs) have achieved great success on a wide range of natural language processing tasks by virtue of the universal language representation ability obtained by self-supervised learning on a large corpus.
Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
In this survey, we provide a comprehensive and comparative review on MRC covering overall research topics about 1) the origin and development of MRC and CLM, with a particular focus on the role of CLMs; 2) the impact of MRC and CLM to the NLP community; 3) the definition, datasets, and evaluation of MRC; 4) general MRC architecture and technical methods in the view of two-stage Encoder-Decoder solving architecture from the insights of the cognitive process of humans; 5) previous highlights, emerging topics, and our empirical analysis, among which we especially focus on what works in different periods of MRC researches.
Neural Machine Translation with Universal Visual Representation
Though visual information has been introduced for enhancing neural machine translation (NMT), its effectiveness strongly relies on the availability of large amounts of bilingual parallel sentence pairs with manual image annotations.
Data-dependent Gaussian Prior Objective for Language Generation
However, MLE focuses on once-to-all matching between the predicted sequence and gold-standard, consequently treating all incorrect predictions as being equally incorrect.
Knowledgeable Dialogue Reading Comprehension on Key Turns
In this paper, the relevance of each turn to the question are calculated to choose key turns.
Retrospective Reader for Machine Reading Comprehension
Inspired by how humans solve reading comprehension questions, we proposed a retrospective reader (Retro-Reader) that integrates two stages of reading and verification strategies: 1) sketchy reading that briefly investigates the overall interactions of passage and question, and yield an initial judgment; 2) intensive reading that verifies the answer and gives the final prediction.
Ranked #7 on
Question Answering
on SQuAD2.0
Explicit Sentence Compression for Neural Machine Translation
In this paper, we propose an explicit sentence compression method to enhance the source sentence representation for NMT.
Probing Contextualized Sentence Representations with Visual Awareness
We present a universal framework to model contextualized sentence representations with visual awareness that is motivated to overcome the shortcomings of the multimodal parallel data with manual annotations.
SJTU-NICT at MRP 2019: Multi-Task Learning for End-to-End Uniform Semantic Graph Parsing
This paper describes our SJTU-NICT{'}s system for participating in the shared task on Cross-Framework Meaning Representation Parsing (MRP) at the 2019 Conference for Computational Language Learning (CoNLL).
LIMIT-BERT : Linguistic Informed Multi-Task BERT
In this paper, we present a Linguistic Informed Multi-Task BERT (LIMIT-BERT) for learning language representations across multiple linguistic tasks by Multi-Task Learning (MTL).
Semantics-aware BERT for Language Understanding
The latest work on language representations carefully integrates contextualized features into language model training, which enables a series of success especially in various machine reading comprehension and natural language inference tasks.
Ranked #6 on
Natural Language Inference
on SNLI
Modeling Named Entity Embedding Distribution into Hypersphere
This work models named entity distribution from a way of visualizing topological structure of embedding space, so that we make an assumption that most, if not all, named entities (NEs) for a language tend to aggregate together to be accommodated by a specific hypersphere in embedding space.
A Smart Sliding Chinese Pinyin Input Method Editor on Touchscreen
This paper presents a smart sliding Chinese pinyin Input Method Editor (IME) for touchscreen devices which allows user finger sliding from one key to another on the touchscreen instead of tapping keys one by one, while the target Chinese character sequence will be predicted during the sliding process to help user input Chinese characters efficiently.
Open Named Entity Modeling from Embedding Distribution
For monolingual cases, the proposed named entity model gives an open description of diverse named entity types and different languages.
DCMN+: Dual Co-Matching Network for Multi-choice Reading Comprehension
Multi-choice reading comprehension is a challenging task to select an answer from a set of candidate options when given passage and question.
SG-Net: Syntax-Guided Machine Reading Comprehension
In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention.
Ranked #5 on
Question Answering
on SQuAD2.0 dev
Judging Chemical Reaction Practicality From Positive Sample only Learning
Chemical reaction practicality is the core task among all symbol intelligence based chemical information processing, for example, it provides indispensable clue for further automatic synthesis route inference.
Dual Co-Matching Network for Multi-choice Reading Comprehension
Multi-choice reading comprehension is a challenging task that requires complex reasoning procedure.
Ranked #3 on
Question Answering
on RACE
Dependency or Span, End-to-End Uniform Semantic Role Labeling
Semantic role labeling (SRL) aims to discover the predicateargument structure of a sentence.
Ranked #9 on
Semantic Role Labeling
on CoNLL 2005
Open Vocabulary Learning for Neural Chinese Pinyin IME
Pinyin-to-character (P2C) conversion is the core component of pinyin-based Chinese input method engine (IME).
Effective Subword Segmentation for Text Comprehension
Representation learning is the foundation of machine reading comprehension and inference.
A Unified Syntax-aware Framework for Semantic Role Labeling
Semantic role labeling (SRL) aims to recognize the predicate-argument structure of a sentence.
Joint Learning of POS and Dependencies for Multilingual Universal Dependency Parsing
This paper describes the system of team LeisureX in the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies.
Explicit Contextual Semantics for Text Comprehension
Who did what to whom is a major focus in natural language understanding, which is right the aim of semantic role labeling (SRL) task.
Ranked #9 on
Natural Language Inference
on SNLI
Machine Reading Comprehension
Natural Language Understanding
+1
Attentive Semantic Role Labeling with Boundary Indicator
The goal of semantic role labeling (SRL) is to discover the predicate-argument structure of a sentence, which plays a critical role in deep processing of natural language.
Lingke: A Fine-grained Multi-turn Chatbot for Customer Service
Traditional chatbots usually need a mass of human dialogue data, especially when using supervised machine learning method.
Effective Character-augmented Word Embedding for Machine Reading Comprehension
Machine reading comprehension is a task to model relationship between passage and query.
Moon IME: Neural-based Chinese Pinyin Aided Input Method with Customizable Association
Chinese pinyin input method engine (IME) lets user conveniently input Chinese into a computer by typing pinyin through the common keyboard.
Modeling Multi-turn Conversation with Deep Utterance Aggregation
In this paper, we formulate previous utterances into context using a proposed deep utterance aggregation model to form a fine-grained context representation.
Ranked #11 on
Conversational Response Selection
on E-commerce
Subword-augmented Embedding for Cloze Reading Comprehension
Representation learning is the foundation of machine reading comprehension.
One-shot Learning for Question-Answering in Gaokao History Challenge
Answering questions from university admission exams (Gaokao in Chinese) is a challenging AI task since it requires effective representation to capture complicated semantic relations between questions and answers.
SJTU-NLP at SemEval-2018 Task 9: Neural Hypernym Discovery with Term Embeddings
This paper describes a hypernym discovery system for our participation in the SemEval-2018 Task 9, which aims to discover the best (set of) candidate hypernyms for input concepts or entities, given the search space of a pre-defined vocabulary.
Ranked #5 on
Hypernym Discovery
on Music domain
