Search Results for author:
Found 225 papers, 74 papers with code
LayoutLM: Pre-training of Text and Layout for Document Image Understanding
In this paper, we propose the \textbf{LayoutLM} to jointly model interactions between text and layout information across scanned document images, which is beneficial for a great number of real-world document image understanding tasks such as information extraction from scanned documents.
Ranked #7 on
Relation Extraction
on FUNSD
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
This paper presents a new sequence-to-sequence pre-training model called ProphetNet, which introduces a novel self-supervised objective named future n-gram prediction and the proposed n-stream self-attention mechanism.
Ranked #6 on
Question Generation
on SQuAD1.1
(using extra training data)
Unified Language Model Pre-training for Natural Language Understanding and Generation
This paper presents a new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks.
Ranked #2 on
Generative Question Answering
on CoQA
(using extra training data)
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
The small model (student) is trained by deeply mimicking the self-attention module, which plays a vital role in Transformer networks, of the large model (teacher).
Ranked #8 on
Zero-shot Text Search
on BEIR
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
We propose to pre-train a unified language model for both autoencoding and partially autoregressive language modeling tasks using a novel training procedure, referred to as a pseudo-masked language model (PMLM).
Ranked #4 on
Question Generation
on SQuAD1.1
(using extra training data)
Pseudo-Masked Language Models for Unified Language Model Pre-Training
We propose to pre-train a unified language model for both autoencoding and partially autoregressive language modeling tasks using a novel training procedure, referred to as a pseudo-masked language model (PMLM).
InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
In this work, we present an information-theoretic framework that formulates cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts.
Ranked #16 on
Zero-Shot Cross-Lingual Transfer
on XTREME
MIND: A Large-scale Dataset for News Recommendation
News recommendation is an important technique for personalized news service.
Neural Speech Synthesis with Transformer Network
Although end-to-end neural text-to-speech (TTS) methods (such as Tacotron2) are proposed and achieve state-of-the-art performance, they still suffer from two problems: 1) low efficiency during training and inference; 2) hard to model long dependency using current recurrent neural networks (RNNs).
Ranked #9 on
Text-To-Speech Synthesis
on LJSpeech
(using extra training data)
CodeBLEU: a Method for Automatic Evaluation of Code Synthesis
Evaluation metrics play a vital role in the growth of an area as it defines the standard of distinguishing between good and bad models.
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Results show that CodeBERT achieves state-of-the-art performance on both natural language code search and code documentation generation tasks.
Ranked #1 on
Code Documentation Generation
on CodeSearchNet - Go
GraphCodeBERT: Pre-training Code Representations with Data Flow
Instead of taking syntactic-level structure of code like abstract syntax tree (AST), we use data flow in the pre-training stage, which is a semantic-level structure of code that encodes the relation of "where-the-value-comes-from" between variables.
Ranked #3 on
Type prediction
on ManyTypes4TypeScript
UniXcoder: Unified Cross-Modal Pre-training for Code Representation
Furthermore, we propose to utilize multi-modal contents to learn representation of code fragment with contrastive learning, and then align representations among programming languages using a cross-modal generation task.
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Benchmark datasets have a significant impact on accelerating research in programming language tasks.
Ranked #1 on
Cloze Test
on CodeXGLUE - CT-maxmin
TableBank: A Benchmark Dataset for Table Detection and Recognition
We present TableBank, a new image-based table detection and recognition dataset built with novel weak supervision from Word and Latex documents on the internet.
TableBank: Table Benchmark for Image-based Table Detection and Recognition
We present TableBank, a new image-based table detection and recognition dataset built with novel weak supervision from Word and Latex documents on the internet.
SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving
We open-source the SMARTS platform and the associated benchmark tasks and evaluation metrics to encourage and empower research on multi-agent learning for autonomous driving.
Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots
Existing work either concatenates utterances in context or matches a response with a highly abstract context vector finally, which may lose relationships among utterances or important contextual information.
Ranked #7 on
Conversational Response Selection
on RRS
Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese
Although pre-trained models (PLMs) have achieved remarkable improvements in a wide range of NLP tasks, they are expensive in terms of time and resources.
DocBank: A Benchmark Dataset for Document Layout Analysis
DocBank is constructed using a simple yet effective way with weak supervision from the \LaTeX{} documents available on the arXiv. com.
MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning
Our framework is comprised of three key components: (1) a centralized task dispatching model, which supports the self-generated tasks and scalable training with heterogeneous policy combinations; (2) a programming architecture named Actor-Evaluator-Learner, which achieves high parallelism for both training and sampling, and meets the evaluation requirement of auto-curriculum learning; (3) a higher-level abstraction of MARL training paradigms, which enables efficient code reuse and flexible deployments on different distributed computing paradigms.
Mean Field Multi-Agent Reinforcement Learning
Existing multi-agent reinforcement learning methods are limited typically to a small number of agents.
UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation
However, most of the existing multimodal models are pre-trained for understanding tasks, leading to a pretrain-finetune discrepancy for generation tasks.
Ranked #2 on
Action Segmentation
on COIN
(using extra training data)
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
Our approach first divides the original BERT into several modules and builds their compact substitutes.
MuTual: A Dataset for Multi-Turn Dialogue Reasoning
Non-task oriented dialogue systems have achieved great success in recent years due to largely accessible conversation data and the development of deep learning techniques.
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
We study the problem of injecting knowledge into large pre-trained models like BERT and RoBERTa.
Ranked #1 on
Entity Typing
on Open Entity
Neural Document Summarization by Jointly Learning to Score and Select Sentences
In this paper, we present a novel end-to-end neural network framework for extractive document summarization by jointly learning to score and select sentences.
Ranked #9 on
Extractive Text Summarization
on CNN / Daily Mail
Neural Question Generation from Text: A Preliminary Study
Automatic question generation aims to generate questions from a text passage where the generated questions can be answered by certain sub-spans of the given passage.
Ranked #13 on
Question Generation
on SQuAD1.1
Reinforced Mnemonic Reader for Machine Reading Comprehension
In this paper, we introduce the Reinforced Mnemonic Reader for machine reading comprehension tasks, which enhances previous attentive readers in two aspects.
Ranked #17 on
Question Answering
on SQuAD1.1 dev
On Realization of Intelligent Decision-Making in the Real World: A Foundation Decision Model Perspective
The pervasive uncertainty and dynamic nature of real-world environments present significant challenges for the widespread implementation of machine-driven Intelligent Decision-Making (IDM) systems.
Topic Aware Neural Response Generation
We consider incorporating topic information into the sequence-to-sequence framework to generate informative and interesting responses for chatbots.
Continuous Speech Separation with Conformer
Continuous speech separation plays a vital role in complicated speech related tasks such as conversation transcription.
Ranked #1 on
Speech Separation
on LibriCSS
(using extra training data)
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
In this paper, we introduce XGLUE, a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora and evaluate their performance across a diverse set of cross-lingual tasks.
Compare to The Knowledge: Graph Neural Fake News Detection with External Knowledge
Specifically, we first construct a \textit{directed heterogeneous document graph} for each news incorporating topics and entities.
Response Generation by Context-aware Prototype Editing
Open domain response generation has achieved remarkable progress in recent years, but sometimes yields short and uninformative responses.
Achieving Human Parity on Automatic Chinese to English News Translation
Machine translation has made rapid advances in recent years.
Ranked #3 on
Machine Translation
on WMT 2017 English-Chinese
Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension
Natural Questions is a new challenging machine reading comprehension benchmark with two-grained answers, which are a long answer (typically a paragraph) and a short answer (one or more entities inside the long answer).
BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation
Standard automatic metrics, e. g. BLEU, are not reliable for document-level MT evaluation.
A Tensorized Transformer for Language Modeling
In this paper, based on the ideas of tensor decomposition and parameters sharing, we propose a novel self-attention model (namely Multi-linear attention) with Block-Term Tensor Decomposition (BTD).
Graph Neural News Recommendation with Unsupervised Preference Disentanglement
Furthermore, the learned representations are disentangled with latent preference factors by a neighborhood routing algorithm, which can enhance expressiveness and interpretability.
Hierarchical Recurrent Attention Network for Response Generation
With the word level attention, hidden vectors of a word level encoder are synthesized as utterance vectors and fed to an utterance level encoder to construct hidden representations of the context.
GLGE: A New General Language Generation Evaluation Benchmark
Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP).
Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder
Generating inferential texts about an event in different perspectives requires reasoning over different contexts that the event occurs.
Ranked #1 on
Common Sense Reasoning
on Event2Mind test
(BLEU metric)
Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text
Logical reasoning of text requires understanding critical logical information in the text and performing inference over them.
Ranked #7 on
Reading Comprehension
on ReClor
CoSQA: 20,000+ Web Queries for Code Search and Question Answering
Finding codes given natural language query isb eneficial to the productivity of software developers.
Constraint-Based Question Answering with Knowledge Graph
WebQuestions and SimpleQuestions are two benchmark data-sets commonly used in recent knowledge-based question answering (KBQA) work.
Dialog-to-Action: Conversational Question Answering Over a Large-Scale Knowledge Base
We present an approach to map utterances in conversation to logical forms, which will be executed on a large-scale knowledge base.
Semantic Mask for Transformer based End-to-End Speech Recognition
Attention-based encoder-decoder model has achieved impressive results for both automatic speech recognition (ASR) and text-to-speech (TTS) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Selective Encoding for Abstractive Sentence Summarization
We propose a selective encoding model to extend the sequence-to-sequence framework for abstractive sentence summarization.
Ranked #8 on
Text Summarization
on DUC 2004 Task 1
Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study
Neural sequence-to-sequence (seq2seq) approaches have proven to be successful in grammatical error correction (GEC).
Ranked #1 on
Grammatical Error Correction
on Unrestricted
Unsupervised Neural Machine Translation with SMT as Posterior Regularization
To address this issue, we introduce phrase based Statistic Machine Translation (SMT) models which are robust to noisy data, as posterior regularizations to guide the training of unsupervised NMT models in the iterative back-translation process.
AR-LSAT: Investigating Analytical Reasoning of Text
Analytical reasoning is an essential and challenging task that requires a system to analyze a scenario involving a set of particular circumstances and perform reasoning over it to make conclusions.
From LSAT: The Progress and Challenges of Complex Reasoning
Complex reasoning aims to draw a correct inference based on complex rules.
Analytical Reasoning of Text
In this paper, we study the challenge of analytical reasoning of text and collect a new dataset consisting of questions from the Law School Admission Test from 1991 to 2016.
Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers
We also find in experiments that our model is less dependent on sentence positions.
BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
In this paper, we propose BANG, a new pretraining model to Bridge the gap between Autoregressive (AR) and Non-autoregressive (NAR) Generation.
Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
In this paper, we propose a novel data augmentation method, referred to as Controllable Rewriting based Question Data Augmentation (CRQDA), for machine reading comprehension (MRC), question generation, and question-answering natural language inference tasks.
Response Selection with Topic Clues for Retrieval-based Chatbots
The message vector, the response vector, and the two topic vectors are fed to neural tensors to calculate a matching score.
Neural Deepfake Detection with Factual Structure of Text
To address this, we propose a graph-based model that utilizes the factual structure of a document for deepfake detection of text.
R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering
To better utilize semantic knowledge in images, we propose a novel framework to learn visual relation facts for VQA.
LLMaAA: Making Large Language Models as Active Annotators
Recently, the superior few-shot performance of large language models (LLMs) has propelled the development of dataset generation, where the training data are solely synthesized from LLMs.
Multi-Agent Interactions Modeling with Correlated Policies
In this paper, we cast the multi-agent interactions modeling problem into a multi-agent imitation learning framework with explicit modeling of correlated policies by approximating opponents' policies, which can recover agents' policies that can regenerate similar interactions.
Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rollouts
We specify the dynamics sample complexity and the opponent sample complexity in MARL, and conduct a theoretic analysis of return discrepancy upper bound.
Multi-agent Reinforcement Learning
Reinforcement Learning (RL)
Multiway Attention Networks for Modeling Sentence Pairs
Modeling sentence pairs plays the vital role for judging the relationship between two sentences, such as paraphrase identification, natural language inference, and answer sentence selection.
Ranked #11 on
Paraphrase Identification
on Quora Question Pairs
(Accuracy metric)
ProQA: Structural Prompt-based Pre-training for Unified Question Answering
Furthermore, ProQA exhibits strong ability in both continual learning and transfer learning by taking the advantages of the structural prompt.
Learning to Ask Conversational Questions by Optimizing Levenshtein Distance
Conversational Question Simplification (CQS) aims to simplify self-contained questions into conversational ones by incorporating some conversational characteristics, e. g., anaphora and ellipsis.
Pre-Training a Graph Recurrent Network for Language Representation
Transformer-based pre-trained models have gained much advance in recent years, becoming one of the most important backbones in natural language processing.
BERT-based Lexical Substitution
Our approach first applies dropout to the target word{'}s embedding for partially masking the word, allowing BERT to take balanced consideration of the target word{'}s semantics and contexts for proposing substitute candidates, and then validates the candidates based on their substitution{'}s influence on the global contextualized representation of the sentence.
Instance Regularization for Discriminative Language Model Pre-training
They treat training instances equally throughout the training process, with little attention on the individual contribution of those instances.
Improving Grammatical Error Correction with Machine Translation Pairs
The poor translation model resembles the ESL (English as a second language) learner and tends to generate translations of low quality in terms of fluency and grammatical correctness, while the good translation model generally generates fluent and grammatically correct translations.
A Retrieve-and-Rewrite Initialization Method for Unsupervised Machine Translation
The commonly used framework for unsupervised machine translation builds initial translation models of both translation directions, and then performs iterative back-translation to jointly boost their translation performance.
Sequential Copying Networks
Copying mechanism shows effectiveness in sequence-to-sequence based neural network models for text generation tasks, such as abstractive sentence summarization and question generation.
Scheduled DropHead: A Regularization Method for Transformer Models
In this paper, we introduce DropHead, a structured dropout method specifically designed for regularizing the multi-head attention mechanism, which is a key component of transformer, a state-of-the-art model for various NLP tasks.
Table-to-Text: Describing Table Region with Natural Language
The model maps a row from a table to a continuous vector and then generates a natural language sentence by leveraging the semantics of a table.
Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization
Normalization techniques such as Batch Normalization have been applied successfully for training deep neural networks.
Triangular Architecture for Rare Language Translation
Neural Machine Translation (NMT) performs poor on the low-resource language pair $(X, Z)$, especially when $Z$ is a rare language.
Neural Open Information Extraction
Conventional Open Information Extraction (Open IE) systems are usually built on hand-crafted patterns from other NLP tools such as syntactic parsing, yet they face problems of error propagation.
Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots
We propose a method that can leverage unlabeled data to learn a matching model for response selection in retrieval-based chatbots.
Semantic Parsing with Syntax- and Table-Aware SQL Generation
We present a generative model to map natural language questions into SQL queries.
Ranked #4 on
Code Generation
on WikiSQL
Generative Bridging Network in Neural Sequence Prediction
In order to alleviate data sparsity and overfitting problems in maximum likelihood estimation (MLE) for sequence prediction tasks, we propose the Generative Bridging Network (GBN), in which a novel bridge module is introduced to assist the training of the sequence prediction model (the generator network).
Joint Training for Neural Machine Translation Models with Monolingual Data
Monolingual data have been demonstrated to be helpful in improving translation quality of both statistical machine translation (SMT) systems and neural machine translation (NMT) systems, especially in resource-poor or domain adaptation tasks where parallel data are not rich enough.
Assertion-based QA with Question-Aware Open Information Extraction
We present assertion based question answering (ABQA), an open domain question answering task that takes a question and a passage as inputs, and outputs a semi-structured assertion consisting of a subject, a predicate and a list of arguments.
S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension
We build the answer extraction model with state-of-the-art neural networks for single passage reading comprehension, and propose an additional task of passage ranking to help answer extraction in multiple passages.
Neural Response Generation with Dynamic Vocabularies
We study response generation for open domain conversation in chatbots.
A Sequential Matching Framework for Multi-turn Response Selection in Retrieval-based Chatbots
The task requires matching a response candidate with a conversation context, whose challenges include how to recognize important parts of the context, and how to model the relationships among utterances in the context.
Question Answering and Question Generation as Dual Tasks
On one side, the QA model judges whether the generated question of a QG model is relevant to the answer.
Content-Based Table Retrieval for Web Queries
Understanding the connections between unstructured text and semi-structured table is an important yet neglected problem in natural language processing.
Entity Linking for Queries by Searching Wikipedia Sentences
The key idea is to search sentences similar to a query from Wikipedia articles and directly use the human-annotated entities in the similar sentences as candidate entities for the query.
Knowledge Enhanced Hybrid Neural Network for Text Matching
Long text brings a big challenge to semantic matching due to their complicated semantic and syntactic structures.
Detecting Context Dependent Messages in a Conversational Environment
While automatic response generation for building chatbot systems has drawn a lot of attention recently, there is limited understanding on when we need to consider the linguistic context of an input text in the generation process.
Aligning Coordinated Text Streams through Burst Information Network Construction and Decipherment
We introduce a novel Burst Information Network (BINet) representation that can display the most important information and illustrate the connections among bursty entities, events and keywords in the corpus.
Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction
In this paper, we develop a novel approach to aspect term extraction based on unsupervised learning of distributed representations of words and dependency paths.
Implicit Distortion and Fertility Models for Attention-based Encoder-Decoder NMT Model
Aiming to resolve these problems, we propose new variations of attention-based encoder-decoder and compare them with other models on machine translation.
Jointly Modeling Topics and Intents with Global Order Structure
Modeling document structure is of great importance for discourse analysis and related applications.
TGSum: Build Tweet Guided Multi-Document Summarization Dataset
Both informativeness and readability of the collected summaries are verified by manual judgment.
A Dependency-Based Neural Network for Relation Classification
Previous research on relation classification has verified the effectiveness of using dependency shortest paths or subtrees.
Ranked #5 on
Relation Classification
on SemEval 2010 Task 8
Multi-Document Summarization via Discriminative Summary Reranking
However, according to our quantitative analysis, none of the existing summarization models can always produce high-quality summaries for different document sets, and even a summarization model with good overall performance may produce low-quality summaries for some document sets.
A Statistical Parsing Framework for Sentiment Classification
Unlike previous works that employ syntactic parsing results for sentiment analysis, we develop a statistical parser to directly analyze the sentiment structure of a sentence.
Beyond Word-based Language Model in Statistical Machine Translation
Language model is one of the most important modules in statistical machine translation and currently the word-based language model dominants this community.
Dictionary-Guided Editing Networks for Paraphrase Generation
An intuitive way for a human to write paraphrase sentences is to replace words or phrases in the original sentence with their corresponding synonyms and make necessary changes to ensure the new sentences are fluent and grammatically correct.
Regularizing Neural Machine Translation by Target-bidirectional Agreement
Although Neural Machine Translation (NMT) has achieved remarkable progress in the past several years, most NMT systems still suffer from a fundamental shortcoming as in other sequence generation tasks: errors made early in generation process are fed as inputs to the model and can be quickly amplified, harming subsequent sequence generation.
Question Generation from SQL Queries Improves Neural Semantic Parsing
We study how to learn a semantic parser of state-of-the-art accuracy with less supervised training data.
Neural Latent Extractive Document Summarization
Extractive summarization models require sentence-level labels, which are usually created heuristically (e. g., with rule-based methods) given that most summarization datasets only have document-summary pairs.
Ranked #11 on
Extractive Text Summarization
on CNN / Daily Mail
Attention-Guided Answer Distillation for Machine Reading Comprehension
Despite that current reading comprehension systems have achieved significant advancements, their promising performances are often obtained at the cost of making an ensemble of numerous models.
Approximate Distribution Matching for Sequence-to-Sequence Learning
Then, we interpret sequence-to-sequence learning as learning a transductive model to transform the source local latent distributions to match their corresponding target distributions.
Style Transfer as Unsupervised Machine Translation
Language style transferring rephrases text with specific stylistic attributes while preserving the original attribute-independent content.
Ranked #3 on
Unsupervised Text Style Transfer
on GYAFC
Neural Melody Composition from Lyrics
In this paper, we study a novel task that learns to compose music from natural language.
Text Morphing
In this paper, we introduce a novel natural language generation task, termed as text morphing, which targets at generating the intermediate sentences that are fluency and smooth with the two input sentences.
Improving Question Answering by Commonsense-Based Pre-Training
Although neural network approaches achieve remarkable success on a variety of NLP tasks, many of them struggle to answer questions that require commonsense knowledge.
Fine-grained Coordinated Cross-lingual Text Stream Alignment for Endless Language Knowledge Acquisition
This paper proposes to study fine-grained coordinated cross-lingual text stream alignment through a novel information network decipherment paradigm.
Bidirectional Generative Adversarial Networks for Neural Machine Translation
To address this issue and stabilize the GAN training, in this paper, we propose a novel Bidirectional Generative Adversarial Network for Neural Machine Translation (BGAN-NMT), which aims to introduce a generator model to act as the discriminator, whereby the discriminator naturally considers the entire translation space so that the inadequate training problem can be alleviated.
Fluency Boost Learning and Inference for Neural Grammatical Error Correction
Most of the neural sequence-to-sequence (seq2seq) models for grammatical error correction (GEC) have two limitations: (1) a seq2seq model may not be well generalized with only limited error-corrected data; (2) a seq2seq model may fail to completely correct a sentence with multiple errors through normal seq2seq inference.
Gated Self-Matching Networks for Reading Comprehension and Question Answering
We first match the question and passage with gated attention-based recurrent networks to obtain the question-aware passage representation.
Ranked #35 on
Question Answering
on SQuAD1.1 dev
Sequence-to-Dependency Neural Machine Translation
Nowadays a typical Neural Machine Translation (NMT) model generates translations from left to right as a linear sequence, during which latent syntactic structures of the target sentences are not explicitly concerned.
Chunk-based Decoder for Neural Machine Translation
The chunk-level decoder models global dependencies while the word-level decoder decides the local word order in a chunk.
Learning to Generate Product Reviews from Attributes
This paper presents an attention-enhanced attribute-to-sequence model to generate product reviews for given attribute information, such as user, product, and rating.
Learning to Collaborate for Question Answering and Asking
Secondly, directly applying GAN that regards all the generated questions as negative instances could not improve the accuracy of the QA model.
Generative Bridging Network for Neural Sequence Prediction
In order to alleviate data sparsity and overfitting problems in maximum likelihood estimation (MLE) for sequence prediction tasks, we propose the Generative Bridging Network (GBN), in which a novel bridge module is introduced to assist the training of the sequence prediction model (the generator network).
Beihang-MSRA at SemEval-2017 Task 3: A Ranking System with Neural Matching Features for Community Question Answering
This paper presents the system in SemEval-2017 Task 3, Community Question Answering (CQA).
Question Generation for Question Answering
This paper presents how to generate questions from given passages using neural networks, where large scale QA pairs are automatically crawled and processed from Community-QA website, and used as training data.
Stack-based Multi-layer Attention for Transition-based Dependency Parsing
Although sequence-to-sequence (seq2seq) network has achieved significant success in many NLP tasks such as machine translation and text summarization, simply applying this approach to transition-based dependency parsing cannot yield a comparable performance gain as in other state-of-the-art methods, such as stack-LSTM and head selection.
A Redundancy-Aware Sentence Regression Framework for Extractive Summarization
Existing sentence regression methods for extractive summarization usually model sentence importance and redundancy in two separate processes.
Improving Attention Modeling with Implicit Distortion and Fertility for Machine Translation
In neural machine translation, the attention mechanism facilitates the translation process by producing a soft alignment between the source sentence and the target sentence.
Event Detection with Burst Information Networks
Retrospective event detection is an important task for discovering previously unidentified events in a text stream.
Factorized Q-Learning for Large-Scale Multi-Agent Systems
Deep Q-learning has achieved a significant success in single-agent decision making tasks.
Multiagent Systems
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
Neural extractive summarization models usually employ a hierarchical encoder for document encoding and they are trained using sentence-level labels, which are created heuristically using rule-based methods.
Ranked #7 on
Extractive Text Summarization
on CNN / Daily Mail
CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms
How to optimally dispatch orders to vehicles and how to trade off between immediate and future returns are fundamental questions for a typical ride-hailing platform.
Multiagent Systems
Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing
In this paper, we present an approach to incorporate retrieved datapoints as supporting evidence for context-dependent semantic parsing, such as generating source code conditioned on the class environment.
Pathologist-Level Grading of Prostate Biopsies with Artificial Intelligence
We additionally evaluated grading performance on 87 biopsies individually graded by 23 experienced urological pathologists from the International Society of Urological Pathology.
Automatic Grammatical Error Correction for Sequence-to-sequence Text Generation: An Empirical Study
Sequence-to-sequence (seq2seq) models have achieved tremendous success in text generation tasks.
Dense Procedure Captioning in Narrated Instructional Videos
Understanding narrated instructional videos is important for both research and real-world web applications.
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training
We propose Unicoder-VL, a universal encoder that aims to learn joint representations of vision and language in a pre-training manner.
Ranked #5 on
Image-to-Text Retrieval
on MS COCO
(Recall@10 metric)
Explicit Cross-lingual Pre-training for Unsupervised Machine Translation
Pre-training has proven to be effective in unsupervised machine translation due to its ability to model deep context information in cross-lingual scenarios.
Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
On XNLI, 1. 8% averaged accuracy improvement (on 15 languages) is obtained.
Cross-Lingual Natural Language Inference
Cross-Lingual Question Answering
+1
Reasoning Over Semantic-Level Graph for Fact Checking
We evaluate our system on FEVER, a benchmark dataset for fact checking, and find that rich structural information is helpful and both our graph-based mechanisms improve the accuracy.
Ranked #2 on
Fact Verification
on FEVER
Signal Instructed Coordination in Cooperative Multi-agent Reinforcement Learning
Although existing works formulate this problem into a centralized learning with decentralized execution framework, which avoids the non-stationary problem in training, their decentralized execution paradigm limits the agents' capability to coordinate.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Sequence-to-sequence Pre-training with Data Augmentation for Sentence Rewriting
We study sequence-to-sequence (seq2seq) pre-training with data augmentation for sentence rewriting.
Bridging the Gap between Pre-Training and Fine-Tuning for End-to-End Speech Translation
End-to-end speech translation, a hot topic in recent years, aims to translate a segment of audio into a specific language with an end-to-end model.
Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching
Improving the efficiency of dispatching orders to vehicles is a research hotspot in online ride-hailing systems.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension
Most machine reading comprehension (MRC) models separately handle encoding and matching with different network architectures.
Asking Clarification Questions in Knowledge-Based Question Answering
We provide representative baselines for these tasks and further introduce a coarse-to-fine model for clarification question generation.
Self-Adversarial Learning with Comparative Discrimination for Text Generation
Conventional Generative Adversarial Networks (GANs) for text generation tend to have issues of reward sparsity and mode collapse that affect the quality and diversity of generated samples.
XGPT: Cross-modal Generative Pre-Training for Image Captioning
While many BERT-based cross-modal pre-trained models produce excellent results on downstream understanding tasks like image-text retrieval and VQA, they cannot be applied to generation tasks directly.
At Which Level Should We Extract? An Empirical Analysis on Extractive Document Summarization
In this work, we show that unnecessity and redundancy issues exist when extracting full sentences, and extracting sub-sentential units is a promising alternative.
Learning to Summarize Passages: Mining Passage-Summary Pairs from Wikipedia Revision Histories
In this paper, we propose a method for automatically constructing a passage-to-summary dataset by mining the Wikipedia page revision histories.
Pre-training for Abstractive Document Summarization by Reinstating Source Text
The main idea is that, given an input text artificially constructed from a document, a model is pre-trained to reinstate the original document.
Inferential Text Generation with Multiple Knowledge Sources and Meta-Learning
In this work, we use multiple knowledge sources as fuels for the model.
Pre-training Text Representations as Meta Learning
In this work, we introduce a learning algorithm which directly optimizes model's ability to learn text representations for effective learning of downstream tasks.
Curriculum Pre-training for End-to-End Speech Translation
End-to-end speech translation poses a heavy burden on the encoder, because it has to transcribe, understand, and learn cross-lingual semantics simultaneously.
A Heterogeneous Graph with Factual, Temporal and Logical Knowledge for Question Answering Over Dynamic Contexts
We study question answering over a dynamic textual environment.
LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network
The graph is used to obtain graph-enhanced contextual representations of words in Transformer-based architecture.
Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection
We study the detection of propagandistic text fragments in news articles.
A Graph-based Coarse-to-fine Method for Unsupervised Bilingual Lexicon Induction
To deal with those issues, in this paper, we propose a novel graph-based paradigm to induce bilingual lexicons in a coarse-to-fine way.
Bilingual Lexicon Induction
Cross-Lingual Word Embeddings
+2
A Simple and Effective Unified Encoder for Document-Level Machine Translation
Most of the existing models for document-level machine translation adopt dual-encoder structures.
Improving Neural Machine Translation with Soft Template Prediction
Although neural machine translation (NMT) has achieved significant progress in recent years, most previous NMT models only depend on the source text to generate translation.
Regioned Episodic Reinforcement Learning
Goal-oriented reinforcement learning algorithms are often good at exploration, not exploitation, while episodic algorithms excel at exploitation, not exploration.
Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction
We propose a novel language-independent approach to improve the efficiency for Grammatical Error Correction (GEC) by dividing the task into two subtasks: Erroneous Span Detection (ESD) and Erroneous Span Correction (ESC).
