Comparing two sentences and their relationship based on their internal representation.
Bi-encoders, on the other hand, require substantial training data and fine-tuning over the target task to achieve competitive performance.
Moreover, it is shown that reasonable performance can be obtained when ZEN is trained on a small corpus, which is important for applying pre-training techniques to scenarios with limited data.
Ranked #1 on
Chinese Part-of-Speech Tagging
on CTB5
CHINESE NAMED ENTITY RECOGNITION CHINESE WORD SEGMENTATION DOCUMENT CLASSIFICATION NATURAL LANGUAGE INFERENCE PART-OF-SPEECH TAGGING SENTENCE PAIR MODELING SENTIMENT ANALYSIS
In this paper, we analyze several neural network designs (and their variations) for sentence pair modeling and compare their performance extensively across eight datasets, including paraphrase identification, semantic textual similarity, natural language inference, and question answering tasks.
Ranked #1 on
Paraphrase Identification
on 2017_test set
NATURAL LANGUAGE INFERENCE PARAPHRASE IDENTIFICATION QUESTION ANSWERING SENTENCE PAIR MODELING
Sentence pair modeling is critical for many NLP tasks, such as paraphrase identification, semantic textual similarity, and natural language inference.
NATURAL LANGUAGE INFERENCE PARAPHRASE IDENTIFICATION SENTENCE PAIR MODELING
In this paper, we introduce Distilled Sentence Embedding (DSE) - a model that is based on knowledge distillation from cross-attentive models, focusing on sentence-pair tasks.
KNOWLEDGE DISTILLATION NATURAL LANGUAGE UNDERSTANDING SEMANTIC SIMILARITY SENTENCE EMBEDDING