Search Results for author:
Found 48 papers, 13 papers with code
XGLUE: A New Benchmark Datasetfor Cross-lingual Pre-training, Understanding and Generation
In this paper, we introduce XGLUE, a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora, and evaluate their performance across a diverse set of cross-lingual tasks.
Alleviating Over-smoothing for Unsupervised Sentence Representation
Currently, learning better unsupervised sentence representations is the pursuit of many natural language processing communities.
Augmenting Passage Representations with Query Generation for Enhanced Cross-Lingual Dense Retrieval
Effective cross-lingual dense retrieval methods that rely on multilingual pre-trained language models (PLMs) need to be trained to encompass both the relevance matching task and the cross-language alignment task.
Typos-aware Bottlenecked Pre-Training for Robust Dense Retrieval
To address this challenge, we propose ToRoDer (TypOs-aware bottlenecked pre-training for RObust DEnse Retrieval), a novel \textit{pre-training} strategy for DRs that increases their robustness to misspelled queries while preserving their effectiveness in downstream retrieval tasks.
TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs
On the other hand, there are also many existing models and systems (symbolic-based or neural-based) that can do some domain-specific tasks very well.
Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval
However, we find that the performance of the cross-encoder re-ranker is heavily influenced by the number of training samples and the quality of negative samples, which is hard to obtain in the cross-lingual setting.
Large Language Models are Diverse Role-Players for Summarization Evaluation
First, we propose to model objective and subjective dimensions of generated text based on roleplayers prompting mechanism.
Lexicon-Enhanced Self-Supervised Training for Multilingual Dense Retrieval
In this paper, we propose to mine and generate self-supervised training data based on a large-scale unlabeled corpus.
Bridge the Gap between Language models and Tabular Understanding
For instance, TPLMs jointly pre-trained with table and text input could be effective for tasks also with table-text joint input like table question answering, but it may fail for tasks with only tables or text as input such as table retrieval.
Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation
This problem is further exacerbated when using DSI for cross-lingual retrieval, where document text and query text are in different languages.
Unsupervised Context Aware Sentence Representation Pretraining for Multi-lingual Dense Retrieval
On the multilingual sentence retrieval task Tatoeba, our model achieves new SOTA results among methods without using bilingual data.
Negative Sampling for Contrastive Representation Learning: A Review
The learn-to-compare paradigm of contrastive representation learning (CRL), which compares positive samples with negative ones for representation learning, has achieved great success in a wide range of domains, including natural language processing, computer vision, information retrieval and graph learning.
Label-aware Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding
Despite the great success of spoken language understanding (SLU) in high-resource languages, it remains challenging in low-resource languages mainly due to the lack of labeled training data.
Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling
Large-scale cross-lingual pre-trained language models (xPLMs) have shown effectiveness in cross-lingual sequence labeling tasks (xSL), such as cross-lingual machine reading comprehension (xMRC) by transferring knowledge from a high-resource language to low-resource languages.
Transformer-Empowered Content-Aware Collaborative Filtering
Knowledge graph (KG) based Collaborative Filtering is an effective approach to personalizing recommendation systems for relatively static domains such as movies and books, by leveraging structured information from KG to enrich both item and user representations.
From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension
Cross-lingual Machine Reading Comprehension (xMRC) is challenging due to the lack of training data in low-resource languages.
Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding
Although various data augmentation approaches have been proposed to synthesize training data in low-resource target languages, the augmented data sets are often noisy, and thus impede the performance of SLU models.
A Joint and Domain-Adaptive Approach to Spoken Language Understanding
In this paper, we attempt to bridge these two lines of research and propose a joint and domain adaptive approach to SLU.
Reinforced Iterative Knowledge Distillation for Cross-Lingual Named Entity Recognition
Named entity recognition (NER) is a fundamental component in many applications, such as Web Search and Voice Assistants.
CoSQA: 20,000+ Web Queries for Code Search and Question Answering
Finding codes given natural language query isb eneficial to the productivity of software developers.
Retrieval Enhanced Model for Commonsense Generation
Commonsense generation is a challenging task of generating a plausible sentence describing an everyday scenario using provided concepts.
WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach
In this work, we conduct a thorough examination of pretrained model based unsupervised sentence embeddings.
Generating Human Readable Transcript for Automatic Speech Recognition with Pre-trained Language Model
Many downstream tasks and human readers rely on the output of the ASR system; therefore, errors introduced by the speaker and ASR system alike will be propagated to the next task in the pipeline.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders
However, the performance of using multiple encoders and decoders on zero-shot translation still lags behind universal NMT.
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Benchmark datasets have a significant impact on accelerating research in programming language tasks.
Ranked #1 on
Cloze Test
on CodeXGLUE - CT-maxmin
Syntax-Enhanced Pre-trained Model
We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa.
Reinforced Multi-Teacher Selection for Knowledge Distillation
When multiple teacher models are available in distillation, the state-of-the-art methods assign a fixed weight to a teacher model in the whole distillation.
GLGE: A New General Language Generation Evaluation Benchmark
Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP).
CalibreNet: Calibration Networks for Multilingual Sequence Labeling
To tackle the challenge of lack of training data in low-resource languages, we dedicatedly develop a novel unsupervised phrase boundary recovery pre-training task to enhance the multilingual boundary detection capability of CalibreNet.
Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation
Then, we devise a multilingual distillation approach to amalgamate knowledge from multiple language branch models to a single model for all target languages.
Learning Better Representation for Tables by Self-Supervised Tasks
Secondly, the target texts in training dataset may contain redundant information or facts do not exist in the input tables.
A Graph Representation of Semi-structured Data for Web Question Answering
The abundant semi-structured data on the Web, such as HTML-based tables and lists, provide commercial search engines a rich information source for question answering (QA).
MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension
In this paper, we propose a novel approach that extends the probability vector to a probability matrix.
No Answer is Better Than Wrong Answer: A Reflection Model for Document Level Machine Reading Comprehension
The Natural Questions (NQ) benchmark set brings new challenges to Machine Reading Comprehension: the answers are not only at different levels of granularity (long and short), but also of richer types (including no-answer, yes/no, single-span and multi-span).
Tag and Correct: Question aware Open Information Extraction with Two-stage Decoding
The semi-structured answer has two advantages which are more readable and falsifiable compared to span answer.
Mining Implicit Relevance Feedback from User Behavior for Web Question Answering
In this paper, we make the first study to explore the correlation between user behavior and passage relevance, and propose a novel approach for mining training data for Web QA.
Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
Multilingual pre-trained models could leverage the training data from a rich source language (such as English) to improve performance on low resource languages.
LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network
The graph is used to obtain graph-enhanced contextual representations of words in Transformer-based architecture.
Pre-training Text Representations as Meta Learning
In this work, we introduce a learning algorithm which directly optimizes model's ability to learn text representations for effective learning of downstream tasks.
Improving Readability for Automatic Speech Recognition Transcription
In this work, we propose a novel NLP task called ASR post-processing for readability (APR) that aims to transform the noisy ASR output into a readable text for humans and downstream tasks while maintaining the semantic meaning of the speaker.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Inferential Text Generation with Multiple Knowledge Sources and Meta-Learning
In this work, we use multiple knowledge sources as fuels for the model.
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
In this paper, we introduce XGLUE, a new benchmark dataset that can be used to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora and evaluate their performance across a diverse set of cross-lingual tasks.
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Results show that CodeBERT achieves state-of-the-art performance on both natural language code search and code documentation generation tasks.
Ranked #1 on
Code Documentation Generation
on CodeSearchNet - Go
Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System
The experiment results show that our method can significantly outperform the baseline methods and even achieve comparable results with the original teacher models, along with substantial speedup of model inference.
Graph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering
In this work, we propose to automatically extract evidence from heterogeneous knowledge sources, and answer questions based on the extracted evidence.
Ranked #9 on
Common Sense Reasoning
on CommonsenseQA
Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
On XNLI, 1. 8% averaged accuracy improvement (on 15 languages) is obtained.
Cross-Lingual Natural Language Inference
Cross-Lingual Question Answering
+1
NeuronBlocks: Building Your NLP DNN Models Like Playing Lego
Deep Neural Networks (DNN) have been widely employed in industry to address various Natural Language Processing (NLP) tasks.
Model Compression with Multi-Task Knowledge Distillation for Web-scale Question Answering System
Deep pre-training and fine-tuning models (like BERT, OpenAI GPT) have demonstrated excellent results in question answering areas.
