Search Results for author:
Found 26 papers, 17 papers with code
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA
In this paper, we introduce ELECTRA-style tasks to cross-lingual language model pre-training.
Ranked #1 on
Zero-Shot Cross-Lingual Transfer
on XTREME
On the Representation Collapse of Sparse Mixture of Experts
We also present a comprehensive analysis on the representation and routing behaviors of our models.
Language Models are General-Purpose Interfaces
Experimental results across various language-only and vision-language benchmarks show that our model outperforms or is competitive with specialized models on finetuning, zero-shot generalization, and few-shot learning.
Ranked #2 on
Image Captioning
on nocaps val
Language Is Not All You Need: Aligning Perception with Language Models
A big convergence of language, multimodal perception, action, and world modeling is a key step toward artificial general intelligence.
InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
In this work, we present an information-theoretic framework that formulates cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts.
Ranked #16 on
Zero-Shot Cross-Lingual Transfer
on XTREME
Optimizing Prompts for Text-to-Image Generation
Instead of laborious human engineering, we propose prompt adaptation, a general framework that automatically adapts original user input to model-preferred prompts.
TorchScale: Transformers at Scale
Large Transformers have achieved state-of-the-art performance across many tasks.
Complicated Table Structure Recognition
It also attracts lots of attention to recognize the table structures in PDF files.
Cross-Lingual Natural Language Generation via Pre-Training
In this work we focus on transferring supervision signals of natural language generation (NLG) tasks between multiple languages.
MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs
Multilingual T5 (mT5) pretrains a sequence-to-sequence model on massive monolingual texts, which has shown promising results on many cross-lingual tasks.
Consistency Regularization for Cross-Lingual Fine-Tuning
Fine-tuning pre-trained cross-lingual language models can transfer task-specific supervision from one language to the others.
Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment
The cross-lingual language models are typically pretrained with masked language modeling on multilingual text or parallel sentences.
A Robust and Domain-Adaptive Approach for Low-Resource Named Entity Recognition
Recently, it has attracted much attention to build reliable named entity recognition (NER) systems using limited annotated data.
Ranked #3 on
Named Entity Recognition (NER)
on SciERC
(using extra training data)
Low Resource Named Entity Recognition
named-entity-recognition
+2
Unifying Cross-lingual Summarization and Machine Translation with Compression Rate
Through introducing compression rate, the information ratio between the source and the target text, we regard the MT task as a special CLS task with a compression rate of 100%.
Cross-Lingual Phrase Retrieval
In this paper, we propose XPR, a cross-lingual phrase retriever that extracts phrase representations from unlabeled example sentences.
Unsupervised Question Answering via Answer Diversifying
Previous works usually make use of heuristic rules as well as pre-trained models to construct data and train QA models.
ET5: A Novel End-to-end Framework for Conversational Machine Reading Comprehension
Conversational machine reading comprehension (CMRC) aims to assist computers to understand an natural language text and thereafter engage in a multi-turn conversation to answer questions related to the text.
Zewen at SemEval-2018 Task 1: An Ensemble Model for Affect Prediction in Tweets
This paper presents a method for Affect in Tweets, which is the task to automatically determine the intensity of emotions and intensity of sentiment of tweets.
Can Monolingual Pretrained Models Help Cross-Lingual Classification?
Multilingual pretrained language models (such as multilingual BERT) have achieved impressive results for cross-lingual transfer.
Generating Informative Dialogue Responses with Keywords-Guided Networks
Specifically, KW-Seq2Seq first uses a keywords decoder to predict some topic keywords, and then generates the final response under the guidance of them.
XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders
Multilingual machine translation enables a single model to translate between different languages.
Cross-Lingual Language Model Meta-Pretraining
The success of pretrained cross-lingual language models relies on two essential abilities, i. e., generalization ability for learning downstream tasks in a source language, and cross-lingual transferability for transferring the task knowledge to other languages.
Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning
In this paper, we elaborate upon recipes for building multilingual representation models that are not only competitive with existing state-of-the-art models but are also more parameter efficient, thereby promoting better adoption in resource-constrained scenarios and practical applications.
Bridging The Gap: Entailment Fused-T5 for Open-retrieval Conversational Machine Reading Comprehension
Open-retrieval conversational machine reading comprehension (OCMRC) simulates real-life conversational interaction scenes.
Measuring Cross-Lingual Transferability of Multilingual Transformers on Sentence Classification
Recent studies have exhibited remarkable capabilities of pre-trained multilingual Transformers, especially cross-lingual transferability.
ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training
We propose ProtLLM, a versatile cross-modal large language model (LLM) for both protein-centric and protein-language tasks.
