Search Results for author:
Found 17 papers, 8 papers with code
Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality
Along with this, we propose novel negative mining techniques in the scene graph space for improving attribute binding and relation understanding.
Speech-to-Speech Translation For A Real-world Unwritten Language
We use English-Taiwanese Hokkien as a case study, and present an end-to-end solution from training data collection, modeling choices to benchmark dataset release.
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations
We present SpeechMatrix, a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings.
Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models
In this work, we adapt prompt-based few-shot learning to ELECTRA and show that it outperforms masked language models in a wide range of tasks.
On the Role of Bidirectionality in Language Model Pre-Training
Prior work on language model pre-training has explored different architectures and learning objectives, but differences in data, hyperparameters and evaluation make a principled comparison difficult.
Improving In-Context Few-Shot Learning via Self-Supervised Training
Self-supervised pretraining has made few-shot learning possible for many NLP tasks.
Efficient Large Scale Language Modeling with Mixtures of Experts
This paper presents a detailed empirical study of how autoregressive MoE language models scale in comparison with dense models in a wide range of settings: in- and out-of-domain language modeling, zero- and few-shot priming, and full-shot fine-tuning.
Few-shot Learning with Multilingual Language Models
Large-scale generative language models such as GPT-3 are competitive few-shot learners.
Larger-Scale Transformers for Multilingual Masked Language Modeling
Our model also outperforms the RoBERTa-Large model on several English tasks of the GLUE benchmark by 0. 3% on average while handling 99 more languages.
Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning
Our proposed fine-tuning objective leads to models that are more robust to different levels of noise in the fine-tuning training data, and can generalize better to related tasks with limited labeled data.
Pretrained Language Models for Biomedical and Clinical Tasks: Understanding and Extending the State-of-the-Art
A large array of pretrained models are available to the biomedical NLP (BioNLP) community.
Self-training Improves Pre-training for Natural Language Understanding
Unsupervised pre-training has led to much recent progress in natural language understanding.
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
We propose a simple and efficient multi-hop dense retrieval approach for answering complex open-domain questions, which achieves state-of-the-art performance on two multi-hop datasets, HotpotQA and multi-evidence FEVER.
Ranked #15 on
Question Answering
on HotpotQA
General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference
The resulting method offers a compelling solution for using large-scale pre-trained models at a fraction of the computational cost when multiple tasks are performed on the same text.
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model
Models trained with our new objective yield significant improvements on the fact completion task.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.
Ranked #2 on
Text Classification
on arXiv-10
Knowledge-Augmented Language Model and its Application to Unsupervised Named-Entity Recognition
Our Knowledge-Augmented Language Model (KALM) continues this line of work by augmenting a traditional model with a KB.
