Search Results for author:
Found 26 papers, 13 papers with code
Continual Few-Shot Learning for Text Classification
In this work, we propose a continual few-shot learning (CFL) task, in which a system is challenged with a difficult phenomenon and asked to learn to correct mistakes with only a few (10 to 15) training examples.
Prompt-free and Efficient Few-shot Learning with Language Models
Current methods for few-shot fine-tuning of pretrained masked language models (PLMs) require carefully engineered prompts and verbalizers for each new task to convert examples into a cloze-format that the PLM can score.
Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model
Large and sparse feed-forward layers (S-FFN) such as Mixture-of-Experts (MoE) have proven effective in scaling up Transformers model size for \textit{pretraining} large language models.
Improving In-Context Few-Shot Learning via Self-Supervised Training
Self-supervised pretraining has made few-shot learning possible for many NLP tasks.
PERFECT: Prompt-free and Efficient Few-shot Learning with Language Models
Current methods for few-shot fine-tuning of pretrained masked language models (PLMs) require carefully engineered prompts and verbalizers for each new task to convert examples into a cloze-format that the PLM can score.
Few-shot Learning with Multilingual Language Models
Large-scale generative language models such as GPT-3 are competitive few-shot learners.
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs
In this paper, we discuss approaches to detecting when models have beliefs about the world, and we improve on methods for updating model beliefs to be more truthful, with a focus on methods based on learned optimizers or hypernetworks.
Multi-task Retrieval for Knowledge-Intensive Tasks
Retrieving relevant contexts from a large corpus is a crucial step for tasks such as open-domain question answering and fact checking.
Pretrained Language Models for Biomedical and Clinical Tasks: Understanding and Extending the State-of-the-Art
A large array of pretrained models are available to the biomedical NLP (BioNLP) community.
General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference
The resulting method offers a compelling solution for using large-scale pre-trained models at a fraction of the computational cost when multiple tasks are performed on the same text.
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model
Models trained with our new objective yield significant improvements on the fact completion task.
SemEval-2013 Task 2: Sentiment Analysis in Twitter
To address this issue, we have proposed SemEval-2013 Task 2: Sentiment Analysis in Twitter, which included two subtasks: A, an expression-level subtask, and B, a message-level subtask.
SemEval-2014 Task 9: Sentiment Analysis in Twitter
We describe the Sentiment Analysis in Twitter task, ran as part of SemEval-2014.
SemEval-2015 Task 10: Sentiment Analysis in Twitter
In this paper, we describe the 2015 iteration of the SemEval shared task on Sentiment Analysis in Twitter.
SemEval-2016 Task 4: Sentiment Analysis in Twitter
The three new subtasks focus on two variants of the basic ``sentiment classification in Twitter'' task.
Unsupervised Cross-lingual Representation Learning at Scale
We also present a detailed empirical analysis of the key factors that are required to achieve these gains, including the trade-offs between (1) positive transfer and capacity dilution and (2) the performance of high and low resource languages at scale.
Emerging Cross-lingual Structure in Pretrained Language Models
We study the problem of multilingual masked language modeling, i. e. the training of a single model on concatenated text from multiple languages, and present a detailed study of several factors that influence why these models are so effective for cross-lingual transfer.
Bridging the domain gap in cross-lingual document classification
We consider the setting of semi-supervised cross-lingual understanding, where labeled data is available in a source language (English), but only unlabeled data is available in the target language.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging.
Ranked #1 on
Only Connect Walls Dataset Task 1 (Grouping)
on OCW
(Wasserstein Distance (WD) metric, using extra
training data)
Knowledge-Augmented Language Model and its Application to Unsupervised Named-Entity Recognition
Our Knowledge-Augmented Language Model (KALM) continues this line of work by augmenting a traditional model with a KB.
XNLI: Evaluating Cross-lingual Sentence Representations
State-of-the-art natural language processing systems rely on supervision in the form of annotated data to learn competent models.
Ranked #5 on
Natural Language Inference
on XNLI French
Cross-Lingual Natural Language Inference
Machine Translation
+2
Simple Fusion: Return of the Language Model
Neural Machine Translation (NMT) typically leverages monolingual data in training through backtranslation.
A Multi-lingual Multi-task Architecture for Low-resource Sequence Labeling
We propose a multi-lingual multi-task architecture to develop supervised models with a minimal amount of labeled data for sequence labeling.
