Semi-Supervised Learning Methods

Self-Training with Task Augmentation

Introduced by Vu et al. in STraTA: Self-Training with Task Augmentation for Better Few-shot Learning

STraTA, or Self-Training with Task Augmentation, is a self-training approach that builds on two key ideas for effective leverage of unlabeled data. First, STraTA uses task augmentation, a technique that synthesizes a large amount of data for auxiliary-task fine-tuning from target-task unlabeling texts. Second, STRATA performs self-training by further fine-tuning the strong base model created by task augmentation on a broad distribution of pseudo-labeled data.

In task augmentation, we train an NLI data generation model and use it to synthesize a large amount of in-domain NLI training data for each given target task, which is then used for auxiliary (intermediate) fine-tuning. The self-training algorithm iteratively learns a better model using a concatenation of labeled and pseudo-labeled examples. At each iteration, we always start with the auxiliary-task model produced by task augmentation and train on a broad distribution of pseudo-labeled data.

Source: STraTA: Self-Training with Task Augmentation for Better Few-shot Learning

Papers


Paper Code Results Date Stars

Tasks


Task Papers Share
Few-Shot Learning 1 100.00%

Components


Component Type
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories