Browse > Methodology > Transfer Learning

Transfer Learning

164 papers with code · Methodology

Transfer learning is a methodology where weights from a model trained on one task are taken and either used (a) to construct a fixed feature extractor, (b) as weight initialization and/or fine-tuning.

State-of-the-art leaderboards

You can find evaluation results in the subtasks. You can also submitting evaluation metrics for this task.

Greatest papers with code

Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

18 Oct 2016tensorflow/models

The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users. Because they rely directly on sensitive data, these models are not published, but instead used as "teachers" for a "student" model.

TRANSFER LEARNING

Large-scale Simple Question Answering with Memory Networks

5 Jun 2015facebookresearch/ParlAI

Training large-scale question answering systems is complicated because training sources usually cover a small portion of the range of possible questions. This paper studies the impact of multitask and transfer learning for simple question answering; a setting for which the reasoning required to answer is quite easy, as long as one can retrieve the correct evidence given a question, which can be difficult in large-scale conditions.

QUESTION ANSWERING TRANSFER LEARNING

DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

6 Oct 2013jetpacapp/DeepBeliefSDK

We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be re-purposed to novel generic tasks. Our generic tasks may differ significantly from the originally trained tasks and there may be insufficient labeled or unlabeled data to conventionally train or adapt a deep architecture to the new tasks.

DOMAIN ADAPTATION OBJECT RECOGNITION SCENE RECOGNITION TRANSFER LEARNING

Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning

7 Jan 2019Tencent/tencent-ml-images

In existing visual representation learning tasks, deep convolutional neural networks (CNNs) are often trained on images annotated with single tags, such as ImageNet. In this work, we propose to train CNNs from images annotated with multiple tags, to enhance the quality of visual representation of the trained CNN model.

IMAGE CLASSIFICATION OBJECT DETECTION REPRESENTATION LEARNING SEMANTIC SEGMENTATION TRANSFER LEARNING

Bag of Tricks for Image Classification with Convolutional Neural Networks

4 Dec 2018dmlc/gluon-cv

Much of the recent progress made in image classification research can be credited to training procedure refinements, such as changes in data augmentations and optimization methods. In the literature, however, most refinements are either briefly mentioned as implementation details or only visible in source code.

IMAGE CLASSIFICATION OBJECT DETECTION SEMANTIC SEGMENTATION TRANSFER LEARNING

The Natural Language Decathlon: Multitask Learning as Question Answering

ICLR 2019 salesforce/decaNLP

Furthermore, we present a new Multitask Question Answering Network (MQAN) jointly learns all tasks in decaNLP without any task-specific modules or parameters in the multitask setting. Though designed for decaNLP, MQAN also achieves state of the art results on the WikiSQL semantic parsing task in the single-task setting.

DOMAIN ADAPTATION MACHINE TRANSLATION NAMED ENTITY RECOGNITION NATURAL LANGUAGE INFERENCE QUESTION ANSWERING RELATION EXTRACTION SEMANTIC PARSING SEMANTIC ROLE LABELING SENTIMENT ANALYSIS TEXT CLASSIFICATION TRANSFER LEARNING

Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning

ICLR 2018 facebookresearch/InferSent

A lot of the recent success in natural language processing (NLP) has been driven by distributed vector representations of words trained on large amounts of text in an unsupervised manner. In this work, we present a simple, effective multi-task learning framework for sentence representations that combines the inductive biases of diverse training objectives in a single model.

MULTI-TASK LEARNING NATURAL LANGUAGE INFERENCE PARAPHRASE IDENTIFICATION SEMANTIC TEXTUAL SIMILARITY

Universal Sentence Encoder

29 Mar 2018facebookresearch/InferSent

For both variants, we investigate and report the relationship between model complexity, resource consumption, the availability of transfer task training data, and task performance. We find that transfer learning using sentence embeddings tends to outperform word level transfer.

SEMANTIC TEXTUAL SIMILARITY SENTENCE EMBEDDINGS SENTIMENT ANALYSIS SUBJECTIVITY ANALYSIS TEXT CLASSIFICATION TRANSFER LEARNING WORD EMBEDDINGS

Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

EMNLP 2017 facebookresearch/InferSent

Many modern NLP systems rely on word embeddings, previously trained in an unsupervised manner on large corpora, as base features. Efforts to obtain embeddings for larger chunks of text, such as sentences, have however not been so successful.

CROSS-LINGUAL NATURAL LANGUAGE INFERENCE SEMANTIC TEXTUAL SIMILARITY TRANSFER LEARNING WORD EMBEDDINGS

Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks

18 Mar 2017jiesutd/NCRFpp

Recent papers have shown that neural networks obtain state-of-the-art performance on several different sequence tagging tasks. One appealing property of such systems is their generality, as excellent performance can be achieved with a unified architecture and without task-specific feature engineering.

NAMED ENTITY RECOGNITION PART-OF-SPEECH TAGGING TRANSFER LEARNING