Search Results for author: Timo Schick

Found 14 papers, 12 papers with code

Few-Shot Text Generation with Natural Language Instructions

no code implementations EMNLP 2021 Timo Schick, Hinrich Schütze

Providing pretrained language models with simple task descriptions in natural language enables them to solve some tasks in a fully unsupervised fashion.

Text Classification Text Generation

Generating Datasets with Pretrained Language Models

1 code implementation EMNLP 2021 Timo Schick, Hinrich Schütze

To obtain high-quality sentence embeddings from pretrained language models (PLMs), they must either be augmented with additional pretraining objectives or finetuned on a large set of labeled text pairs.

Semantic Textual Similarity Sentence Embeddings

Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference

1 code implementation EACL 2021 Timo Schick, Hinrich Sch{\"u}tze

Some NLP tasks can be solved in a fully unsupervised fashion by providing a pretrained language model with {``}task descriptions{''} in natural language (e. g., Radford et al., 2019).

Few-Shot Text Classification Language Modelling +2

Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP

1 code implementation28 Feb 2021 Timo Schick, Sahana Udupa, Hinrich Schütze

In this paper, we first demonstrate a surprising finding: pretrained language models recognize, to a considerable degree, their undesirable biases and the toxicity of the content they produce.

Language Modelling

Few-Shot Text Generation with Pattern-Exploiting Training

2 code implementations22 Dec 2020 Timo Schick, Hinrich Schütze

Providing pretrained language models with simple task descriptions in natural language enables them to solve some tasks in a fully unsupervised fashion.

Text Classification Text Generation +1

Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification

1 code implementation COLING 2020 Timo Schick, Helmut Schmid, Hinrich Schütze

A recent approach for few-shot text classification is to convert textual inputs to cloze questions that contain some form of task description, process them with a pretrained language model and map the predicted words to labels.

Few-Shot Text Classification General Classification +2

It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners

4 code implementations NAACL 2021 Timo Schick, Hinrich Schütze

When scaled to hundreds of billions of parameters, pretrained language models such as GPT-3 (Brown et al., 2020) achieve remarkable few-shot performance.

Natural Language Understanding

Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference

4 code implementations21 Jan 2020 Timo Schick, Hinrich Schütze

Some NLP tasks can be solved in a fully unsupervised fashion by providing a pretrained language model with "task descriptions" in natural language (e. g., Radford et al., 2019).

Few-Shot Text Classification General Classification +3

BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance

1 code implementation ACL 2020 Timo Schick, Hinrich Schütze

In this work, we transfer this idea to pretrained language models: We introduce BERTRAM, a powerful architecture based on BERT that is capable of inferring high-quality embeddings for rare words that are suitable as input representations for deep language models.

Language Modelling Word Embeddings

Rare Words: A Major Problem for Contextualized Embeddings And How to Fix it by Attentive Mimicking

2 code implementations14 Apr 2019 Timo Schick, Hinrich Schütze

Pretraining deep neural network architectures with a language modeling objective has brought large improvements for many natural language processing tasks.

Language Modelling

Attentive Mimicking: Better Word Embeddings by Attending to Informative Contexts

1 code implementation NAACL 2019 Timo Schick, Hinrich Schütze

Learning high-quality embeddings for rare words is a hard problem because of sparse context information.

Word Embeddings

Learning Semantic Representations for Novel Words: Leveraging Both Form and Context

1 code implementation9 Nov 2018 Timo Schick, Hinrich Schütze

The general problem setting is that word embeddings are induced on an unlabeled training corpus and then a model is trained that embeds novel words into this induced embedding space.

Learning Semantic Representations Word Embeddings

Transition-Based Generation from Abstract Meaning Representations

1 code implementation24 Jul 2017 Timo Schick

This work addresses the task of generating English sentences from Abstract Meaning Representation (AMR) graphs.

Language Modelling

Cannot find the paper you are looking for? You can Submit a new open access paper.