Search Results for author: Jonas Pfeiffer

Found 32 papers, 17 papers with code

MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer

no code implementations Findings (EMNLP) 2021 Alan Ansell, Edoardo Maria Ponti, Jonas Pfeiffer, Sebastian Ruder, Goran Glavaš, Ivan Vulić, Anna Korhonen

While offering (1) improved fine-tuning efficiency (by a factor of around 50 in our experiments), (2) a smaller parameter budget, and (3) increased language coverage, MAD-G remains competitive with more expensive methods for language-specific adapter training across the board.

Dependency Parsing named-entity-recognition +4

TUDa at WMT21: Sentence-Level Direct Assessment with Adapters

no code implementations WMT (EMNLP) 2021 Gregor Geigle, Jonas Stadtmüller, Wei Zhao, Jonas Pfeiffer, Steffen Eger

This paper presents our submissions to the WMT2021 Shared Task on Quality Estimation, Task 1 Sentence-Level Direct Assessment.

CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models

1 code implementation23 May 2023 Benjamin Minixhofer, Jonas Pfeiffer, Ivan Vulić

We first address the data gap by introducing a dataset of 255k compound and non-compound words across 56 diverse languages obtained from Wiktionary.

mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations

no code implementations23 May 2023 Jonas Pfeiffer, Francesco Piccinno, Massimo Nicosia, Xinyi Wang, Machel Reid, Sebastian Ruder

Multilingual sequence-to-sequence models perform poorly with increased language coverage and fail to consistently generate text in the correct target language in few-shot settings.

Natural Language Understanding

Romanization-based Large-scale Adaptation of Multilingual Language Models

no code implementations18 Apr 2023 Sukannya Purkayastha, Sebastian Ruder, Jonas Pfeiffer, Iryna Gurevych, Ivan Vulić

In order to boost the capacity of mPLMs to deal with low-resource and unseen languages, we explore the potential of leveraging transliteration on a massive scale.

Cross-Lingual Transfer Transliteration

Modular Deep Learning

no code implementations22 Feb 2023 Jonas Pfeiffer, Sebastian Ruder, Ivan Vulić, Edoardo Maria Ponti

Modular deep learning has emerged as a promising solution to these challenges.

Causal Inference Transfer Learning

Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing

no code implementations13 Jan 2023 Chen Cecilia Liu, Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych

Our in-depth experiments reveal that scheduled unfreezing induces different learning dynamics compared to standard fine-tuning, and provide evidence that the dynamics of Fisher Information during training correlate with cross-lingual generalization performance.

Cross-Lingual Transfer Transfer Learning

One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks

no code implementations12 Oct 2022 Gregor Geigle, Chen Cecilia Liu, Jonas Pfeiffer, Iryna Gurevych

While many VEs -- of different architectures, trained on different data and objectives -- are publicly available, they are not designed for the downstream V+L tasks.

Lifting the Curse of Multilinguality by Pre-training Modular Transformers

no code implementations NAACL 2022 Jonas Pfeiffer, Naman Goyal, Xi Victoria Lin, Xian Li, James Cross, Sebastian Riedel, Mikel Artetxe

Multilingual pre-trained models are known to suffer from the curse of multilinguality, which causes per-language performance to drop as they cover more languages.

named-entity-recognition Named Entity Recognition +3

UKP-SQUARE: An Online Platform for Question Answering Research

1 code implementation ACL 2022 Tim Baumgärtner, Kexin Wang, Rachneet Sachdeva, Max Eichler, Gregor Geigle, Clifton Poth, Hannah Sterz, Haritz Puerto, Leonardo F. R. Ribeiro, Jonas Pfeiffer, Nils Reimers, Gözde Gül Şahin, Iryna Gurevych

Recent advances in NLP and information retrieval have given rise to a diverse set of question answering tasks that are of different formats (e. g., extractive, abstractive), require different model architectures (e. g., generative, discriminative), and setups (e. g., with or without retrieval).

Explainable Models Information Retrieval +2

Delving Deeper into Cross-lingual Visual Question Answering

1 code implementation15 Feb 2022 Chen Liu, Jonas Pfeiffer, Anna Korhonen, Ivan Vulić, Iryna Gurevych

2) We analyze cross-lingual VQA across different question types of varying complexity for different multilingual multimodal Transformers, and identify question types that are the most difficult to improve on.

Inductive Bias Question Answering +1

IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages

2 code implementations27 Jan 2022 Emanuele Bugliarello, Fangyu Liu, Jonas Pfeiffer, Siva Reddy, Desmond Elliott, Edoardo Maria Ponti, Ivan Vulić

Our benchmark enables the evaluation of multilingual multimodal models for transfer learning, not only in a zero-shot setting, but also in newly defined few-shot learning setups.

Cross-Modal Retrieval Few-Shot Learning +5

TxT: Crossmodal End-to-End Learning with Transformers

no code implementations9 Sep 2021 Jan-Martin O. Steitz, Jonas Pfeiffer, Iryna Gurevych, Stefan Roth

Reasoning over multiple modalities, e. g. in Visual Question Answering (VQA), requires an alignment of semantic concepts across domains.

Question Answering Visual Question Answering

AdapterHub Playground: Simple and Flexible Few-Shot Learning with Adapters

1 code implementation ACL 2022 Tilman Beck, Bela Bohlender, Christina Viehmann, Vincent Hane, Yanik Adamson, Jaber Khuri, Jonas Brossmann, Jonas Pfeiffer, Iryna Gurevych

The open-access dissemination of pretrained language models through online repositories has led to a democratization of state-of-the-art natural language processing (NLP) research.

Few-Shot Learning Transfer Learning

What to Pre-Train on? Efficient Intermediate Task Selection

1 code implementation EMNLP 2021 Clifton Poth, Jonas Pfeiffer, Andreas Rücklé, Iryna Gurevych

Our best methods achieve an average Regret@3 of less than 1% across all target tasks, demonstrating that we are able to efficiently identify the best datasets for intermediate training.

Multiple-choice Question Answering +1

Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval

1 code implementation22 Mar 2021 Gregor Geigle, Jonas Pfeiffer, Nils Reimers, Ivan Vulić, Iryna Gurevych

Current state-of-the-art approaches to cross-modal retrieval process text and visual input jointly, relying on Transformer-based architectures with cross-attention mechanisms that attend over all words and objects in an image.

Cross-Modal Retrieval Retrieval

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

1 code implementation ACL 2021 Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, Iryna Gurevych

In this work, we provide a systematic and comprehensive empirical comparison of pretrained multilingual language models versus their monolingual counterparts with regard to their monolingual task performance.

Pretrained Multilingual Language Models

UNKs Everywhere: Adapting Multilingual Language Models to New Scripts

2 code implementations EMNLP 2021 Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, Sebastian Ruder

The ultimate challenge is dealing with under-resourced languages not covered at all by the models and written in scripts unseen during pretraining.

Cross-Lingual Transfer

AdapterDrop: On the Efficiency of Adapters in Transformers

1 code implementation EMNLP 2021 Andreas Rücklé, Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, Iryna Gurevych

Massively pre-trained transformer models are computationally expensive to fine-tune, slow for inference, and have large storage requirements.

MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale

1 code implementation EMNLP 2020 Andreas Rücklé, Jonas Pfeiffer, Iryna Gurevych

We investigate the model performances on nine benchmarks of answer selection and question similarity tasks, and show that all 140 models transfer surprisingly well, where the large majority of models substantially outperforms common IR baselines.

Answer Selection Community Question Answering +3

AdapterHub: A Framework for Adapting Transformers

5 code implementations EMNLP 2020 Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, Iryna Gurevych

We propose AdapterHub, a framework that allows dynamic "stitching-in" of pre-trained adapters for different tasks and languages.


AdapterFusion: Non-Destructive Task Composition for Transfer Learning

3 code implementations EACL 2021 Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, Iryna Gurevych

We show that by separating the two stages, i. e., knowledge extraction and knowledge composition, the classifier can effectively exploit the representations learned from multiple tasks in a non-destructive manner.

Language Modelling Multi-Task Learning

Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic Conditional Random Fields

no code implementations1 May 2020 Jonas Pfeiffer, Edwin Simpson, Iryna Gurevych

We compare different models for low resource multi-task sequence tagging that leverage dependencies between label sequences for different tasks.

Multi-Task Learning

MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer

3 code implementations EMNLP 2020 Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, Sebastian Ruder

The main goal behind state-of-the-art pre-trained multilingual models such as multilingual BERT and XLM-R is enabling and bootstrapping NLP applications in low-resource languages through zero-shot or few-shot cross-lingual transfer.

Ranked #5 on Cross-Lingual Transfer on XCOPA (using extra training data)

Cross-Lingual Transfer named-entity-recognition +4

What do Deep Networks Like to Read?

no code implementations10 Sep 2019 Jonas Pfeiffer, Aishwarya Kamath, Iryna Gurevych, Sebastian Ruder

Recent research towards understanding neural networks probes models in a top-down manner, but is only able to identify model tendencies that are known a priori.

FAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning

no code implementations IJCNLP 2019 Jonas Pfeiffer, Christian M. Meyer, Claudia Schulz, Jan Kiesewetter, Jan Zottmann, Michael Sailer, Elisabeth Bauer, Frank Fischer, Martin R. Fischer, Iryna Gurevych

Our proposed system FAMULUS helps students learn to diagnose based on automatic feedback in virtual patient simulations, and it supports instructors in labeling training data.


Specializing Distributional Vectors of All Words for Lexical Entailment

no code implementations WS 2019 Aishwarya Kamath, Jonas Pfeiffer, Edoardo Maria Ponti, Goran Glava{\v{s}}, Ivan Vuli{\'c}

Semantic specialization methods fine-tune distributional word vectors using lexical knowledge from external resources (e. g. WordNet) to accentuate a particular relation between words.

Cross-Lingual Transfer Lexical Entailment +2

Cannot find the paper you are looking for? You can Submit a new open access paper.