Search Results for author:
Found 34 papers, 19 papers with code
AdapterFusion: Non-Destructive Task Composition for Transfer Learning
We show that by separating the two stages, i. e., knowledge extraction and knowledge composition, the classifier can effectively exploit the representations learned from multiple tasks in a non-destructive manner.
MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer
The main goal behind state-of-the-art pre-trained multilingual models such as multilingual BERT and XLM-R is enabling and bootstrapping NLP applications in low-resource languages through zero-shot or few-shot cross-lingual transfer.
Ranked #5 on
Cross-Lingual Transfer
on XCOPA
(using extra training data)
AdapterHub: A Framework for Adapting Transformers
We propose AdapterHub, a framework that allows dynamic "stitching-in" of pre-trained adapters for different tasks and languages.
AdapterDrop: On the Efficiency of Adapters in Transformers
Massively pre-trained transformer models are computationally expensive to fine-tune, slow for inference, and have large storage requirements.
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning
We introduce Adapters, an open-source library that unifies parameter-efficient and modular transfer learning in large language models.
Where's the Point? Self-Supervised Multilingual Punctuation-Agnostic Sentence Segmentation
Many NLP pipelines split text into sentences as one of the crucial preprocessing steps.
Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval
Current state-of-the-art approaches to cross-modal retrieval process text and visual input jointly, relying on Transformer-based architectures with cross-attention mechanisms that attend over all words and objects in an image.
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages
Our benchmark enables the evaluation of multilingual multimodal models for transfer learning, not only in a zero-shot setting, but also in newly defined few-shot learning setups.
UKP-SQUARE: An Online Platform for Question Answering Research
Recent advances in NLP and information retrieval have given rise to a diverse set of question answering tasks that are of different formats (e. g., extractive, abstractive), require different model architectures (e. g., generative, discriminative), and setups (e. g., with or without retrieval).
What to Pre-Train on? Efficient Intermediate Task Selection
Our best methods achieve an average Regret@3 of less than 1% across all target tasks, demonstrating that we are able to efficiently identify the best datasets for intermediate training.
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models
In this work, we provide a systematic and comprehensive empirical comparison of pretrained multilingual language models versus their monolingual counterparts with regard to their monolingual task performance.
xGQA: Cross-Lingual Visual Question Answering
In this work, we address this gap and provide xGQA, a new multilingual evaluation benchmark for the visual question answering task.
MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale
We investigate the model performances on nine benchmarks of answer selection and question similarity tasks, and show that all 140 models transfer surprisingly well, where the large majority of models substantially outperforms common IR baselines.
AdapterHub Playground: Simple and Flexible Few-Shot Learning with Adapters
The open-access dissemination of pretrained language models through online repositories has led to a democratization of state-of-the-art natural language processing (NLP) research.
Smelting Gold and Silver for Improved Multilingual AMR-to-Text Generation
Recent work on multilingual AMR-to-text generation has exclusively focused on data augmentation strategies that utilize silver AMR.
CompoundPiece: Evaluating and Improving Decompounding Performance of Language Models
We first address the data gap by introducing a dataset of 255k compound and non-compound words across 56 diverse languages obtained from Wiktionary.
UNKs Everywhere: Adapting Multilingual Language Models to New Scripts
The ultimate challenge is dealing with under-resourced languages not covered at all by the models and written in scripts unseen during pretraining.
Delving Deeper into Cross-lingual Visual Question Answering
2) We analyze cross-lingual VQA across different question types of varying complexity for different multilingual multimodal Transformers, and identify question types that are the most difficult to improve on.
A Neural Autoencoder Approach for Document Ranking and Query Refinement in Pharmacogenomic Information Retrieval
In this study, we investigate learning-to-rank and query refinement approaches for information retrieval in the pharmacogenomic domain.
Specializing Distributional Vectors of All Words for Lexical Entailment
Semantic specialization methods fine-tune distributional word vectors using lexical knowledge from external resources (e. g. WordNet) to accentuate a particular relation between words.
FAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning
Our proposed system FAMULUS helps students learn to diagnose based on automatic feedback in virtual patient simulations, and it supports instructors in labeling training data.
What do Deep Networks Like to Read?
Recent research towards understanding neural networks probes models in a top-down manner, but is only able to identify model tendencies that are known a priori.
Fine-Tuned Neural Models for Propaganda Detection at the Sentence and Fragment levels
This paper presents the CUNLP submission for the NLP4IF 2019 shared-task on FineGrained Propaganda Detection.
Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic Conditional Random Fields
We compare different models for low resource multi-task sequence tagging that leverage dependencies between label sequences for different tasks.
TxT: Crossmodal End-to-End Learning with Transformers
Reasoning over multiple modalities, e. g. in Visual Question Answering (VQA), requires an alignment of semantic concepts across domains.
MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer
While offering (1) improved fine-tuning efficiency (by a factor of around 50 in our experiments), (2) a smaller parameter budget, and (3) increased language coverage, MAD-G remains competitive with more expensive methods for language-specific adapter training across the board.
TUDa at WMT21: Sentence-Level Direct Assessment with Adapters
This paper presents our submissions to the WMT2021 Shared Task on Quality Estimation, Task 1 Sentence-Level Direct Assessment.
Lifting the Curse of Multilinguality by Pre-training Modular Transformers
Multilingual pre-trained models are known to suffer from the curse of multilinguality, which causes per-language performance to drop as they cover more languages.
One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks
While many VEs -- of different architectures, trained on different data and objectives -- are publicly available, they are not designed for the downstream V+L tasks.
FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing
Our experiments reveal that scheduled unfreezing induces different learning dynamics compared to standard fine-tuning, and provide evidence that the dynamics of Fisher Information during training correlate with cross-lingual generalization performance.
Modular Deep Learning
Modular deep learning has emerged as a promising solution to these challenges.
Romanization-based Large-scale Adaptation of Multilingual Language Models
In order to boost the capacity of mPLMs to deal with low-resource and unseen languages, we explore the potential of leveraging transliteration on a massive scale.
mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations
Multilingual sequence-to-sequence models perform poorly with increased language coverage and fail to consistently generate text in the correct target language in few-shot settings.
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization
Parameter-efficient fine-tuning (PEFT) using labeled task data can significantly improve the performance of large language models (LLMs) on the downstream task.
