Search Results for author:
Found 82 papers, 57 papers with code
DS-TOD: Efficient Domain Specialization for Task-Oriented Dialog
Recent work has shown that self-supervised dialog-specific pretraining on large conversational datasets yields substantial gains over traditional language modeling (LM) pretraining in downstream task-oriented dialog (TOD).
BAD-X: Bilingual Adapters Improve Zero-Shot Cross-Lingual Transfer
Adapter modules enable modular and efficient zero-shot cross-lingual transfer, where current state-of-the-art adapter-based approaches learn specialized language adapters (LAs) for individual languages.
Natural Language Processing for Multilingual Task-Oriented Dialogue
In this tutorial, we will thus discuss and demonstrate the importance of (building) multilingual ToD systems, and then provide a systematic overview of current research gaps, challenges and initiatives related to multilingual ToD systems, with a particular focus on their connections to current research and challenges in multilingual and low-resource NLP.
Training and Domain Adaptation for Supervised Text Segmentation
In this work, we focus on the domain transfer performance of supervised neural text segmentation in the educational domain.
MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer
While offering (1) improved fine-tuning efficiency (by a factor of around 50 in our experiments), (2) a smaller parameter budget, and (3) increased language coverage, MAD-G remains competitive with more expensive methods for language-specific adapter training across the board.
ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding
Language models (LMs) have become a staple of the code-writing toolbox.
Don't Stop the Multi-Party! On Generating Synthetic Multi-Party Conversations with Constraints
Multi-Party Conversations (MPCs) are widely studied across disciplines, with social media as a primary data source due to their accessibility.
How Much Do LLMs Hallucinate across Languages? On Multilingual Estimation of LLM Hallucination in the Wild
In the age of misinformation, hallucination -- the tendency of Large Language Models (LLMs) to generate non-factual or unfaithful responses -- represents the main risk for their global utility.
MVL-SIB: A Massively Multilingual Vision-Language Benchmark for Cross-Modal Topical Matching
By correlating performance on MVL-SIB with other multilingual VL benchmarks, we highlight that MVL-SIB serves as a comprehensive probe of multilingual VL understanding in LVLMs.
Fleurs-SLU: A Massively Multilingual Benchmark for Spoken Language Understanding
While recent multilingual automatic speech recognition models claim to support thousands of languages, ASR for low-resource languages remains highly unreliable due to limited bimodal speech and text training data.
Centurio: On Drivers of Multilingual Ability of Large Vision-Language Model
Most Large Vision-Language Models (LVLMs) to date are trained predominantly on English data, which makes them struggle to understand non-English input and fail to generate output in the desired target language.
Peeling Back the Layers: An In-Depth Evaluation of Encoder Architectures in Neural News Recommenders
In this work, we perform a comprehensive analysis of encoder architectures in neural news recommender systems.
SpeechTaxi: On Multilingual Semantic Speech Classification
Recent advancements in multilingual speech encoding as well as transcription raise the question of the most effective approach to semantic speech classification.
Problem Solving Through Human-AI Preference-Based Cooperation
While there is a widespread belief that artificial general intelligence (AGI) -- or even superhuman AI -- is imminent, complex problems in expert domains are far from being solved.
Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment
Another limitation of multilingual sentence encoders is the trade-off between monolingual and cross-lingual performance.
ANHALTEN: Cross-Lingual Transfer for German Token-Level Reference-Free Hallucination Detection
Research on token-level reference-free hallucination detection has predominantly focused on English, primarily due to the scarcity of robust datasets in other languages.
Evaluating the Ability of LLMs to Solve Semantics-Aware Process Mining Tasks
This latter property suggests that LLMs could also be used to tackle process mining tasks that benefit from an understanding of process behavior.
Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models?
Large vision-language models (LVLMs) have recently dramatically pushed the state of the art in image captioning and many image understanding tasks (e. g., visual question answering).
African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classification
We benchmark 12 public LVLMs on \texttt{FOCI} and show that it tests for a \textit{complementary skill} to established image understanding and reasoning benchmarks.
Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages
MT encoders, however, lack the knowledge necessary for comprehensive NLU that LLMs obtain through language modeling training on immense corpora.
News Without Borders: Domain Adaptation of Multilingual Sentence Embeddings for Cross-lingual News Recommendation
Rapidly growing numbers of multilingual news consumers pose an increasing challenge to news recommender systems in terms of providing customized recommendations.
Knowledge Distillation vs. Pretraining from Scratch under a Fixed (Computation) Budget
We further find that KD yields larger gains over pretraining from scratch when the data must be repeated under the fixed computation budget.
MIND Your Language: A Multilingual Dataset for Cross-lingual News Recommendation
Our findings reveal that (i) current NNRs, even when based on a multilingual language model, suffer from substantial performance losses under ZS-XLT and that (ii) inclusion of target-language data in FS-XLT training has limited benefits, particularly when combined with a bilingual news consumption.
IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generators
In particular, most mainstream Code-LMs have been pre-trained on source code files alone.
On Task Performance and Model Calibration with Supervised and Self-Ensembled In-Context Learning
Following the standard supervised fine-tuning (SFT) paradigm, in-context learning (ICL) has become an efficient approach propelled by the recent advancements in large language models (LLMs), yielding promising performance across various tasks in few-shot data setups.
SQATIN: Supervised Instruction Tuning Meets Question Answering for Improved Dialogue NLU
Task-oriented dialogue (ToD) systems help users execute well-defined tasks across a variety of domains (e. g., $\textit{flight booking}$ or $\textit{food ordering}$), with their Natural Language Understanding (NLU) components being dedicated to the analysis of user utterances, predicting users' intents ($\textit{Intent Detection}$, ID) and extracting values for informational slots ($\textit{Value Extraction}$, VE).
To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages
Given, on the one hand, the large body of work on improving XLT with mLMs and, on the other hand, recent advances in massively multilingual MT, in this work, we systematically evaluate existing and propose new translation-based XLT approaches for transfer to low-resource languages.
Vicinal Risk Minimization for Few-Shot Cross-lingual Transfer in Abusive Language Detection
In this work, we resort to data augmentation and continual pre-training for domain adaptation to improve cross-lingual abusive language detection.
AdaSent: Efficient Domain-Adapted Sentence Embeddings for Few-Shot Classification
As a solution, we propose AdaSent, which decouples SEPT from DAPT by training a SEPT adapter on the base PLM.
Linking Surface Facts to Large-Scale Knowledge Graphs
Open Information Extraction (OIE) methods extract facts from natural language text in the form of ("subject"; "relation"; "object") triples.
One For All & All For One: Bypassing Hyperparameter Tuning with Model Averaging For Cross-Lingual Transfer
Because of this, model selection based on source-language validation is unreliable: it picks model snapshots with suboptimal target-language performance.
NewsRecLib: A PyTorch-Lightning Library for Neural News Recommendation
NewsRecLib is an open-source library based on Pytorch-Lightning and Hydra developed for training and evaluating neural news recommendation models.
Train Once, Use Flexibly: A Modular Framework for Multi-Aspect Neural News Recommendation
Recent neural news recommenders (NNRs) extend content-based recommendation (1) by aligning additional aspects (e. g., topic, sentiment) between candidate news and user history or (2) by diversifying recommendations w. r. t.
mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMs
Modular vision-language models (Vision-LLMs) align pretrained image encoders with (frozen) large language models (LLMs) and post-hoc condition LLMs to `understand' the image input.
Babel-ImageNet: Massively Multilingual Evaluation of Vision-and-Language Representations
Vision-and-language (VL) models with separate encoders for each modality (e. g., CLIP) have become the go-to models for zero-shot image classification and image-text retrieval.
Free Lunch: Robust Cross-Lingual Transfer via Model Checkpoint Averaging
The results indicate that averaging model checkpoints yields systematic and consistent performance gains across diverse target languages in all tasks.
Leveraging Open Information Extraction for More Robust Domain Transfer of Event Trigger Detection
We address the problem of negative transfer in TD by coupling triggers between domains using subject-object relations obtained from a rule-based open information extraction (OIE) system.
A General-Purpose Multilingual Document Encoder
While a large body of work leveraged MMTs to mine parallel data and induce bilingual document embeddings, much less effort has been devoted to training general-purpose (massively) multilingual document encoder that can be used for both supervised and unsupervised document-level tasks.
Transfer to a Low-Resource Language via Close Relatives: The Case Study on Faroese
Multilingual language models have pushed state-of-the-art in cross-lingual NLP transfer.
Simplifying Content-Based Neural News Recommendation: On User Modeling and Training Objectives
Most neural news recommenders rely on user click behavior and typically introduce dedicated user encoders that aggregate the content of clicked news into user embeddings (early fusion).
Can Demographic Factors Improve Text Classification? Revisiting Demographic Adaptation in the Age of Transformers
Previous work showed that incorporating demographic factors can consistently improve performance for various NLP tasks with traditional NLP models.
SLICER: Sliced Fine-Tuning for Low-Resource Cross-Lingual Transfer for Named Entity Recognition
Large multilingual language models generally demonstrate impressive results in zero-shot cross-lingual transfer, yet often fail to successfully transfer to low-resource languages, even for token-level prediction tasks like named entity recognition (NER).
Multilingual text classification
named-entity-recognition
+3
On the Limitations of Sociodemographic Adaptation with Transformers
We adapt the language representations for the sociodemographic dimensions of gender and age, using continuous language modeling and dynamic multi-task learning for adaptation, where we couple language modeling with the prediction of a sociodemographic class.
Massively Multilingual Lexical Specialization of Multilingual Transformers
While pretrained language models (PLMs) primarily serve as general-purpose text encoders that can be fine-tuned for a wide variety of downstream tasks, recent work has shown that they can also be rewired to produce high-quality word representations (i. e., static word embeddings) and yield good performance in type-level lexical tasks.
ZusammenQA: Data Augmentation with Specialized Models for Cross-lingual Open-retrieval Question Answering System
This paper introduces our proposed system for the MIA Shared Task on Cross-lingual Open-retrieval Question Answering (COQA).
Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog
We then introduce a new framework for multilingual conversational specialization of pretrained language models (PrLMs) that aims to facilitate cross-lingual transfer for arbitrary downstream TOD tasks.
Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders
In this work, we probe SEs for the amount of cross-lingual lexical knowledge stored in their parameters, and compare them against the original multilingual LMs.
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval
Current approaches therefore commonly transfer rankers trained on English data to other languages and cross-lingual setups by means of multilingual encoders: they fine-tune all parameters of pretrained massively multilingual Transformers (MMTs, e. g., multilingual BERT) on English relevance judgments, and then deploy them in the target language(s).
Geographic Adaptation of Pretrained Language Models
While pretrained language models (PLMs) have been shown to possess a plethora of linguistic knowledge, the existing body of research has largely neglected extralinguistic knowledge, which is generally difficult to obtain by pretraining on text alone.
On Cross-Lingual Retrieval with Multilingual Text Encoders
In this work we present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks across a number of diverse language pairs.
DS-TOD: Efficient Domain Specialization for Task Oriented Dialog
Recent work has shown that self-supervised dialog-specific pretraining on large conversational datasets yields substantial gains over traditional language modeling (LM) pretraining in downstream task-oriented dialog (TOD).
AnnIE: An Annotation Platform for Constructing Complete Open Information Extraction Benchmark
Open Information Extraction (OIE) is the task of extracting facts from sentences in the form of relations and their corresponding arguments in schema-free manner.
BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation
In this work, we introduce BenchIE: a benchmark and evaluation framework for comprehensive evaluation of OIE systems for English, Chinese, and German.
Ranked #1 on
Open Information Extraction
on BenchIE
Sustainable Modular Debiasing of Language Models
Unfair stereotypical biases (e. g., gender, racial, or religious biases) encoded in modern pretrained language models (PLMs) have negative ethical implications for widespread adoption of state-of-the-art language technology.
Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biases
We analyze bias in historical corpora as encoded in diachronic distributional semantic models by focusing on two specific forms of bias, namely a political (i. e., anti-communism) and racist (i. e., antisemitism) one.
Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentation
Despite extensive research efforts in recent years, computational argumentation (CA) remains one of the most challenging areas of natural language processing.
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models
We use the evaluation framework to benchmark the widely used conversational DialoGPT model along with the adaptations of four debiasing methods.
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems
We find that the most critical factor preventing the creation of truly multilingual ToD systems is the lack of datasets in most languages for both training and evaluation.
DebIE: A Platform for Implicit and Explicit Debiasing of Word Embedding Spaces
In this work, we present DebIE, the first integrated platform for (1) measuring and (2) mitigating bias in word embeddings.
Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval
Therefore, in this work we present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks across a large number of language pairs.
Verb Knowledge Injection for Multilingual Event Processing
We investigate whether injecting explicit information on verbs' semantic-syntactic behaviour improves the performance of LM-pretrained Transformers in event extraction tasks -- downstream tasks for which accurate verb processing is paramount.
Self-Supervised Learning for Visual Summary Identification in Scientific Publications
Providing visual summaries of scientific publications can increase information access for readers and thereby help deal with the exponential growth in the number of scientific publications.
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer
Adapter modules, additional trainable parameters that enable efficient fine-tuning of pretrained transformers, have recently been used for language specialization of multilingual transformers, improving downstream zero-shot cross-lingual transfer.
AraWEAT: Multidimensional Analysis of Biases in Arabic Word Embeddings
Our analysis yields several interesting findings, e. g., that implicit gender bias in embeddings trained on Arabic news corpora steadily increases over time (between 2007 and 2017).
Probing Pretrained Language Models for Lexical Semantics
The success of large pretrained language models (LMs) such as BERT and RoBERTa has sparked interest in probing their representations, in order to unveil what types of knowledge they implicitly capture.
Is Supervised Syntactic Parsing Beneficial for Language Understanding? An Empirical Investigation
Traditional NLP has long held (supervised) syntactic parsing necessary for successful higher-level semantic language understanding (LU).
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers
Following the major success of neural language models (LMs) such as BERT or GPT-2 on a variety of language understanding tasks, recent work focused on injecting (structured) knowledge from external resources into these models.
On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation
We systematically investigate a range of metrics based on state-of-the-art cross-lingual semantic representations obtained with pretrained M-BERT and LASER.
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning
In order to simulate human language capacity, natural language processing systems must be able to reason about the dynamics of everyday situations, including their possible causes and effects.
Ranked #3 on
Cross-Lingual Transfer
on XCOPA
(using extra training data)
From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers
Massively multilingual transformers pretrained with language modeling objectives (e. g., mBERT, XLM-R) have become a de facto default transfer paradigm for zero-shot cross-lingual transfer in NLP, offering unmatched transfer performance.
Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers
Current methods of cross-lingual parser transfer focus on predicting the best parser for a low-resource target language globally, that is, "at treebank level".
Windowing Models for Abstractive Summarization of Long Texts
Neural summarization models suffer from the fixed-size input limitation: if text length surpasses the model's maximal number of input tokens, some document content (possibly summary-relevant) gets truncated Independently summarizing windows of maximal input size disallows for information flow between windows and leads to incoherent summaries.
Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation
Breaking down the structure of long texts into semantically coherent segments makes the texts more readable and supports downstream applications like summarization and retrieval.
A General Framework for Implicit and Explicit Debiasing of Distributional Word Vector Spaces
Moreover, we successfully transfer debiasing models, by means of cross-lingual embedding spaces, and remove or attenuate biases in distributional word vector spaces of languages that lack readily available bias specifications.
Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity
In this work, we complement such distributional knowledge with external lexical knowledge, that is, we integrate the discrete knowledge on word-level semantic similarity into pretraining.
Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?
A series of bilingual lexicon induction (BLI) experiments with 15 diverse languages (210 language pairs) show that fully unsupervised CLWE methods still fail for a large number of language pairs (e. g., they yield zero BLI performance for 87/210 pairs).
Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors
In this work, we present a systematic study of biases encoded in distributional word vector spaces: we analyze how consistent the bias effects are across languages, corpora, and embedding models.
Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specialization
Our adversarial post-specialization method propagates the external lexical knowledge to the full distributional space.
Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources
Word vector specialisation (also known as retrofitting) is a portable, light-weight approach to fine-tuning arbitrary distributional word vector spaces by injecting external knowledge from rich lexical resources such as WordNet.
Unsupervised Cross-Lingual Information Retrieval using Monolingual Data Only
We propose a fully unsupervised framework for ad-hoc cross-lingual information retrieval (CLIR) which requires no bilingual data at all.
A Resource-Light Method for Cross-Lingual Semantic Textual Similarity
In contrast, we propose an unsupervised and a very resource-light approach for measuring semantic similarity between texts in different languages.
Cross-Lingual Information Retrieval
Cross-Lingual Semantic Textual Similarity
+9
