Search Results for author:
Found 41 papers, 24 papers with code
Training and Domain Adaptation for Supervised Text Segmentation
In this work, we focus on the domain transfer performance of supervised neural text segmentation in the educational domain.
Natural Language Processing for Multilingual Task-Oriented Dialogue
In this tutorial, we will thus discuss and demonstrate the importance of (building) multilingual ToD systems, and then provide a systematic overview of current research gaps, challenges and initiatives related to multilingual ToD systems, with a particular focus on their connections to current research and challenges in multilingual and low-resource NLP.
DS-TOD: Efficient Domain Specialization for Task-Oriented Dialog
Recent work has shown that self-supervised dialog-specific pretraining on large conversational datasets yields substantial gains over traditional language modeling (LM) pretraining in downstream task-oriented dialog (TOD).
MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer
While offering (1) improved fine-tuning efficiency (by a factor of around 50 in our experiments), (2) a smaller parameter budget, and (3) increased language coverage, MAD-G remains competitive with more expensive methods for language-specific adapter training across the board.
Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog
We then introduce a new framework for multilingual conversational specialization of pretrained language models (PrLMs) that aims to facilitate cross-lingual transfer for arbitrary downstream TOD tasks.
Exposing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders
Pretrained multilingual language models (LMs) can be successfully transformed into multilingual sentence encoders (SEs; e. g., LaBSE, xMPNET) via additional fine-tuning or model distillation on parallel data.
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval
Current approaches therefore typically transfer rankers trained on English data to other languages and cross-lingual setups by means of multilingual encoders: they fine-tune all the parameters of a pretrained massively multilingual Transformer (MMT, e. g., multilingual BERT) on English relevance judgments and then deploy it in the target language.
Geographic Adaptation of Pretrained Language Models
Geographic linguistic features are commonly used to improve the performance of pretrained language models (PLMs) on NLP tasks where geographic knowledge is intuitively beneficial (e. g., geolocation prediction and dialect feature prediction).
On Cross-Lingual Retrieval with Multilingual Text Encoders
In this work we present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks across a number of diverse language pairs.
DS-TOD: Efficient Domain Specialization for Task Oriented Dialog
Recent work has shown that self-supervised dialog-specific pretraining on large conversational datasets yields substantial gains over traditional language modeling (LM) pretraining in downstream task-oriented dialog (TOD).
AnnIE: An Annotation Platform for Constructing Complete Open Information Extraction Benchmark
Open Information Extraction (OIE) is the task of extracting facts from sentences in the form of relations and their corresponding arguments in schema-free manner.
BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation
In this work, we introduce BenchIE: a benchmark and evaluation framework for comprehensive evaluation of OIE systems for English, Chinese, and German.
Ranked #1 on
Open Information Extraction
on BenchIE
Sustainable Modular Debiasing of Language Models
Unfair stereotypical biases (e. g., gender, racial, or religious biases) encoded in modern pretrained language models (PLMs) have negative ethical implications for widespread adoption of state-of-the-art language technology.
Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biases
We analyze bias in historical corpora as encoded in diachronic distributional semantic models by focusing on two specific forms of bias, namely a political (i. e., anti-communism) and racist (i. e., antisemitism) one.
Scientia Potentia Est -- On the Role of Knowledge in Computational Argumentation
In this survey paper, we fill this gap by (1) proposing a pyramid of types of knowledge required in CA tasks, (2) analysing the state of the art with respect to the reliance and exploitation of these types of knowledge, for each of the for main research areas in CA, and (3) outlining and discussing directions for future research efforts in CA.
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models
We use the evaluation framework to benchmark the widely used conversational DialoGPT model along with the adaptations of four debiasing methods.
Conversational Response Generation
Pretrained Language Models
+1
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems
We find that the most critical factor preventing the creation of truly multilingual ToD systems is the lack of datasets in most languages for both training and evaluation.
DebIE: A Platform for Implicit and Explicit Debiasing of Word Embedding Spaces
In this work, we present DebIE, the first integrated platform for (1) measuring and (2) mitigating bias in word embeddings.
Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval
Therefore, in this work we present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks across a large number of language pairs.
Verb Knowledge Injection for Multilingual Event Processing
We investigate whether injecting explicit information on verbs' semantic-syntactic behaviour improves the performance of LM-pretrained Transformers in event extraction tasks -- downstream tasks for which accurate verb processing is paramount.
Self-Supervised Learning for Visual Summary Identification in Scientific Publications
Providing visual summaries of scientific publications can increase information access for readers and thereby help deal with the exponential growth in the number of scientific publications.
Orthogonal Language and Task Adapters in Zero-Shot Cross-Lingual Transfer
Adapter modules, additional trainable parameters that enable efficient fine-tuning of pretrained transformers, have recently been used for language specialization of multilingual transformers, improving downstream zero-shot cross-lingual transfer.
AraWEAT: Multidimensional Analysis of Biases in Arabic Word Embeddings
Our analysis yields several interesting findings, e. g., that implicit gender bias in embeddings trained on Arabic news corpora steadily increases over time (between 2007 and 2017).
Probing Pretrained Language Models for Lexical Semantics
The success of large pretrained language models (LMs) such as BERT and RoBERTa has sparked interest in probing their representations, in order to unveil what types of knowledge they implicitly capture.
Is Supervised Syntactic Parsing Beneficial for Language Understanding? An Empirical Investigation
Traditional NLP has long held (supervised) syntactic parsing necessary for successful higher-level semantic language understanding (LU).
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers
Following the major success of neural language models (LMs) such as BERT or GPT-2 on a variety of language understanding tasks, recent work focused on injecting (structured) knowledge from external resources into these models.
On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation
We systematically investigate a range of metrics based on state-of-the-art cross-lingual semantic representations obtained with pretrained M-BERT and LASER.
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning
In order to simulate human language capacity, natural language processing systems must be able to reason about the dynamics of everyday situations, including their possible causes and effects.
Ranked #1 on
Cross-Lingual Transfer
on XCOPA
(using extra training data)
From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers
Massively multilingual transformers pretrained with language modeling objectives (e. g., mBERT, XLM-R) have become a de facto default transfer paradigm for zero-shot cross-lingual transfer in NLP, offering unmatched transfer performance.
Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers
Current methods of cross-lingual parser transfer focus on predicting the best parser for a low-resource target language globally, that is, "at treebank level".
Windowing Models for Abstractive Summarization of Long Texts
Neural summarization models suffer from the fixed-size input limitation: if text length surpasses the model's maximal number of input tokens, some document content (possibly summary-relevant) gets truncated Independently summarizing windows of maximal input size disallows for information flow between windows and leads to incoherent summaries.
Two-Level Transformer and Auxiliary Coherence Modeling for Improved Text Segmentation
Breaking down the structure of long texts into semantically coherent segments makes the texts more readable and supports downstream applications like summarization and retrieval.
A General Framework for Implicit and Explicit Debiasing of Distributional Word Vector Spaces
Moreover, we successfully transfer debiasing models, by means of cross-lingual embedding spaces, and remove or attenuate biases in distributional word vector spaces of languages that lack readily available bias specifications.
Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity
In this work, we complement such distributional knowledge with external lexical knowledge, that is, we integrate the discrete knowledge on word-level semantic similarity into pretraining.
Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?
A series of bilingual lexicon induction (BLI) experiments with 15 diverse languages (210 language pairs) show that fully unsupervised CLWE methods still fail for a large number of language pairs (e. g., they yield zero BLI performance for 87/210 pairs).
Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors
In this work, we present a systematic study of biases encoded in distributional word vector spaces: we analyze how consistent the bias effects are across languages, corpora, and embedding models.
Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specialization
Our adversarial post-specialization method propagates the external lexical knowledge to the full distributional space.
Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources
Word vector specialisation (also known as retrofitting) is a portable, light-weight approach to fine-tuning arbitrary distributional word vector spaces by injecting external knowledge from rich lexical resources such as WordNet.
Unsupervised Cross-Lingual Information Retrieval using Monolingual Data Only
We propose a fully unsupervised framework for ad-hoc cross-lingual information retrieval (CLIR) which requires no bilingual data at all.
A Resource-Light Method for Cross-Lingual Semantic Textual Similarity
In contrast, we propose an unsupervised and a very resource-light approach for measuring semantic similarity between texts in different languages.
Cross-Lingual Semantic Textual Similarity
Information Retrieval
+6
