Search Results for author: Dietrich Klakow

Found 107 papers, 36 papers with code

incom.py 2.0 - Calculating Linguistic Distances and Asymmetries in Auditory Perception of Closely Related Languages

no code implementations RANLP 2021 Marius Mosbach, Irina Stenger, Tania Avgustinova, Bernd Möbius, Dietrich Klakow

We present an extended version of a tool developed for calculating linguistic distances and asymmetries in auditory perception of closely related languages.

regression

Discourse-based Argument Segmentation and Annotation

no code implementations ACL (ISA, IWCS) 2021 Ekaterina Saveleva, Volha Petukhova, Marius Mosbach, Dietrich Klakow

We tested the widely used Penn Discourse Tree Bank full parser (Lin et al., 2010) and the state-of-the-art neural network NeuralEDUSeg (Wang et al., 2018) and XLNet (Yang et al., 2019) models on the two-stage discourse segmentation and discourse relation recognition.

Discourse Segmentation

Modeling the Impact of Syntactic Distance and Surprisal on Cross-Slavic Text Comprehension

no code implementations LREC 2022 Irina Stenger, Philip Georgis, Tania Avgustinova, Bernd Möbius, Dietrich Klakow

We focus on the syntactic variation and measure syntactic distances between nine Slavic languages (Belarusian, Bulgarian, Croatian, Czech, Polish, Slovak, Slovene, Russian, and Ukrainian) using symmetric measures of insertion, deletion and movement of syntactic units in the parallel sentences of the fable “The North Wind and the Sun”.

Cloze Test Reading Comprehension

Adapting Language Models When Training on Privacy-Transformed Data

no code implementations LREC 2022 Tugtekin Turan, Dietrich Klakow, Emmanuel Vincent, Denis Jouvet

In recent years, voice-controlled personal assistants have revolutionized the interaction with smart devices and mobile applications.

Assessment of Sales Negotiation Strategies with ISO 24617-2 Dialogue Act Annotations

no code implementations ISA (LREC) 2022 Jutta Stock, Volha Petukhova, Dietrich Klakow

We hypothesise that the ISO 24617-2 dialogue act annotation framework adequately supports sales negotiation assessment in the domain of call centre conversations.

Some steps towards the generation of diachronic WordNets

no code implementations WS (NoDaLiDa) 2019 Yuri Bizzoni, Marius Mosbach, Dietrich Klakow, Stefania Degaetano-Ortlieb

We apply hyperbolic embeddings to trace the dynamics of change of conceptual-semantic relationships in a large diachronic scientific corpus (200 years).

Graph-based Argument Quality Assessment

no code implementations RANLP 2021 Ekaterina Saveleva, Volha Petukhova, Marius Mosbach, Dietrich Klakow

The paper presents a novel discourse-based approach to argument quality assessment defined as a graph classification task, where the depth of reasoning (argumentation) is evident from the number and type of detected discourse units and relations between them.

Graph Classification

Analyzing the Representational Geometry of Acoustic Word Embeddings

no code implementations8 Jan 2023 Badr M. Abdullah, Dietrich Klakow

In this paper, we take a closer analytical look at AWEs learned from English speech and study how the choice of the learning objective and the architecture shapes their representational profile.

Keyword Spotting Word Embeddings

Integrating Form and Meaning: A Multi-Task Learning Model for Acoustic Word Embeddings

1 code implementation14 Sep 2022 Badr M. Abdullah, Bernd Möbius, Dietrich Klakow

Models of acoustic word embeddings (AWEs) learn to map variable-length spoken word segments onto fixed-dimensionality vector representations such that different acoustic exemplars of the same word are projected nearby in the embedding space.

Multi-Task Learning Word Embeddings

Fusing Sentence Embeddings Into LSTM-based Autoregressive Language Models

1 code implementation4 Aug 2022 Vilém Zouhar, Marius Mosbach, Dietrich Klakow

We present an LSTM-based autoregressive language model which uses prefix embeddings (from a pretrained masked language model) via fusion (e. g. concatenation) to obtain a richer context representation for language modelling.

Language Modelling Sentence Embeddings

TOKEN is a MASK: Few-shot Named Entity Recognition with Pre-trained Language Models

1 code implementation15 Jun 2022 Ali Davody, David Ifeoluwa Adelani, Thomas Kleinbauer, Dietrich Klakow

Transferring knowledge from one domain to another is of practical importance for many tasks in natural language processing, especially when the amount of available data in the target domain is limited.

Domain Adaptation named-entity-recognition +1

StereoKG: Data-Driven Knowledge Graph Construction for Cultural Knowledge and Stereotypes

1 code implementation NAACL (WOAH) 2022 Awantee Deshpande, Dana Ruiter, Marius Mosbach, Dietrich Klakow

Analyzing ethnic or religious bias is important for improving fairness, accountability, and transparency of natural language processing models.

Fairness graph construction +2

Multilingual Normalization of Temporal Expressions with Masked Language Models

no code implementations20 May 2022 Lukas Lange, Jannik Strötgen, Heike Adel, Dietrich Klakow

The detection and normalization of temporal expressions is an important task and a preprocessing step for many applications.

Language Modelling Masked Language Modeling

Exploiting Social Media Content for Self-Supervised Style Transfer

1 code implementation NAACL (SocialNLP) 2022 Dana Ruiter, Thomas Kleinbauer, Cristina España-Bonet, Josef van Genabith, Dietrich Klakow

Recent research on style transfer takes inspiration from unsupervised neural machine translation (UNMT), learning from large amounts of non-parallel data by exploiting cycle consistency loss, back-translation, and denoising autoencoders.

Denoising Machine Translation +3

Meta Self-Refinement for Robust Learning with Weak Supervision

no code implementations15 May 2022 Dawei Zhu, Xiaoyu Shen, Michael A. Hedderich, Dietrich Klakow

However, labels from weak supervision can be rather noisy and the high capacity of DNNs makes them easy to overfit the noisy labels.

Placing M-Phasis on the Plurality of Hate: A Feature-Based Corpus of Hate Online

1 code implementation LREC 2022 Dana Ruiter, Liane Reiners, Ashwin Geet D'Sa, Thomas Kleinbauer, Dominique Fohr, Irina Illina, Dietrich Klakow, Christian Schemer, Angeliki Monnier

Even though hate speech (HS) online has been an important object of research in the last decade, most HS-related corpora over-simplify the phenomenon of hate by attempting to label user comments as "hate" or "neutral".

Hate Speech Detection

Call-sign recognition and understanding for noisy air-traffic transcripts using surveillance information

no code implementations13 Apr 2022 Alexander Blatt, Martin Kocour, Karel Veselý, Igor Szöke, Dietrich Klakow

The introduced data augmentation adds additional performance on high WER transcripts and allows the adaptation of the model to unseen airspaces.

Data Augmentation

Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning

1 code implementation COLING 2022 Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, Dietrich Klakow

Multilingual pre-trained language models (PLMs) have demonstrated impressive performance on several downstream tasks for both high-resourced and low-resourced languages.

NER Sentiment Analysis +4

CLIN-X: pre-trained language models and a study on cross-task transfer for concept extraction in the clinical domain

1 code implementation16 Dec 2021 Lukas Lange, Heike Adel, Jannik Strötgen, Dietrich Klakow

The field of natural language processing (NLP) has recently seen a large change towards using pre-trained language models for solving almost any task.

Clinical Concept Extraction XLM-R

Enhancing Reinforcement Learning with discrete interfaces to learn the Dyck Language

no code implementations27 Oct 2021 Florian Dietz, Dietrich Klakow

Even though most interfaces in the real world are discrete, no efficient way exists to train neural networks to make use of them, yet.

reinforcement-learning reinforcement Learning

Label-Descriptive Patterns and Their Application to Characterizing Classification Errors

1 code implementation18 Oct 2021 Michael Hedderich, Jonas Fischer, Dietrich Klakow, Jilles Vreeken

Characterizing these errors in easily interpretable terms gives insight into whether a classifier is prone to making systematic errors, but also gives a way to act and improve the classifier.

named-entity-recognition Named Entity Recognition +2

Preventing Author Profiling through Zero-Shot Multilingual Back-Translation

1 code implementation EMNLP 2021 David Ifeoluwa Adelani, Miaoran Zhang, Xiaoyu Shen, Ali Davody, Thomas Kleinbauer, Dietrich Klakow

Documents as short as a single sentence may inadvertently reveal sensitive information about their authors, including e. g. their gender or ethnicity.

Style Transfer Text Style Transfer +1

Exploring the Potential of Lexical Paraphrases for Mitigating Noise-Induced Comprehension Errors

no code implementations18 Jul 2021 Anupama Chingacham, Vera Demberg, Dietrich Klakow

We evaluate the intelligibility of synonyms in context and find that choosing a lexical unit that is less risky to be misheard than its synonym introduced an average gain in comprehension of 37% at SNR -5 dB and 21% at SNR 0 dB for babble noise.

Speech Synthesis

Proceedings of the First Workshop on Weakly Supervised Learning (WeaSuL)

no code implementations8 Jul 2021 Michael A. Hedderich, Benjamin Roth, Katharina Kann, Barbara Plank, Alex Ratner, Dietrich Klakow

Welcome to WeaSuL 2021, the First Workshop on Weakly Supervised Learning, co-located with ICLR 2021.

Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study

1 code implementation16 Jun 2021 Badr M. Abdullah, Marius Mosbach, Iuliia Zaitova, Bernd Möbius, Dietrich Klakow

Our experiments show that (1) the distance in the embedding space in the best cases only moderately correlates with phonological distance, and (2) improving the performance on the word discrimination task does not necessarily yield models that better reflect word phonological similarity.

Word Embeddings

Modeling Profanity and Hate Speech in Social Media with Semantic Subspaces

1 code implementation ACL (WOAH) 2021 Vanessa Hahn, Dana Ruiter, Thomas Kleinbauer, Dietrich Klakow

We observe that, on both similar and distant target tasks and across all languages, the subspace-based representations transfer more effectively than standard BERT representations in the zero-shot setting, with improvements between F1 +10. 9 and F1 +42. 9 over the baselines across all tested monolingual and cross-lingual scenarios.

To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning

1 code implementation EMNLP 2021 Lukas Lange, Jannik Strötgen, Heike Adel, Dietrich Klakow

For this, we study the effects of model transfer on sequence labeling across various domains and tasks and show that our methods based on model similarity and support vector machines are able to predict promising sources, resulting in performance increases of up to 24 F1 points.

text similarity Transfer Learning

Familiar words but strange voices: Modelling the influence of speech variability on word recognition

no code implementations EACL 2021 Alexandra Mayn, Badr M. Abdullah, Dietrich Klakow

We present a deep neural model of spoken word recognition which is trained to retrieve the meaning of a word (in the form of a word embedding) given its spoken form, a task which resembles that faced by a human listener.

Do we read what we hear? Modeling orthographic influences on spoken word recognition

no code implementations EACL 2021 Nicole Macher, Badr M. Abdullah, Harm Brouwer, Dietrich Klakow

Theories and models of spoken word recognition aim to explain the process of accessing lexical knowledge given an acoustic realization of a word form.

ANEA: Distant Supervision for Low-Resource Named Entity Recognition

1 code implementation25 Feb 2021 Michael A. Hedderich, Lukas Lange, Dietrich Klakow

Distant supervision allows obtaining labeled training corpora for low-resource settings where only limited hand-annotated data exists.

Low Resource Named Entity Recognition named-entity-recognition +1

Emoji-Based Transfer Learning for Sentiment Tasks

1 code implementation EACL 2021 Susann Boy, Dana Ruiter, Dietrich Klakow

This is done using a transfer learning approach, where the parameters learned by an emoji-based source task are transferred to a sentiment target task.

Hate Speech Detection Sentiment Analysis +1

Analysing the Noise Model Error for Realistic Noisy Label Data

3 code implementations24 Jan 2021 Michael A. Hedderich, Dawei Zhu, Dietrich Klakow

Distant and weak supervision allow to obtain large amounts of labeled training data quickly and cheaply, but these automatic annotations tend to contain a high amount of errors.

CoLi at UdS at SemEval-2020 Task 12: Offensive Tweet Detection with Ensembling

no code implementations SEMEVAL 2020 Kathryn Chapman, Johannes Bernhard, Dietrich Klakow

We present our submission and results for SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) where we participated in offensive tweet classification tasks in English, Arabic, Greek, Turkish and Danish.

BIG-bench Machine Learning Language Identification +2

A Closer Look at Linguistic Knowledge in Masked Language Models: The Case of Relative Clauses in American English

1 code implementation COLING 2020 Marius Mosbach, Stefania Degaetano-Ortlieb, Marie-Pauline Krielke, Badr M. Abdullah, Dietrich Klakow

Transformer-based language models achieve high performance on various tasks, but we still lack understanding of the kind of linguistic knowledge they learn and rely on.

Fusion Models for Improved Visual Captioning

no code implementations28 Oct 2020 Marimuthu Kalimuthu, Aditya Mogadala, Marius Mosbach, Dietrich Klakow

Building on these recent developments, and with the aim of improving the quality of generated captions, the contribution of our work in this paper is two-fold: First, we propose a generic multimodal model fusion framework for caption generation as well as emendation where we utilize different fusion strategies to integrate a pretrained Auxiliary Language Model (AuxLM) within the traditional encoder-decoder visual captioning frameworks.

Automatic Speech Recognition Image Captioning +3

FAME: Feature-Based Adversarial Meta-Embeddings for Robust Input Representations

1 code implementation EMNLP 2021 Lukas Lange, Heike Adel, Jannik Strötgen, Dietrich Klakow

Combining several embeddings typically improves performance in downstream tasks as different embeddings encode different information.

NER POS +2

Rediscovering the Slavic Continuum in Representations Emerging from Neural Models of Spoken Language Identification

no code implementations VarDial (COLING) 2020 Badr M. Abdullah, Jacek Kudera, Tania Avgustinova, Bernd Möbius, Dietrich Klakow

In this paper, we present a neural model for Slavic language identification in speech signals and analyze its emergent representations to investigate whether they reflect objective measures of language relatedness and/or non-linguists' perception of language similarity.

Language Identification Spoken language identification

On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers

no code implementations EMNLP (BlackboxNLP) 2020 Marius Mosbach, Anna Khokhlova, Michael A. Hedderich, Dietrich Klakow

Our analysis reveals that while fine-tuning indeed changes the representations of a pre-trained model and these changes are typically larger for higher layers, only in very few cases, fine-tuning has a positive effect on probing accuracy that is larger than just using the pre-trained model with a strong pooling method.

Privacy Guarantees for De-identifying Text Transformations

1 code implementation7 Aug 2020 David Ifeoluwa Adelani, Ali Davody, Thomas Kleinbauer, Dietrich Klakow

Machine Learning approaches to Natural Language Processing tasks benefit from a comprehensive collection of real-life user data.

BIG-bench Machine Learning De-identification +5

Cross-Domain Adaptation of Spoken Language Identification for Related Languages: The Curious Case of Slavic Languages

1 code implementation2 Aug 2020 Badr M. Abdullah, Tania Avgustinova, Bernd Möbius, Dietrich Klakow

State-of-the-art spoken language identification (LID) systems, which are based on end-to-end deep neural networks, have shown remarkable success not only in discriminating between distant languages but also between closely-related languages or even different spoken varieties of the same language.

Language Identification Spoken language identification +1

Integrating Image Captioning with Rule-based Entity Masking

no code implementations22 Jul 2020 Aditya Mogadala, Xiaoyu Shen, Dietrich Klakow

Particularly, these image features are subdivided into global and local features, where global features are extracted from the global representation of the image, while local features are extracted from the objects detected locally in an image.

Image Captioning

Sparse Graph to Sequence Learning for Vision Conditioned Long Textual Sequence Generation

no code implementations12 Jul 2020 Aditya Mogadala, Marius Mosbach, Dietrich Klakow

Generating longer textual sequences when conditioned on the visual information is an interesting problem to explore.

Graph-to-Sequence Video Captioning

On the effect of normalization layers on Differentially Private training of deep Neural networks

1 code implementation19 Jun 2020 Ali Davody, David Ifeoluwa Adelani, Thomas Kleinbauer, Dietrich Klakow

Differentially private stochastic gradient descent (DPSGD) is a variation of stochastic gradient descent based on the Differential Privacy (DP) paradigm, which can mitigate privacy threats that arise from the presence of sensitive information in training data.

Learning Functions to Study the Benefit of Multitask Learning

no code implementations9 Jun 2020 Gabriele Bettgenhäuser, Michael A. Hedderich, Dietrich Klakow

Although multitask learning has achieved improved performance in some problems, there are also tasks that lose performance when trained together.

Mathematical Proofs Symbolic Regression

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

1 code implementation ICLR 2021 Marius Mosbach, Maksym Andriushchenko, Dietrich Klakow

Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks.

Misconceptions

Image Manipulation with Natural Language using Two-sidedAttentive Conditional Generative Adversarial Network

no code implementations16 Dec 2019 Dawei Zhu, Aditya Mogadala, Dietrich Klakow

We propose the Two-sidEd Attentive conditional Generative Adversarial Network (TEA-cGAN) to generate semantically manipulated images while preserving other contents such as background intact.

Image Manipulation

Feature-Dependent Confusion Matrices for Low-Resource NER Labeling with Noisy Labels

1 code implementation IJCNLP 2019 Lukas Lange, Michael A. Hedderich, Dietrich Klakow

In low-resource settings, the performance of supervised labeling models can be improved with automatically annotated or distantly supervised data, which is cheap to create but often noisy.

Low Resource Named Entity Recognition named-entity-recognition +3

Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods

no code implementations22 Jul 2019 Aditya Mogadala, Marimuthu Kalimuthu, Dietrich Klakow

Interest in Artificial Intelligence (AI) and its applications has seen unprecedented growth in the last few years.

BIG-bench Machine Learning

Using Multi-Sense Vector Embeddings for Reverse Dictionaries

1 code implementation WS 2019 Michael A. Hedderich, Andrew Yates, Dietrich Klakow, Gerard de Melo

However, they typically cannot serve as a drop-in replacement for conventional single-sense embeddings, because the correct sense vector needs to be selected for each word.

On the security relevance of weights in deep learning

no code implementations8 Feb 2019 Kathrin Grosse, Thomas A. Trost, Marius Mosbach, Michael Backes, Dietrich Klakow

Recently, a weight-based attack on stochastic gradient descent inducing overfitting has been proposed.

Toward Bayesian Synchronous Tree Substitution Grammars for Sentence Planning

no code implementations WS 2018 David M. Howcroft, Dietrich Klakow, Vera Demberg

Developing conventional natural language generation systems requires extensive attention from human experts in order to craft complex sets of sentence planning rules.

Text Generation

Logit Pairing Methods Can Fool Gradient-Based Attacks

1 code implementation29 Oct 2018 Marius Mosbach, Maksym Andriushchenko, Thomas Trost, Matthias Hein, Dietrich Klakow

Recently, Kannan et al. [2018] proposed several logit regularization methods to improve the adversarial robustness of classifiers.

Adversarial Robustness

NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation

no code implementations EMNLP 2018 Hui Su, Xiaoyu Shen, Wenjie Li, Dietrich Klakow

Sequence-to-Sequence (seq2seq) models have become overwhelmingly popular in building end-to-end trainable dialogue systems.

Dialogue Generation

Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Data

1 code implementation WS 2018 Michael A. Hedderich, Dietrich Klakow

Manually labeled corpora are expensive to create and often not available for low-resource languages or domains.

NER

A Neural Network Approach for Mixing Language Models

no code implementations23 Aug 2017 Youssef Oualil, Dietrich Klakow

The performance of Neural Network (NN)-based language models is steadily improving due to the emergence of new architectures, which are able to learn different natural language characteristics.

Text Compression

Long-Short Range Context Neural Networks for Language Modeling

no code implementations EMNLP 2016 Youssef Oualil, Mittul Singh, Clayton Greenberg, Dietrich Klakow

The goal of language modeling techniques is to capture the statistical and structural properties of natural languages from training corpora.

Language Modelling Text Compression

A Batch Noise Contrastive Estimation Approach for Training Large Vocabulary Language Models

1 code implementation20 Aug 2017 Youssef Oualil, Dietrich Klakow

Training large vocabulary Neural Network Language Models (NNLMs) is a difficult task due to the explicit requirement of the output layer normalization, which typically involves the evaluation of the full softmax function over the complete vocabulary.

Text Compression

Sequential Recurrent Neural Networks for Language Modeling

no code implementations23 Mar 2017 Youssef Oualil, Clayton Greenberg, Mittul Singh, Dietrich Klakow

Feedforward Neural Network (FNN)-based language models estimate the probability of the next word based on the history of the last N words, whereas Recurrent Neural Networks (RNN) perform the same task based only on the last word and some context information that cycles in the network.

Language Modelling Text Compression

Sub-Word Similarity based Search for Embeddings: Inducing Rare-Word Embeddings for Word Similarity Tasks and Language Modelling

no code implementations COLING 2016 Mittul Singh, Clayton Greenberg, Youssef Oualil, Dietrich Klakow

We augmented pre-trained word embeddings with these novel embeddings and evaluated on a rare word similarity task, obtaining up to 3 times improvement in correlation over the original set of embeddings.

Language Modelling Morphological Analysis +2

Creating Annotated Dialogue Resources: Cross-domain Dialogue Act Classification

no code implementations LREC 2016 Dilafruz Amanova, Volha Petukhova, Dietrich Klakow

This paper describes a method to automatically create dialogue resources annotated with dialogue act information by reusing existing dialogue corpora.

Classification Dialogue Act Classification +1

The DBOX Corpus Collection of Spoken Human-Human and Human-Machine Dialogues

no code implementations LREC 2014 Volha Petukhova, Martin Gropp, Dietrich Klakow, Gregor Eigner, Mario Topf, Stefan Srb, Petr Motlicek, Blaise Potard, John Dines, Olivier Deroo, Ronny Egeler, Uwe Meinz, Steffen Liersch, Anna Schmidt

We first start with human-human Wizard of Oz experiments to collect human-human data in order to model natural human dialogue behaviour, for better understanding of phenomena of human interactions and predicting interlocutors actions, and then replace the human Wizard by an increasingly advanced dialogue system, using evaluation data for system improvement.

Question Answering

Effective Slot Filling Based on Shallow Distant Supervision Methods

no code implementations6 Jan 2014 Benjamin Roth, Tassilo Barth, Michael Wiegand, Mittul Singh, Dietrich Klakow

In the TAC KBP 2013 English Slotfilling evaluation, the submitted main run of the LSV RelationFactory system achieved the top-ranked F1-score of 37. 3%.

Relation Extraction Retrieval +3

Task-Driven Linguistic Analysis based on an Underspecified Features Representation

no code implementations LREC 2012 Stasinos Konstantopoulos, Valia Kordoni, Nicola Cancedda, Vangelis Karkaletsis, Dietrich Klakow, Jean-Michel Renders

In this paper we explore a task-driven approach to interfacing NLP components, where language processing is guided by the end-task that each application requires.

Cannot find the paper you are looking for? You can Submit a new open access paper.