Search Results for author: Dietrich Klakow

Found 76 papers, 20 papers with code

Enhancing Reinforcement Learning with discrete interfaces to learn the Dyck Language

no code implementations27 Oct 2021 Florian Dietz, Dietrich Klakow

Even though most interfaces in the real world are discrete, no efficient way exists to train neural networks to make use of them, yet.

Label-Descriptive Patterns and their Application to Characterizing Classification Errors

no code implementations18 Oct 2021 Michael Hedderich, Jonas Fischer, Dietrich Klakow, Jilles Vreeken

Through two real-world case studies we confirm that Premise gives clear and actionable insight into the systematic errors made by modern NLP classifiers.

Classification

Preventing Author Profiling through Zero-Shot Multilingual Back-Translation

1 code implementation19 Sep 2021 David Ifeoluwa Adelani, Miaoran Zhang, Xiaoyu Shen, Ali Davody, Thomas Kleinbauer, Dietrich Klakow

Documents as short as a single sentence may inadvertently reveal sensitive information about their authors, including e. g. their gender or ethnicity.

Style Transfer Text Style Transfer +1

Exploring the Potential of Lexical Paraphrases for Mitigating Noise-Induced Comprehension Errors

no code implementations18 Jul 2021 Anupama Chingacham, Vera Demberg, Dietrich Klakow

We evaluate the intelligibility of synonyms in context and find that choosing a lexical unit that is less risky to be misheard than its synonym introduced an average gain in comprehension of 37% at SNR -5 dB and 21% at SNR 0 dB for babble noise.

Speech Synthesis

Proceedings of the First Workshop on Weakly Supervised Learning (WeaSuL)

no code implementations8 Jul 2021 Michael A. Hedderich, Benjamin Roth, Katharina Kann, Barbara Plank, Alex Ratner, Dietrich Klakow

Welcome to WeaSuL 2021, the First Workshop on Weakly Supervised Learning, co-located with ICLR 2021.

Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study

1 code implementation16 Jun 2021 Badr M. Abdullah, Marius Mosbach, Iuliia Zaitova, Bernd Möbius, Dietrich Klakow

Our experiments show that (1) the distance in the embedding space in the best cases only moderately correlates with phonological distance, and (2) improving the performance on the word discrimination task does not necessarily yield models that better reflect word phonological similarity.

Word Embeddings

Modeling Profanity and Hate Speech in Social Media with Semantic Subspaces

1 code implementation14 Jun 2021 Vanessa Hahn, Dana Ruiter, Thomas Kleinbauer, Dietrich Klakow

We observe that, on both similar and distant target tasks and across all languages, the subspace-based representations transfer more effectively than standard BERT representations in the zero-shot setting, with improvements between F1 +10. 9 and F1 +42. 9 over the baselines across all tested monolingual and cross-lingual scenarios.

To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning

no code implementations16 Apr 2021 Lukas Lange, Jannik Strötgen, Heike Adel, Dietrich Klakow

For this, we study the effects of model transfer on sequence labeling across various domains and tasks and show that our methods based on model similarity and support vector machines are able to predict promising sources, resulting in performance increases of up to 24 F1 points.

Transfer Learning

Do we read what we hear? Modeling orthographic influences on spoken word recognition

no code implementations EACL 2021 Nicole Macher, Badr M. Abdullah, Harm Brouwer, Dietrich Klakow

Theories and models of spoken word recognition aim to explain the process of accessing lexical knowledge given an acoustic realization of a word form.

Familiar words but strange voices: Modelling the influence of speech variability on word recognition

no code implementations EACL 2021 Alexandra Mayn, Badr M. Abdullah, Dietrich Klakow

We present a deep neural model of spoken word recognition which is trained to retrieve the meaning of a word (in the form of a word embedding) given its spoken form, a task which resembles that faced by a human listener.

ANEA: Distant Supervision for Low-Resource Named Entity Recognition

1 code implementation25 Feb 2021 Michael A. Hedderich, Lukas Lange, Dietrich Klakow

Distant supervision allows obtaining labeled training corpora for low-resource settings where only limited hand-annotated data exists.

Low Resource Named Entity Recognition Named Entity Recognition

Emoji-Based Transfer Learning for Sentiment Tasks

1 code implementation EACL 2021 Susann Boy, Dana Ruiter, Dietrich Klakow

This is done using a transfer learning approach, where the parameters learned by an emoji-based source task are transferred to a sentiment target task.

Hate Speech Detection Sentiment Analysis +1

Analysing the Noise Model Error for Realistic Noisy Label Data

3 code implementations24 Jan 2021 Michael A. Hedderich, Dawei Zhu, Dietrich Klakow

Distant and weak supervision allow to obtain large amounts of labeled training data quickly and cheaply, but these automatic annotations tend to contain a high amount of errors.

CoLi at UdS at SemEval-2020 Task 12: Offensive Tweet Detection with Ensembling

no code implementations SEMEVAL 2020 Kathryn Chapman, Johannes Bernhard, Dietrich Klakow

We present our submission and results for SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) where we participated in offensive tweet classification tasks in English, Arabic, Greek, Turkish and Danish.

Language Identification

A Closer Look at Linguistic Knowledge in Masked Language Models: The Case of Relative Clauses in American English

1 code implementation COLING 2020 Marius Mosbach, Stefania Degaetano-Ortlieb, Marie-Pauline Krielke, Badr M. Abdullah, Dietrich Klakow

Transformer-based language models achieve high performance on various tasks, but we still lack understanding of the kind of linguistic knowledge they learn and rely on.

Fusion Models for Improved Visual Captioning

no code implementations28 Oct 2020 Marimuthu Kalimuthu, Aditya Mogadala, Marius Mosbach, Dietrich Klakow

Building on these recent developments, and with the aim of improving the quality of generated captions, the contribution of our work in this paper is two-fold: First, we propose a generic multimodal model fusion framework for caption generation as well as emendation where we utilize different fusion strategies to integrate a pretrained Auxiliary Language Model (AuxLM) within the traditional encoder-decoder visual captioning frameworks.

automatic-speech-recognition Image Captioning +3

Adversarial Learning of Feature-based Meta-Embeddings

no code implementations23 Oct 2020 Lukas Lange, Heike Adel, Jannik Strötgen, Dietrich Klakow

Second, the different embedding types can form clusters in the common embedding space, preventing the computation of a meaningful average of different embeddings and thus, reducing performance.

Sentence Classification

Rediscovering the Slavic Continuum in Representations Emerging from Neural Models of Spoken Language Identification

no code implementations22 Oct 2020 Badr M. Abdullah, Jacek Kudera, Tania Avgustinova, Bernd Möbius, Dietrich Klakow

In this paper, we present a neural model for Slavic language identification in speech signals and analyze its emergent representations to investigate whether they reflect objective measures of language relatedness and/or non-linguists' perception of language similarity.

Language Identification Spoken language identification

On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers

no code implementations Findings of the Association for Computational Linguistics 2020 Marius Mosbach, Anna Khokhlova, Michael A. Hedderich, Dietrich Klakow

Our analysis reveals that while fine-tuning indeed changes the representations of a pre-trained model and these changes are typically larger for higher layers, only in very few cases, fine-tuning has a positive effect on probing accuracy that is larger than just using the pre-trained model with a strong pooling method.

Privacy Guarantees for De-identifying Text Transformations

no code implementations7 Aug 2020 David Ifeoluwa Adelani, Ali Davody, Thomas Kleinbauer, Dietrich Klakow

Machine Learning approaches to Natural Language Processing tasks benefit from a comprehensive collection of real-life user data.

De-identification Dialog Act Classification +3

Cross-Domain Adaptation of Spoken Language Identification for Related Languages: The Curious Case of Slavic Languages

1 code implementation2 Aug 2020 Badr M. Abdullah, Tania Avgustinova, Bernd Möbius, Dietrich Klakow

State-of-the-art spoken language identification (LID) systems, which are based on end-to-end deep neural networks, have shown remarkable success not only in discriminating between distant languages but also between closely-related languages or even different spoken varieties of the same language.

Language Identification Spoken language identification +1

Integrating Image Captioning with Rule-based Entity Masking

no code implementations22 Jul 2020 Aditya Mogadala, Xiaoyu Shen, Dietrich Klakow

Particularly, these image features are subdivided into global and local features, where global features are extracted from the global representation of the image, while local features are extracted from the objects detected locally in an image.

Image Captioning

Sparse Graph to Sequence Learning for Vision Conditioned Long Textual Sequence Generation

no code implementations12 Jul 2020 Aditya Mogadala, Marius Mosbach, Dietrich Klakow

Generating longer textual sequences when conditioned on the visual information is an interesting problem to explore.

Graph-to-Sequence Video Captioning

Robust Differentially Private Training of Deep Neural Networks

1 code implementation19 Jun 2020 Ali Davody, David Ifeoluwa Adelani, Thomas Kleinbauer, Dietrich Klakow

One major drawback of training deep neural networks with DPSGD is a reduction in the model's accuracy.

Learning Functions to Study the Benefit of Multitask Learning

no code implementations9 Jun 2020 Gabriele Bettgenhäuser, Michael A. Hedderich, Dietrich Klakow

Although multitask learning has achieved improved performance in some problems, there are also tasks that lose performance when trained together.

Mathematical Proofs

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

1 code implementation ICLR 2021 Marius Mosbach, Maksym Andriushchenko, Dietrich Klakow

Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks.

Image Manipulation with Natural Language using Two-sidedAttentive Conditional Generative Adversarial Network

no code implementations16 Dec 2019 Dawei Zhu, Aditya Mogadala, Dietrich Klakow

We propose the Two-sidEd Attentive conditional Generative Adversarial Network (TEA-cGAN) to generate semantically manipulated images while preserving other contents such as background intact.

Image Manipulation

Feature-Dependent Confusion Matrices for Low-Resource NER Labeling with Noisy Labels

1 code implementation IJCNLP 2019 Lukas Lange, Michael A. Hedderich, Dietrich Klakow

In low-resource settings, the performance of supervised labeling models can be improved with automatically annotated or distantly supervised data, which is cheap to create but often noisy.

Low Resource Named Entity Recognition Named Entity Recognition +3

Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods

no code implementations22 Jul 2019 Aditya Mogadala, Marimuthu Kalimuthu, Dietrich Klakow

The interest in Artificial Intelligence (AI) and its applications has seen unprecedented growth in the last few years.

Using Multi-Sense Vector Embeddings for Reverse Dictionaries

1 code implementation WS 2019 Michael A. Hedderich, Andrew Yates, Dietrich Klakow, Gerard de Melo

However, they typically cannot serve as a drop-in replacement for conventional single-sense embeddings, because the correct sense vector needs to be selected for each word.

On the security relevance of weights in deep learning

no code implementations8 Feb 2019 Kathrin Grosse, Thomas A. Trost, Marius Mosbach, Michael Backes, Dietrich Klakow

Recently, a weight-based attack on stochastic gradient descent inducing overfitting has been proposed.

Toward Bayesian Synchronous Tree Substitution Grammars for Sentence Planning

no code implementations WS 2018 David M. Howcroft, Dietrich Klakow, Vera Demberg

Developing conventional natural language generation systems requires extensive attention from human experts in order to craft complex sets of sentence planning rules.

Text Generation

Logit Pairing Methods Can Fool Gradient-Based Attacks

1 code implementation29 Oct 2018 Marius Mosbach, Maksym Andriushchenko, Thomas Trost, Matthias Hein, Dietrich Klakow

Recently, Kannan et al. [2018] proposed several logit regularization methods to improve the adversarial robustness of classifiers.

NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation

no code implementations EMNLP 2018 Hui Su, Xiaoyu Shen, Wenjie Li, Dietrich Klakow

Sequence-to-Sequence (seq2seq) models have become overwhelmingly popular in building end-to-end trainable dialogue systems.

Dialogue Generation

Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Data

1 code implementation WS 2018 Michael A. Hedderich, Dietrich Klakow

Manually labeled corpora are expensive to create and often not available for low-resource languages or domains.

NER

A Neural Network Approach for Mixing Language Models

no code implementations23 Aug 2017 Youssef Oualil, Dietrich Klakow

The performance of Neural Network (NN)-based language models is steadily improving due to the emergence of new architectures, which are able to learn different natural language characteristics.

Text Compression

Long-Short Range Context Neural Networks for Language Modeling

no code implementations EMNLP 2016 Youssef Oualil, Mittul Singh, Clayton Greenberg, Dietrich Klakow

The goal of language modeling techniques is to capture the statistical and structural properties of natural languages from training corpora.

Language Modelling Text Compression

A Batch Noise Contrastive Estimation Approach for Training Large Vocabulary Language Models

1 code implementation20 Aug 2017 Youssef Oualil, Dietrich Klakow

Training large vocabulary Neural Network Language Models (NNLMs) is a difficult task due to the explicit requirement of the output layer normalization, which typically involves the evaluation of the full softmax function over the complete vocabulary.

Text Compression

Sequential Recurrent Neural Networks for Language Modeling

no code implementations23 Mar 2017 Youssef Oualil, Clayton Greenberg, Mittul Singh, Dietrich Klakow

Feedforward Neural Network (FNN)-based language models estimate the probability of the next word based on the history of the last N words, whereas Recurrent Neural Networks (RNN) perform the same task based only on the last word and some context information that cycles in the network.

Language Modelling Text Compression

Sub-Word Similarity based Search for Embeddings: Inducing Rare-Word Embeddings for Word Similarity Tasks and Language Modelling

no code implementations COLING 2016 Mittul Singh, Clayton Greenberg, Youssef Oualil, Dietrich Klakow

We augmented pre-trained word embeddings with these novel embeddings and evaluated on a rare word similarity task, obtaining up to 3 times improvement in correlation over the original set of embeddings.

Language Modelling Morphological Analysis +2

Creating Annotated Dialogue Resources: Cross-domain Dialogue Act Classification

no code implementations LREC 2016 Dilafruz Amanova, Volha Petukhova, Dietrich Klakow

This paper describes a method to automatically create dialogue resources annotated with dialogue act information by reusing existing dialogue corpora.

Classification Dialogue Act Classification +1

The DBOX Corpus Collection of Spoken Human-Human and Human-Machine Dialogues

no code implementations LREC 2014 Volha Petukhova, Martin Gropp, Dietrich Klakow, Gregor Eigner, Mario Topf, Stefan Srb, Petr Motlicek, Blaise Potard, John Dines, Olivier Deroo, Ronny Egeler, Uwe Meinz, Steffen Liersch, Anna Schmidt

We first start with human-human Wizard of Oz experiments to collect human-human data in order to model natural human dialogue behaviour, for better understanding of phenomena of human interactions and predicting interlocutors actions, and then replace the human Wizard by an increasingly advanced dialogue system, using evaluation data for system improvement.

Question Answering

Effective Slot Filling Based on Shallow Distant Supervision Methods

no code implementations6 Jan 2014 Benjamin Roth, Tassilo Barth, Michael Wiegand, Mittul Singh, Dietrich Klakow

In the TAC KBP 2013 English Slotfilling evaluation, the submitted main run of the LSV RelationFactory system achieved the top-ranked F1-score of 37. 3%.

Relation Extraction Slot Filling +1

Task-Driven Linguistic Analysis based on an Underspecified Features Representation

no code implementations LREC 2012 Stasinos Konstantopoulos, Valia Kordoni, Nicola Cancedda, Vangelis Karkaletsis, Dietrich Klakow, Jean-Michel Renders

In this paper we explore a task-driven approach to interfacing NLP components, where language processing is guided by the end-task that each application requires.

Cannot find the paper you are looking for? You can Submit a new open access paper.