Search Results for author:
Found 56 papers, 18 papers with code
The LMU Munich System for the WMT20 Very Low Resource Supervised MT Task
We present our systems for the WMT20 Very Low Resource MT Task for translation between German and Upper Sorbian.
Why don’t people use character-level machine translation?
We present a literature and empirical survey that critically assesses the state of the art in character-level modeling for machine translation (MT).
Unsupervised Parallel Sentence Extraction from Comparable Corpora
We show that our approach is effective, on three language-pairs, without the use of any bilingual signal which is important because parallel sentence mining is most useful in low resource scenarios.
Adapting Entities across Languages and Cultures
He is associated with founding a company in the United States, so perhaps the German founder Carl Benz could stand in for Gates in those contexts.
Do not neglect related languages: The case of low-resource Occitan cross-lingual word embeddings
In contrast to previous approaches which leverage independently pre-trained embeddings of languages, we (i) train CLWEs for the low-resource and a related language jointly and (ii) map them to the target language to build the final multilingual space.
Bilingual Lexicon Induction
Cross-Lingual Word Embeddings
+1
Findings of the WMT 2020 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT
We describe the WMT 2020 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT.
The LMU Munich System for the WMT 2021 Large-Scale Multilingual Machine Translation Shared Task
This paper describes the submission of LMU Munich to the WMT 2021 multilingual machine translation task for small track #1, which studies translation between 6 languages (Croatian, Hungarian, Estonian, Serbian, Macedonian, English) in 30 directions.
Improving Machine Translation of Rare and Unseen Word Senses
The performance of NMT systems has improved drastically in the past few years but the translation of multi-sense words still poses a challenge.
Findings of the WMT 2021 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT
We present the findings of the WMT2021 Shared Tasks in Unsupervised MT and Very Low Resource Supervised MT.
The LMU Munich Systems for the WMT21 Unsupervised and Very Low-Resource Translation Task
We present our submissions to the WMT21 shared task in Unsupervised and Very Low Resource machine translation between German and Upper Sorbian, German and Lower Sorbian, and Russian and Chuvash.
Don’t Forget Cheap Training Signals Before Building Unsupervised Bilingual Word Embeddings
The two approaches we find most effective are: 1) using identical words as seed lexicons (which unsupervised approaches incorrectly assume are not available for orthographically distinct language pairs) and 2) combining such lexicons with pairs extracted by matching romanized versions of words with an edit distance threshold.
Cross-Lingual Transfer Learning for Hate Speech Detection
Rather than collecting and annotating new hate speech data, we show how to use cross-lingual transfer learning to leverage already existing data from higher-resource languages.
Are BabyLMs Second Language Learners?
This paper describes a linguistically-motivated approach to the 2024 edition of the BabyLM Challenge (Warstadt et al. 2023).
Hate Personified: Investigating the role of LLMs in content moderation
For subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model's (LLM) ability to represent diverse groups is unclear.
Style-Specific Neurons for Steering LLMs in Text Style Transfer
Text style transfer (TST) aims to modify the style of a text without altering its original meaning.
CUTE: Measuring LLMs' Understanding of Their Tokens
Large Language Models (LLMs) show remarkable performance on a wide variety of tasks.
LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback
We perform multilingual instruction tuning on the constructed instruction data and further align the LLMs with human feedback using the DPO algorithm on our cross-lingual human feedback dataset.
Joint Lemmatization and Morphological Tagging with LEMMING
We present LEMMING, a modular log-linear model that jointly models lemmatization and tagging and supports the integration of arbitrary global features.
Labeled Morphological Segmentation with Semi-Markov Models
We present labeled morphological segmentation, an alternative view of morphological processing that unifies several tasks.
Understanding Cross-Lingual Alignment -- A Survey
Cross-lingual alignment, the meaningful similarity of representations across languages in multilingual language models, has been an active field of research in recent years.
Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You
Our results show that not only do models exhibit strong gender biases but they also behave differently across languages.
Multilingual Word Embeddings for Low-Resource Languages using Anchors and a Chain of Related Languages
In this paper, we propose to build multilingual word embeddings (MWEs) via a novel language chain-based approach, that incorporates intermediate related languages to bridge the gap between the distant source and target.
Extending Multilingual Machine Translation through Imitation Learning
Despite the growing variety of languages supported by existing multilingual neural machine translation (MNMT) models, most of the world's languages are still being left behind.
Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity
We investigate outlier dimensions and their relationship to anisotropy in multiple pre-trained multilingual language models.
On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss
By conducting extensive experiments on different language pairs, including similar and distant, high and low-resource languages, we find that our method alleviates the copying problem, thus improving the translation performance on low-resource languages.
How to Solve Few-Shot Abusive Content Detection Using the Data We Actually Have
Our experiments show that using already existing datasets and only a few-shots of the target task the performance of models improve both monolingually and across languages.
Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translation
Despite advances in multilingual neural machine translation (MNMT), we argue that there are still two major challenges in this area: data imbalance and representation degeneration.
AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Models
We also explore weight averaging of adapters trained on the same domain with different hyper-parameters, and show that it preserves the performance of a PLM on new domains while obtaining strong in-domain results.
Speaking Multiple Languages Affects the Moral Bias of Language Models
Do the models capture moral norms from English and impose them on other languages?
$m^4Adapter$: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter
We consider a very challenging scenario: adapting the MNMT model both to a new domain and to a new language pair at the same time.
A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing
With this survey, the first overview on class imbalance in deep-learning based NLP, we provide guidance for NLP researchers and practitioners dealing with imbalanced data.
Language-Family Adapters for Low-Resource Multilingual Neural Machine Translation
Training a new adapter on each language pair or training a single adapter on all language pairs without updating the pretrained model has been proposed as a parameter-efficient alternative.
Don't Forget Cheap Training Signals Before Building Unsupervised Bilingual Word Embeddings
The two approaches we find most effective are: 1) using identical words as seed lexicons (which unsupervised approaches incorrectly assume are not available for orthographically distinct language pairs) and 2) combining such lexicons with pairs extracted by matching romanized versions of words with an edit distance threshold.
Demonstrating CAT: Synthesizing Data-Aware Conversational Agents for Transactional Databases
However, developing a conversational agent (i. e., a chatbot-like interface) to allow end-users to interact with an application using natural language requires both immense amounts of training data and NLP expertise.
Modeling Target-Side Morphology in Neural Machine Translation: A Comparison of Strategies
Key challenges of rich target-side morphology in data-driven machine translation include: (1) A large amount of differently inflected word surface forms entails a larger vocabulary and thus data sparsity.
Do Multilingual Language Models Capture Differing Moral Norms?
Massively multilingual sentence representations are trained on large corpora of uncurated data, with a very imbalanced proportion of languages included in the training.
Combining Static and Contextualised Multilingual Embeddings
We combine the strengths of static and contextual models to improve multilingual representations.
Addressing the Challenges of Cross-Lingual Hate Speech Detection
The goal of hate speech detection is to filter negative online content aiming at certain groups of people.
Improving Both Domain Robustness and Domain Adaptability in Machine Translation
First, we want to reach domain robustness, i. e., we want to reach high quality on both domains seen in the training data and unseen domains.
Why don't people use character-level machine translation?
We present a literature and empirical survey that critically assesses the state of the art in character-level modeling for machine translation (MT).
Neural String Edit Distance
We propose the neural string edit distance model for string-pair matching and string transduction based on learnable string edit distance.
Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation
Successful methods for unsupervised neural machine translation (UNMT) employ crosslingual pretraining via self-supervision, often in the form of a masked language modeling or a sequence generation task, which requires the model to align the lexical- and high-level representations of the two languages.
Combining Word Embeddings with Bilingual Orthography Embeddings for Bilingual Dictionary Induction
In this paper, we enrich BWE-based BDI with transliteration information by using Bilingual Orthography Embeddings (BOEs).
ContraCAT: Contrastive Coreference Analytical Templates for Machine Translation
Recent high scores on pronoun translation using context-aware neural machine translation have suggested that current approaches work well.
The LMU Munich System for the WMT 2020 Unsupervised Machine Translation Shared Task
Our core unsupervised neural machine translation (UNMT) system follows the strategy of Chronopoulou et al. (2020), using a monolingual pretrained language generation model (on German) and fine-tuning it on both German and Upper Sorbian, before initializing a UNMT model, which is trained with online backtranslation.
Anchor-based Bilingual Word Embeddings for Low-Resource Languages
For low resource languages training MWEs monolingually results in MWEs of poor quality, and thus poor bilingual word embeddings (BWEs) as well.
Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT
Using a language model (LM) pretrained on two languages with large monolingual data in order to initialize an unsupervised neural machine translation (UNMT) system yields state-of-the-art results.
Pragmatic information in translation: a corpus-based study of tense and mood in English and German
Grammatical tense and mood are important linguistic phenomena to consider in natural language processing (NLP) research.
Addressing Zero-Resource Domains Using Document-Level Context in Neural Machine Translation
Achieving satisfying performance in machine translation on domains for which there is no training data is challenging.
Towards Reasonably-Sized Character-Level Transformer NMT by Finetuning Subword Systems
Applying the Transformer architecture on the character level usually requires very deep architectures that are difficult and slow to train.
On the Language Neutrality of Pre-trained Multilingual Representations
Multilingual contextual embeddings, such as multilingual BERT and XLM-RoBERTa, have proved useful for many multi-lingual tasks.
Embedding Learning Through Multilingual Concept Induction
We present a new method for estimating vector space representations of words: embedding learning by concept induction.
Modeling Target-Side Inflection in Neural Machine Translation
NMT systems have problems with large vocabulary sizes.
Target-Side Context for Discriminative Models in Statistical Machine Translation
Discriminative translation models utilizing source context have been shown to help statistical machine translation performance.
