Search Results for author:
Found 23 papers, 14 papers with code
SIGMORPHON 2021 Shared Task on Morphological Reinflection: Generalization Across Languages
This year's iteration of the SIGMORPHON Shared Task on morphological reinflection focuses on typological diversity and cross-lingual variation of morphosyntactic features.
MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropies
Autoregressive language models are trained by minimizing the cross-entropy of the model distribution Q relative to the data distribution P -- that is, minimizing the forward cross-entropy, which is equivalent to maximum likelihood estimation (MLE).
Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning
Real-life multilingual systems should be able to efficiently incorporate new languages as data distributions fed to the system evolve and shift over time.
BloombergGPT: A Large Language Model for Finance
The use of NLP in the realm of financial technology is broad and complex, with applications ranging from sentiment analysis and named entity recognition to question answering.
BoundaryFace: A mining framework with noise label self-correction for Face Recognition
Specifically, a closed-set noise label self-correction module is put forward, making this framework work well on datasets containing a lot of label noise.
How Do Multilingual Encoders Learn Cross-lingual Representation?
We also look at how to inject different cross-lingual signals into multilingual encoders, and the optimization behavior of cross-lingual transfer with these models.
Zero-shot Cross-lingual Transfer is Under-specified Optimization
Pretrained multilingual encoders enable zero-shot cross-lingual transfer, but often produce unreliable models that exhibit high performance variance on the target language.
Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction
Zero-shot cross-lingual information extraction (IE) describes the construction of an IE model for some target language, given existing annotations exclusively in some other language, typically English.
Differentiable Generative Phonology
As the main contribution of our work, we implement the phonological generative system as a neural model differentiable end-to-end, rather than as a set of rules or constraints.
Do Explicit Alignments Robustly Improve Multilingual Encoders?
Multilingual BERT (mBERT), XLM-RoBERTa (XLMR) and other unsupervised multilingual encoders can effectively learn cross-lingual representation.
Which *BERT? A Survey Organizing Contextualized Encoders
Pretrained contextualized text encoders are now a staple of the NLP community.
The SIGMORPHON 2020 Shared Task on Multilingual Grapheme-to-Phoneme Conversion
We describe the design and findings of the SIGMORPHON 2020 shared task on multilingual grapheme-to-phoneme conversion.
SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection
Systems were developed using data from 45 languages and just 5 language families, fine-tuned with data from an additional 45 languages and 10 language families (13 in total), and evaluated on all 90 languages.
Applying the Transformer to Character-level Transduction
The transformer has been shown to outperform recurrent neural network-based sequence-to-sequence models in various word-level NLP tasks.
Are All Languages Created Equal in Multilingual BERT?
Multilingual BERT (mBERT) trained on 104 languages has shown surprisingly good cross-lingual performance on several NLP tasks, even without explicit cross-lingual signals.
The Paradigm Discovery Problem
Our benchmark system first makes use of word embeddings and string similarity to cluster forms by cell and by paradigm.
Emerging Cross-lingual Structure in Pretrained Language Models
We study the problem of multilingual masked language modeling, i. e. the training of a single model on concatenated text from multiple languages, and present a detailed study of several factors that influence why these models are so effective for cross-lingual transfer.
The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection
The SIGMORPHON 2019 shared task on cross-lingual transfer and contextual analysis in morphology examined transfer learning of inflection between 100 language pairs, as well as contextual lemmatization and morphosyntactic description in 66 languages.
Morphological Irregularity Correlates with Frequency
We present a study of morphological irregularity.
Exact Hard Monotonic Attention for Character-Level Transduction
Our models achieve state-of-the-art performance on morphological inflection.
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
Pretrained contextual representation models (Peters et al., 2018; Devlin et al., 2018) have pushed forward the state-of-the-art on many NLP tasks.
Ranked #7 on
Cross-Lingual NER
on CoNLL Spanish
A Simple Joint Model for Improved Contextual Neural Lemmatization
English verbs have multiple forms.
Hard Non-Monotonic Attention for Character-Level Transduction
We compare soft and hard non-monotonic attention experimentally and find that the exact algorithm significantly improves performance over the stochastic approximation and outperforms soft attention.
