Search Results for author:
Found 12 papers, 4 papers with code
TeDDi Sample: Text Data Diversity Sample for Language Comparison and Multilingual NLP
We present the TeDDi sample, a diversity sample of text data for language comparison and multilingual Natural Language Processing.
A Measure for Transparent Comparison of Linguistic Diversity in Multilingual NLP Data Sets
Typologically diverse benchmarks are increasingly created to track the progress achieved in multilingual NLP.
The optimality of word lengths. Theoretical foundations and an empirical study
Zipf's law of abbreviation, namely the tendency of more frequent words to be shorter, has been viewed as a manifestation of compression, i. e. the minimization of the length of forms -- a universal principle of natural communication.
From characters to words: the turning point of BPE merges
The distributions of orthographic word types are very different across languages due to typological characteristics, different writing traditions and potentially other factors.
Grammatical error detection in transcriptions of spoken English
We describe the collection of transcription corrections and grammatical error annotations for the CrowdED Corpus of spoken English monologues on business topics.
Optimal coding and the origins of Zipfian laws
Here we consider the problem of optimal coding -- under an arbitrary coding scheme -- and show that it predicts Zipf's law of abbreviation, namely a tendency in natural languages for more frequent words to be shorter.
Using Universal Dependencies in cross-linguistic complexity research
We evaluate corpus-based measures of linguistic complexity obtained using Universal Dependencies (UD) treebanks.
Learning pressures reduce morphological complexity: Linking corpus, computational and experimental evidence
The morphological complexity of languages differs widely and changes over time.
A Comparison Between Morphological Complexity Measures: Typological Data vs. Language Corpora
Language complexity is an intriguing phenomenon argued to play an important role in both language learning and processing.
The word entropy of natural languages
The average uncertainty associated with words is an information-theoretic concept at the heart of quantitative and computational linguistics.
Crowdsourcing a Multi-lingual Speech Corpus: Recording, Transcription and Annotation of the CrowdIS Corpora
We announce the release of the CROWDED CORPUS: a pair of speech corpora collected via crowdsourcing, containing a native speaker corpus of English (CROWDED{\_}ENGLISH), and a corpus of German/English bilinguals (CROWDED{\_}BILINGUAL).
