Search Results for author:
Found 191 papers, 90 papers with code
On the Machine Learning of Ethical Judgments from Natural Language
Ethics is one of the longest standing intellectual endeavors of humanity.
SIGMORPHON 2021 Shared Task on Morphological Reinflection: Generalization Across Languages
This year's iteration of the SIGMORPHON Shared Task on morphological reinflection focuses on typological diversity and cross-lingual variation of morphosyntactic features.
Efficient Sampling of Dependency Structure
In this paper, we adapt two spanning tree sampling algorithms to faithfully sample dependency trees from a graph subject to the root constraint.
SIGMORPHON–UniMorph 2022 Shared Task 0: Generalization and Typologically Diverse Morphological Inflection
The 2022 SIGMORPHON–UniMorph shared task on large scale morphological inflection generation included a wide range of typologically diverse languages: 33 languages from 11 top-level language families: Arabic (Modern Standard), Assamese, Braj, Chukchi, Eastern Armenian, Evenki, Georgian, Gothic, Gujarati, Hebrew, Hungarian, Itelmen, Karelian, Kazakh, Ket, Khalkha Mongolian, Kholosi, Korean, Lamahalot, Low German, Ludic, Magahi, Middle Low German, Old English, Old High German, Old Norse, Polish, Pomak, Slovak, Turkish, Upper Sorbian, Veps, and Xibe.
Measuring the Similarity of Grammatical Gender Systems by Comparing Partitions
Of particular interest, languages on the same branch of our phylogenetic tree are notably similar, whereas languages from separate branches are no more similar than chance.
A surprisal–duration trade-off across and within the world’s languages
We thus conclude that there is strong evidence of a surprisal–duration trade-off in operation, both across and within the world’s languages.
Conditional Poisson Stochastic Beams
Beam search is the default decoding strategy for many sequence generation tasks in NLP.
The SIGTYP 2022 Shared Task on the Prediction of Cognate Reflexes
This study describes the structure and the results of the SIGTYP 2022 shared task on the prediction of cognate reflexes from multilingual wordlists.
High probability or low information? The probability–quality paradox in language generation
When generating natural language from neural probabilistic models, high probability does not always coincide with high quality.
An Analysis of On-the-fly Determinization of Finite-state Automata
In this paper we establish an abstraction of on-the-fly determinization of finite-state automata using transition monoids and demonstrate how it can be applied to bound the asymptotics.
A Geometric Notion of Causal Probing
We show that our counterfactual notion of information in a subspace is optimized by a $\textit{causal}$ concept subspace.
On the Efficacy of Sampling Adapters
While this trade-off is not reflected in standard metrics of distribution quality (such as perplexity), we find that several precision-emphasizing measures indeed indicate that sampling adapters can lead to probability distributions more aligned with the true distribution.
Testing the Predictions of Surprisal Theory in 11 Languages
We address this gap in the current literature by investigating the relationship between surprisal and reading times in eleven different languages, distributed across five language families.
Generalizing Backpropagation for Gradient-Based Interpretability
Many popular feature-attribution methods for interpreting deep neural networks rely on computing the gradients of a model's output with respect to its inputs.
Efficient Semiring-Weighted Earley Parsing
This paper provides a reference description, in the form of a deduction system, of Earley's (1970) context-free parsing algorithm with various speed-ups.
A Formal Perspective on Byte-Pair Encoding
Via submodular functions, we prove that the iterative greedy version is a $\frac{1}{{\sigma(\boldsymbol{\mu}^\star)}}(1-e^{-{\sigma(\boldsymbol{\mu}^\star)}})$-approximation of an optimal merge sequence, where ${\sigma(\boldsymbol{\mu}^\star)}$ is the total backward curvature with respect to the optimal merge sequence $\boldsymbol{\mu}^\star$.
Hexatagging: Projective Dependency Parsing as Tagging
We introduce a novel dependency parser, the hexatagger, that constructs dependency trees by tagging the words in a sentence with elements from a finite set of possible tags.
Convergence and Diversity in the Control Hierarchy
Furthermore, using an even stricter notion of equivalence called d-strong equivalence, we make precise the intuition that a CFG controlling a CFG is a TAG, a PDA controlling a PDA is an embedded PDA, and a PDA controlling a CFG is a LIG.
A Cross-Linguistic Pressure for Uniform Information Density in Word Order
Here, we ask whether a pressure for UID may have influenced word order patterns cross-linguistically.
LEACE: Perfect linear concept erasure in closed form
Concept erasure aims to remove specified features from a representation.
Structured Voronoi Sampling
In this paper, we take an important step toward building a principled approach for sampling from language models with gradient-based methods.
Learning the String Partial Order
We show that most structured prediction problems can be solved in linear time and space by considering them as partial orderings of the tokens in the input string.
All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations
Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models.
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
In addition to producing AI-generated content (AIGC), we also demonstrate the possibility of using RecurrentGPT as an interactive fiction that directly interacts with consumers.
Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus
Several recent papers claim human parity at sentence-level Machine Translation (MT), especially in high-resource languages.
Efficient Prompting via Dynamic In-Context Learning
To achieve this, we train a meta controller that predicts the number of in-context examples suitable for the generalist model to make a good prediction based on the performance-efficiency trade-off for a specific input.
Controlled Text Generation with Natural Language Instructions
Large language models generate fluent texts and can follow natural language instructions to solve a wide range of tasks without task-specific training.
Discriminative Class Tokens for Text-to-Image Diffusion Models
This approach has two disadvantages: (i) supervised datasets are generally small compared to large-scale scraped text-image datasets on which text-to-image models are trained, affecting the quality and diversity of the generated images, or (ii) the input is a hard-coded label, as opposed to free-form text, limiting the control over the generated images.
Algorithms for Acyclic Weighted Finite-State Automata with Failure Arcs
The pathsum in ordinary acyclic WFSAs is efficiently computed by the backward algorithm in time $O(|E|)$, where $E$ is the set of transitions.
A Measure-Theoretic Characterization of Tight Language Models
Language modeling, a central task in natural language processing, involves estimating a probability distribution over strings.
The Ordered Matrix Dirichlet for State-Space Models
For discrete data, SSMs commonly do so through a state-to-action emission matrix and a state-to-state transition matrix.
On the Effect of Anticipation on Reading Times
We assess the effect of anticipation on reading by comparing how well surprisal and contextual entropy predict reading times on four naturalistic reading datasets: two self-paced and two eye-tracking.
Schrödinger's Bat: Diffusion Models Sometimes Generate Polysemous Words in Superposition
Recent work has shown that despite their impressive capabilities, text-to-image diffusion models such as DALL-E 2 (Ramesh et al., 2022) can display strange behaviours when a prompt contains a word with multiple possible meanings, often generating images containing both senses of the word (Rassin et al., 2022).
On Parsing as Tagging
There have been many proposals to reduce constituency parsing to tagging in the literature.
The Architectural Bottleneck Principle
This shift in perspective leads us to propose a new principle for probing, the architectural bottleneck principle: In order to estimate how much information a given component could extract, a probe should look exactly like the component.
Autoregressive Structured Prediction with Language Models
Recent years have seen a paradigm shift in NLP towards using pretrained language models ({PLM}) for a wide range of tasks.
Ranked #1 on
Relation Extraction
on CoNLL04
(RE+ Micro F1 metric)
Investigating the Role of Centering Theory in the Context of Neural Coreference Resolution Systems
Our analysis further shows that contextualized embeddings contain much of the coherence information, which helps explain why CT can only provide little gains to modern neural coreference resolvers which make use of pretrained representations.
A Bilingual Parallel Corpus with Discourse Annotations
The BWB corpus consists of Chinese novels translated by experts into English, and the annotated test set is designed to probe the ability of machine translation systems to model various discourse phenomena.
Mutual Information Alleviates Hallucinations in Abstractive Summarization
Despite significant progress in the quality of language generated from abstractive summarization models, these models still exhibit the tendency to hallucinate, i. e., output content not supported by the source document.
Log-linear Guardedness and its Implications
Methods for erasing human-interpretable concepts from neural representations that assume linearity have been found to be tractable and useful.
Algorithms for Weighted Pushdown Automata
Weighted pushdown automata (WPDAs) are at the core of many natural language processing tasks, like syntax-based statistical machine translation and transition-based dependency parsing.
An Ordinal Latent Variable Model of Conflict Intensity
It is based only on the action category ("what") and disregards the subject ("who") and object ("to whom") of an event, as well as contextual information, like associated casualty count, that should contribute to the perception of an event's "intensity".
State-of-the-art generalisation research in NLP: A taxonomy and review
We present a taxonomy for characterising and understanding generalisation research in NLP.
Equivariant Transduction through Invariant Alignment
The ability to generalize compositionally is key to understanding the potentially infinite number of sentences that can be constructed in a human language from only a finite number of words.
On the Intersection of Context-Free and Regular Languages
It shows, by a simple construction, that the intersection of a context-free language and a regular language is itself context-free.
On the Role of Negative Precedent in Legal Outcome Prediction
In contrast, we turn our focus to negative outcomes here, and introduce a new task of negative outcome prediction.
Benchmarking Compositionality with Formal Languages
Recombining known primitive concepts into larger novel combinations is a quintessentially human cognitive capability.
Visual Comparison of Language Model Adaptation
The simplicity of adapter training and composition comes along with new challenges, such as maintaining an overview of adapter properties and effectively comparing their produced embedding spaces.
Probing via Prompting
We then examine the usefulness of a specific linguistic property for pre-training by removing the heads that are essential to that property and evaluating the resulting model's performance on language modeling.
The SIGMORPHON 2022 Shared Task on Morpheme Segmentation
The SIGMORPHON 2022 shared task on morpheme segmentation challenged systems to decompose a word into a sequence of morphemes and covered most types of morphology: compounds, derivations, and inflections.
On the Usefulness of Embeddings, Clusters and Strings for Text Generator Evaluation
As we show, however, this is not a tight approximation -- in either theory or practice.
Naturalistic Causal Probing for Morpho-Syntax
Probing has become a go-to methodology for interpreting and analyzing deep neural models in natural language processing.
A Structured Span Selector
Many natural language processing tasks, e. g., coreference resolution and semantic role labeling, require selecting text spans and making decisions about them.
UniMorph 4.0: Universal Morphology
The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema.
Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models
The success of multilingual pre-trained models is underpinned by their ability to learn representations shared by multiple languages even in absence of any explicit supervision.
Exact Paired-Permutation Testing for Structured Test Statistics
However, practitioners rely on Monte Carlo approximation to perform this test due to a lack of a suitable exact algorithm.
Probing for the Usage of Grammatical Number
We also find that BERT uses a separate encoding of grammatical number for nouns and verbs.
Estimating the Entropy of Linguistic Distributions
Shannon entropy is often a quantity of interest to linguists studying the communicative capacity of human language.
On the probability-quality paradox in language generation
Specifically, we posit that human-like language should contain an amount of information (quantified as negative log-probability) that is close to the entropy of the distribution over natural strings.
Analyzing Wrap-Up Effects through an Information-Theoretic Lens
Numerous analyses of reading time (RT) data have been implemented -- all in an effort to better understand the cognitive processes driving reading comprehension.
On Decoding Strategies for Neural Text Generators
For example, the nature of the diversity-quality trade-off in language generation is very task-specific; the length bias often attributed to beam search is not constant across tasks.
Locally Typical Sampling
Automatic and human evaluations show that, in comparison to nucleus and top-k sampling, locally typical sampling offers competitive performance (in both abstractive summarization and story generation) in terms of quality while consistently reducing degenerate repetitions.
Linear Adversarial Concept Erasure
Modern neural models trained on textual data rely on pre-trained representations that emerge without direct supervision.
Kernelized Concept Erasure
One prominent approach for the identification of concepts in neural representations is searching for a linear subspace whose erasure prevents the prediction of the concept from the representations.
A Latent-Variable Model for Intrinsic Probing
The success of pre-trained contextualized representations has prompted researchers to analyze them for the presence of linguistic information.
A Word on Machine Ethics: A Response to Jiang et al. (2021)
Ethics is one of the longest standing intellectual endeavors of humanity.
Probing as Quantifying Inductive Bias
Such performance improvements have motivated researchers to quantify and understand the linguistic information encoded in these representations.
A surprisal--duration trade-off across and within the world's languages
We thus conclude that there is strong evidence of a surprisal--duration trade-off in operation, both across and within the world's languages.
On Homophony and Rényi Entropy
Homophony's widespread presence in natural languages is a controversial topic.
Classifying Dyads for Militarized Conflict Analysis
We do this by devising a set of textual and graph-based features which represent each of the causes.
Revisiting the Uniform Information Density Hypothesis
The uniform information density (UID) hypothesis posits a preference among language users for utterances structured such that information is distributed uniformly across a signal.
Conditional Poisson Stochastic Beam Search
In this work, we propose a new method for turning beam search into a stochastic process: Conditional Poisson stochastic beam search.
A Plug-and-Play Method for Controlled Text Generation
Large pre-trained language models have repeatedly shown their ability to produce fluent text.
Efficient Sampling of Dependency Structures
Colbourn (1996)'s sampling algorithm has a running time of $\mathcal{O}(N^3)$, which is often greater than the mean hitting time of a directed graph.
Searching for More Efficient Dynamic Programs
To this end, we describe a set of program transformations, a simple metric for assessing the efficiency of a transformed program, and a heuristic search procedure to improve this metric.
A Bayesian Framework for Information-Theoretic Probing
Pimentel et al. (2020) recently analysed probing from an information-theoretic perspective.
Differentiable Subset Pruning of Transformer Heads
Multi-head attention, a collection of several attention mechanisms that independently attend to different parts of the input, is the key ingredient in the Transformer.
Towards Zero-shot Language Modeling
Motivated by this question, we aim at constructing an informative prior over neural weights, in order to adapt quickly to held-out languages in the task of character-level language modeling.
On Finding the K-best Non-projective Dependency Trees
Furthermore, we present a novel extension of the algorithm for decoding the K-best dependency trees of a graph which are subject to a root constraint.
SIGTYP 2021 Shared Task: Robust Spoken Language Identification
While language identification is a fundamental speech and language processing task, for many languages and language families it remains a challenging task.
Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probing
One method of doing so, which is frequently cited to support the claim that models like BERT encode syntax, is called probing; probes are small supervised models trained to extract linguistic information from another model's output.
Modeling the Unigram Distribution
The unigram distribution is the non-contextual probability of finding a specific word form in a corpus.
Is Sparse Attention more Interpretable?
Sparse attention has been claimed to increase model interpretability under the assumption that it highlights influential inputs.
Examining the Inductive Bias of Neural Language Models with Artificial Languages
Since language models are used to model a wide variety of languages, it is natural to ask whether the neural architectures used for the task have inductive biases towards modeling particular types of languages.
Higher-order Derivatives of Weighted Finite-state Machines
In the case of second-order derivatives, our scheme runs in the optimal $\mathcal{O}(A^2 N^4)$ time where $A$ is the alphabet size and $N$ is the number of states.
On Finding the $K$-best Non-projective Dependency Trees
Furthermore, we present a novel extension of the algorithm for decoding the $K$-best dependency trees of a graph which are subject to a root constraint.
Language Model Evaluation Beyond Perplexity
As concrete examples, text generated under the nucleus sampling scheme adheres more closely to the type--token relationship of natural language than text produced using standard ancestral sampling; text from LSTMs reflects the natural language distributions over length, stopwords, and symbols surprisingly well.
A Non-Linear Structural Probe
Probes are models devised to investigate the encoding of knowledge -- e. g. syntactic structure -- in contextual representations.
A Cognitive Regularizer for Language Modeling
The uniform information density (UID) hypothesis, which posits that speakers behaving optimally tend to distribute information uniformly across a linguistic signal, has gained traction in psycholinguistics as an explanation for certain syntactic, morphological, and prosodic choices.
How (Non-)Optimal is the Lexicon?
Examining corpora from 7 typologically diverse languages, we use those upper bounds to quantify the lexicon's optimality and to explore the relative costs of major constraints on natural codes.
Quantifying Gender Bias Towards Politicians in Cross-Lingual Language Models
While the prevalence of large pre-trained language models has led to significant improvements in the performance of NLP systems, recent research has demonstrated that these models inherit societal biases extant in natural language.
Finding Concept-specific Biases in Form--Meaning Associations
It is not a new idea that there are small, cross-linguistic associations between the forms and meanings of words.
BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation
Standard automatic metrics, e. g. BLEU, are not reliable for document-level MT evaluation.
Searching for Search Errors in Neural Morphological Inflection
Yet, on word-level tasks, exact inference of these models reveals the empty string is often the global optimum.
Differentiable Generative Phonology
As the main contribution of our work, we implement the phonological generative system as a neural model differentiable end-to-end, rather than as a set of rules or constraints.
Disambiguatory Signals are Stronger in Word-initial Positions
Psycholinguistic studies of human word processing and lexical access provide ample evidence of the preferred nature of word-initial versus word-final segments, e. g., in terms of attention paid by listeners (greater) or the likelihood of reduction by speakers (lower).
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs
Large-scale pretraining and task-specific fine-tuning is now the standard methodology for many tasks in computer vision and natural language processing.
Morphologically Aware Word-Level Translation
We propose a novel morphologically aware probability model for bilingual lexicon induction, which jointly models lexeme translation and inflectional morphology in a structured way.
SIGTYP 2020 Shared Task: Prediction of Typological Features
Typological knowledge bases (KBs) such as WALS (Dryer and Haspelmath, 2013) contain information about linguistic properties of the world's languages.
Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model
Across languages, multiple consecutive adjectives modifying a noun (e. g. "the big red dog") follow certain unmarked ordering rules.
If beam search is the answer, what was the question?
This implies that the MAP objective alone does not express the properties we desire in text, which merits the question: if beam search is the answer, what was the question?
Please Mind the Root: Decoding Arborescences for Dependency Parsing
The connection between dependency trees and spanning trees is exploited by the NLP community to train and to decode graph-based dependency parsers.
Intrinsic Probing through Dimension Selection
Most modern NLP systems make use of pre-trained contextual representations that attain astonishingly high performance on a variety of tasks.
Speakers Fill Lexical Semantic Gaps with Context
For a language to be clear and efficiently encoded, we posit that the lexical ambiguity of a word type should correlate with how much information context provides about it, on average.
Pareto Probing: Trading Off Accuracy for Complexity
In our contribution to this discussion, we argue for a probe metric that reflects the fundamental trade-off between probe complexity and performance: the Pareto hypervolume.
Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation
Their method takes pre-trained word embeddings as input and attempts to isolate a linear subspace that captures most of the gender bias in the embeddings.
Efficient Computation of Expectations under Spanning Tree Distributions
We propose unified algorithms for the important cases of first-order expectations and second-order expectations in edge-factored, non-projective spanning-tree models.
Best-First Beam Search
Decoding for many NLP tasks requires an effective heuristic algorithm for approximating exact search since the problem of searching the full output space is often intractable, or impractical in many settings.
Metaphor Detection using Context and Concreteness
We report the results of our system on the Metaphor Detection Shared Task at the Second Workshop on Figurative Language Processing 2020.
SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection
Systems were developed using data from 45 languages and just 5 language families, fine-tuned with data from an additional 45 languages and 10 language families (13 in total), and evaluated on all 90 languages.
A Corpus for Large-Scale Phonetic Typology
A major hurdle in data-driven research on typology is having sufficient data in many languages to draw meaningful conclusions.
Applying the Transformer to Character-level Transduction
The transformer has been shown to outperform recurrent neural network-based sequence-to-sequence models in various word-level NLP tasks.
Phonotactic Complexity and its Trade-offs
We present methods for calculating a measure of phonotactic complexity---bits per phoneme---that permits a straightforward cross-linguistic comparison.
It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information
The performance of neural machine translation systems is commonly evaluated in terms of BLEU.
A Tale of a Probe and a Parser
One such probe is the structural probe (Hewitt and Manning, 2019), designed to quantify the extent to which syntactic information is encoded in contextualised word representations.
The Paradigm Discovery Problem
Our benchmark system first makes use of word embeddings and string similarity to cluster forms by cell and by paradigm.
On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbs
We also find that there are statistically significant relationships between the grammatical genders of inanimate nouns and the verbs that take those nouns as direct objects, as indirect objects, and as subjects.
Generalized Entropy Regularization or: There's Nothing Special about Label Smoothing
Prior work has explored directly regularizing the output distributions of probabilistic models to alleviate peaky (i. e. over-confident) predictions, a common sign of overfitting.
UniMorph 3.0: Universal Morphology
The project comprises two major thrusts: a language-independent feature schema for rich morphological annotation and a type-level resource of annotated data in diverse languages realizing that schema.
Predicting Declension Class from Form and Meaning
We find for two Indo-European languages (Czech and German) that form and meaning respectively share significant amounts of information with class (and contribute additional information above and beyond gender).
Information-Theoretic Probing for Linguistic Structure
The success of neural networks on a diverse set of NLP tasks has led researchers to question how much these networks actually ``know'' about natural language.
Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages
In this work, we propose a Bayesian generative model for the space of neural parameters.
Morphological Segmentation Inside-Out
Morphological segmentation has traditionally been modeled with non-hierarchical models, which yield flat segmentations as output.
Weird Inflects but OK: Making Sense of Morphological Generation Errors
We conduct a manual error analysis of the CoNLL-SIGMORPHON Shared Task on Morphological Reinflection.
Quantifying the Semantic Core of Gender Systems
To that end, we use canonical correlation analysis to correlate the grammatical gender of inanimate nouns with an externally grounded definition of their lexical semantics.
The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection
The SIGMORPHON 2019 shared task on cross-lingual transfer and contextual analysis in morphology examined transfer learning of inflection between 100 language pairs, as well as contextual lemmatization and morphosyntactic description in 66 languages.
Don't Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction
Human translators routinely have to translate rare inflections of words - due to the Zipfian distribution of words in a language.
Examining Gender Bias in Languages with Grammatical Gender
Recent studies have shown that word embeddings exhibit gender bias inherited from the training corpora.
It's All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution
An alternative approach is Counterfactual Data Augmentation (CDA), in which a corpus is duplicated and augmented to remove bias, e. g. by swapping all inherently-gendered words in the copy.
Rethinking Phonotactic Complexity
In this work, we propose the use of phone-level language models to estimate phonotactic complexity{---}measured in bits per phoneme{---}which makes cross-linguistic comparison straightforward.
On the Distribution of Deep Clausal Embeddings: A Large Cross-linguistic Study
Embedding a clause inside another ({``}the girl [who likes cars [that run fast]] has arrived{''}) is a fundamental resource that has been argued to be a key driver of linguistic expressiveness.
Morphological Irregularity Correlates with Frequency
We present a study of morphological irregularity.
Uncovering Probabilistic Implications in Typological Knowledge Bases
The study of linguistic typology is rooted in the implications we find between linguistic features, such as the fact that languages with object-verb word ordering tend to have post-positions.
Meaning to Form: Measuring Systematicity as Information
A longstanding debate in semiotics centers on the relationship between linguistic signs and their corresponding semantics: is there an arbitrary relationship between a word form and its meaning, or does some systematic phenomenon pervade?
What Kind of Language Is Hard to Language-Model?
Trying to answer the question of what features difficult languages have in common, we try and fail to reproduce our earlier (Cotterell et al., 2018) observation about morphological complexity and instead reveal far simpler statistics of the data that seem to drive complexity in a much larger sample.
Unsupervised Discovery of Gendered Language through Latent-Variable Modeling
Studying the ways in which language is gendered has long been an area of interest in sociolinguistics.
Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology
Gender stereotypes are manifest in most of the world's languages and are consequently propagated or amplified by NLP systems.
Exact Hard Monotonic Attention for Character-Level Transduction
Our models achieve state-of-the-art performance on morphological inflection.
Contextualization of Morphological Inflection
Critical to natural language generation is the production of correctly inflected text.
Gender Bias in Contextualized Word Embeddings
In this paper, we quantify, analyze and mitigate gender bias exhibited in ELMo's contextualized word vectors.
Combining Sentiment Lexica with a Multi-View Variational Autoencoder
When assigning quantitative labels to a dataset, different methodologies may rely on different scales.
A Simple Joint Model for Improved Contextual Neural Lemmatization
English verbs have multiple forms.
A Probabilistic Generative Model of Linguistic Typology
In the principles-and-parameters framework, the structural features of languages depend on parameters that may be toggled on or off, with a single parameter often dictating the status of multiple features.
On the Idiosyncrasies of the Mandarin Chinese Classifier System
While idiosyncrasies of the Chinese classifier system have been a richly studied topic among linguists (Adams and Conklin, 1973; Erbaugh, 1986; Lakoff, 1986), not much work has been done to quantify them with statistical methods.
UniMorph 2.0: Universal Morphology
The Universal Morphology UniMorph project is a collaborative effort to improve how NLP handles complex morphology across the world's languages.
The CoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection
Apart from extending the number of languages involved in earlier supervised tasks of generating inflected forms, this year the shared task also featured a new second task which asked participants to inflect words in sentential context, similar to a cloze task.
Marrying Universal Dependencies and Universal Morphology
The Universal Dependencies (UD) and Universal Morphology (UniMorph) projects each present schemata for annotating the morphosyntactic details of language.
Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction
Most recent approaches to bilingual dictionary induction find a linear alignment between the word vector spaces of two languages.
Hard Non-Monotonic Attention for Character-Level Transduction
We compare soft and hard non-monotonic attention experimentally and find that the exact algorithm significantly improves performance over the stochastic approximation and outperforms soft attention.
A Discriminative Latent-Variable Model for Bilingual Lexicon Induction
We introduce a novel discriminative latent-variable model for the task of bilingual lexicon induction.
Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate
We suggest that the empirical performance of modern networks warrants a re-examination of their utility in linguistic and cognitive modeling.
A Deep Generative Model of Vowel Formant Typology
What makes some types of languages more probable than others?
On the Complexity and Typology of Inflectional Morphological Systems
We quantify the linguistic complexity of different languages' morphological systems.
Explaining and Generalizing Back-Translation through Wake-Sleep
Back-translation has become a commonly employed heuristic for semi-supervised neural machine translation.
A Structured Variational Autoencoder for Contextual Morphological Inflection
Statistical morphological inflectors are typically trained on fully supervised, type-level data.
Are All Languages Equally Hard to Language-Model?
For general modeling methods applied to diverse languages, a natural question is: how well should we expect our models to work on languages with differing typological profiles?
Unsupervised Disambiguation of Syncretism in Inflected Lexicons
Lexical ambiguity makes it difficult to compute various useful statistics of a corpus.
On the Diachronic Stability of Irregularity in Inflectional Morphology
Many languages' inflectional morphological systems are replete with irregulars, i. e., words that do not seem to follow standard inflectional rules.
Low-Resource Named Entity Recognition with Cross-lingual, Character-Level Neural Conditional Random Fields
Low-resource named entity recognition is still an open problem in NLP.
Cross-Lingual Transfer
Low Resource Named Entity Recognition
+4
Cross-lingual Character-Level Neural Morphological Tagging
Even for common NLP tasks, sufficient supervision is not available in many languages {--} morphological tagging is no exception.
Paradigm Completion for Derivational Morphology
The generation of complex derived word forms has been an overlooked problem in NLP; we fill this gap by applying neural sequence-to-sequence models to the task.
Cross-lingual, Character-Level Neural Morphological Tagging
Even for common NLP tasks, sufficient supervision is not available in many languages -- morphological tagging is no exception.
Frame-Based Continuous Lexical Semantics through Exponential Family Tensor Factorization and Semantic Proto-Roles
We study how different frame annotations complement one another when learning continuous lexical semantics.
CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages
In sub-task 2, systems were given a lemma and some of its specific inflected forms, and asked to complete the inflectional paradigm by predicting all of the remaining inflected forms.
Probabilistic Typology: Deep Generative Models of Vowel Inventories
Linguistic typology studies the range of structures present in human language.