Search Results for author:
Found 122 papers, 57 papers with code
Empirical Gaussian priors for cross-lingual transfer learning
We instead propose to use the $k$ source language models to estimate the parameters of a Gaussian prior for learning new POS taggers.
Improving sentence compression by learning to predict gaze
We show how eye-tracking corpora can be used to improve sentence compression models, presenting a novel multi-task learning algorithm based on multi-layer LSTMs.
Ranked #5 on
Sentence Compression
on Google Dataset
Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss
Bidirectional long short-term memory (bi-LSTM) networks have recently proven successful for various NLP sequence modeling tasks, but little is known about their reliance to input representations, target languages, data set size, and label noise.
Ranked #4 on
Part-Of-Speech Tagging
on UD
A Strong Baseline for Learning Cross-Lingual Word Embeddings from Sentence Alignments
While cross-lingual word embeddings have been studied extensively in recent years, the qualitative differences between the different algorithms remain vague.
Improving historical spelling normalization with bi-directional LSTMs and multi-task learning
Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data.
Spikes as regularizers
We present a confidence-based single-layer feed-forward learning algorithm SPIRAL (Spike Regularized Adaptive Learning) relying on an encoding of activation spikes.
Cross-Lingual Dependency Parsing with Late Decoding for Truly Low-Resource Languages
In cross-lingual dependency annotation projection, information is often lost during transfer because of early decoding.
Cross-lingual RST Discourse Parsing
Discourse parsing is an integral part of understanding information flow and argumentative structure in documents.
Ranked #10 on
Discourse Parsing
on RST-DT
Parsing Universal Dependencies without training
We propose UDP, the first training-free parser for Universal Dependencies (UD).
Identifying beneficial task relations for multi-task learning in deep neural networks
Multi-task learning (MTL) in deep neural networks for NLP has recently received increasing interest due to some compelling benefits, including its potential to efficiently regularize models and to reduce the need for labeled data.
Multi-Task Learning of Keyphrase Boundary Classification
Keyphrase boundary classification (KBC) is the task of detecting keyphrases in scientific articles and labelling them with respect to predefined types.
Cross-lingual and cross-domain discourse segmentation of entire documents
Discourse segmentation is a crucial step in building end-to-end discourse parsers.
Latent Multi-task Architecture Learning
In practice, however, MTL involves searching an enormous space of possible parameter sharing architectures to find (a) the layers or subspaces that benefit from sharing, (b) the appropriate amount of sharing, and (c) the appropriate relative weights of the different task losses.
A Survey Of Cross-lingual Word Embedding Models
Cross-lingual representations of words enable us to reason about word meaning in multilingual contexts and are a key facilitator of cross-lingual transfer when developing natural language processing models for low-resource languages.
Is writing style predictive of scientific fraud?
The problem of detecting scientific fraud using machine learning was recently introduced, with initial, positive results from a model taking into account various general indicators.
Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm
NLP tasks are often limited by scarcity of manually annotated data.
Ranked #1 on
Transfer Learning
on Amazon Review Polarity
Multi-task Learning of Pairwise Sequence Classification Tasks Over Disparate Label Spaces
We combine multi-task learning and semi-supervised learning by inducing a joint embedding space between disparate label spaces and learning transfer functions between label embeddings, enabling us to jointly leverage unlabelled data and auxiliary, annotated datasets.
Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens
Can attention- or gradient-based visualization techniques be used to infer token-level labels for binary sequence tagging problems, using networks trained only on sentence-level labels?
On the Limitations of Unsupervised Bilingual Dictionary Induction
Unsupervised machine translation---i. e., not assuming any cross-lingual supervision signal, whether a dictionary, translations, or comparable corpora---seems impossible, but nevertheless, Lample et al. (2018) recently proposed a fully unsupervised machine translation (MT) model.
Parameter sharing between dependency parsers for related languages
We find that sharing transition classifier parameters always helps, whereas the usefulness of sharing word and/or character LSTM parameters varies.
A strong baseline for question relevancy ranking
The best systems at the SemEval-16 and SemEval-17 community question answering shared tasks -- a task that amounts to question relevancy ranking -- involve complex pipelines and manual feature engineering.
A Discriminative Latent-Variable Model for Bilingual Lexicon Induction
We introduce a novel discriminative latent variable model for bilingual lexicon induction.
Generalizing Procrustes Analysis for Better Bilingual Dictionary Induction
Most recent approaches to bilingual dictionary induction find a linear alignment between the word vector spaces of two languages.
Nightmare at test time: How punctuation prevents parsers from generalizing
Punctuation is a strong indicator of syntactic structure, and parsers trained on text with punctuation often rely heavily on this signal.
Why is unsupervised alignment of English embeddings from different algorithms so hard?
This paper presents a challenge to the community: Generative adversarial networks (GANs) can perfectly align independent English word embeddings induced using the same algorithm, based on distributional information alone; but fails to do so, for two different embeddings algorithms.
Jointly Learning to Label Sentences and Tokens
Learning to construct text representations in end-to-end systems can be difficult, as natural languages are highly compositional and task-specific annotated datasets are often limited in size.
Ranked #1 on
Grammatical Error Detection
on JFLEG
Better, Faster, Stronger Sequence Tagging Constituent Parsers
Combining these techniques, we clearly surpass the performance of sequence tagging constituent parsers on the English and Chinese Penn Treebanks, and reduce their parsing time even further.
Few-Shot and Zero-Shot Learning for Historical Text Normalization
Historical text normalization often relies on small training datasets.
Issue Framing in Online Discussion Fora
In online discussion fora, speakers often make arguments for or against something, say birth control, by highlighting certain aspects of the topic.
Multi-Task Semantic Dependency Parsing with Policy Gradient for Learning Easy-First Strategies
In Semantic Dependency Parsing (SDP), semantic relations form directed acyclic graphs, rather than trees.
Model-based annotation of coreference
Humans do not make inferences over texts, but over models of what texts are about.
X-WikiRE: A Large, Multilingual Resource for Relation Extraction as Machine Comprehension
Although the vast majority of knowledge bases KBs are heavily biased towards English, Wikipedias do cover very different topics in different languages.
Ellipsis Resolution as Question Answering: An Evaluation
Most, if not all forms of ellipsis (e. g., so does Mary) are similar to reading comprehension questions (what does Mary do), in that in order to resolve them, we need to identify an appropriate text span in the preceding discourse.
Higher-order Comparisons of Sentence Encoder Representations
Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different measurement modalities (e. g., fMRI, electrophysiology, behavior).
Rewarding Coreference Resolvers for Being Consistent with World Knowledge
Unresolved coreference is a bottleneck for relation extraction, and high-quality coreference resolvers may produce an output that makes it a lot easier to extract knowledge triples.
A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages
Parsers are available for only a handful of the world's languages, since they require lots of training data.
Lost in Evaluation: Misleading Benchmarks for Bilingual Dictionary Induction
We study the composition and quality of the test sets for five diverse languages from this dataset, with concerning findings: (1) a quarter of the data consists of proper nouns, which can be hardly indicative of BDI performance, and (2) there are pervasive gaps in the gold-standard targets.
Domain Transfer in Dialogue Systems without Turn-Level Supervision
Task oriented dialogue systems rely heavily on specialized dialogue state tracking (DST) modules for dynamically predicting user intent throughout the conversation.
Retrieval-based Goal-Oriented Dialogue Generation
Most research on dialogue has focused either on dialogue generation for openended chit chat or on state tracking for goal-directed dialogue.
What Do You Mean `Why?': Resolving Sluices in Conversations
In conversation, we often ask one-word questions such as `Why?'
Comparing Unsupervised Word Translation Methods Step by Step
Cross-lingual word vector space alignment is the task of mapping the vocabularies of two languages into a shared semantic space, which can be used for dictionary induction, unsupervised machine translation, and transfer learning.
Are All Good Word Vector Spaces Isomorphic?
Existing algorithms for aligning cross-lingual word vector spaces assume that vector spaces are approximately isomorphic.
Weakly Supervised POS Taggers Perform Poorly on Truly Low-Resource Languages
Part-of-speech (POS) taggers for low-resource languages which are exclusively based on various forms of weak supervision - e. g., cross-lingual transfer, type-level supervision, or a combination thereof - have been reported to perform almost as well as supervised ones.
We Need to Talk About Random Splits
We argue that random splits, like standard splits, lead to overly optimistic performance estimates.
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations
Large-scale pretrained language models are the major driving force behind recent improvements in performance on the Winograd Schema Challenge, a widely employed test of common sense reasoning ability.
Worst-Case-Aware Curriculum Learning for Zero and Few Shot Transfer
Multi-task transfer learning based on pre-trained language encoders achieves state-of-the-art performance across a range of tasks.
Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Bias
The one-sided focus on English in previous studies of gender bias in NLP misses out on opportunities in other languages: English challenge datasets such as GAP and WinoGender highlight model preferences that are "hallucinatory", e. g., disambiguating gender-ambiguous occurrences of 'doctor' as male doctors.
Joint Semantic Analysis with Document-Level Cross-Task Coherence Rewards
Coreference resolution and semantic role labeling are NLP tasks that capture different aspects of semantics, indicating respectively, which expressions refer to the same entity, and what semantic roles expressions serve in the sentence.
Grammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses
Evaluation of grammatical error correction (GEC) systems has primarily focused on essays written by non-native learners of English, which however is only part of the full spectrum of GEC applications.
Attention Can Reflect Syntactic Structure (If You Let It)
Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head attention mechanism.
Does injecting linguistic structure into language models lead to better alignment with brain recordings?
We evaluate across two fMRI datasets whether language models align better with brain recordings, if their attention is biased by annotations from syntactic or semantic formalisms.
Do End-to-End Speech Recognition Models Care About Context?
We test this hypothesis by measuring temporal context sensitivity and evaluate how the models perform when we constrain the amount of contextual information in the audio input.
Replicating and Extending "Because Their Treebanks Leak": Graph Isomorphism, Covariants, and Parser Performance
S{\o}gaard (2020) obtained results suggesting the fraction of trees occurring in the test data isomorphic to trees in the training set accounts for a non-trivial variation in parser performance.
Minimax and Neyman-Pearson Meta-Learning for Outlier Languages
In fact, under a decision-theoretic framework, MAML can be interpreted as minimising the expected risk across training languages (with a uniform prior), which is known as Bayes criterion.
John praised Mary because he? Implicit Causality Bias and Its Interaction with Explicit Cues in LMs
We hypothesize that the temporary challenge humans face in integrating the two contradicting signals, one from the lexical semantics of the verb, one from the sentence-level semantics, would be reflected in higher error rates for models on tasks dependent on causal links.
Itihasa: A large-scale corpus for Sanskrit to English translation
This work introduces Itihasa, a large-scale translation dataset containing 93, 000 pairs of Sanskrit shlokas and their English translations.
Ranked #1 on
Machine Translation
on Itihasa
On the Interaction of Belief Bias and Explanations
A myriad of explainability methods have been proposed in recent years, but there is little consensus on how to evaluate them.
The Impact of Positional Encodings on Multilingual Compression
In order to preserve word-order information in a non-autoregressive setting, transformer architectures tend to include positional knowledge, by (for instance) adding positional encodings to token embeddings.
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color
Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases -- (Paris, Capital, France).
On Language Models for Creoles
Creole languages such as Nigerian Pidgin English and Haitian Creole are under-resourced and largely ignored in the NLP literature.
Do Language Models Know the Way to Rome?
The global geometry of language models is important for a range of applications, but language model probes tend to evaluate rather local relations, for which ground truths are easily obtained.
Evaluation of Summarization Systems across Gender, Age, and Race
Summarization systems are ultimately evaluated by human annotators and raters.
Dynamic Forecasting of Conversation Derailment
This approach shows mixed results: in a high-quality data setting, a longer average forecast horizon can be achieved at the cost of a small drop in F1; in a low-quality data setting, however, dynamic training propagates the noise and is highly detrimental to performance.
Revisiting Methods for Finding Influential Examples
Several instance-based explainability methods for finding influential training examples for test-time decisions have been proposed recently, including Influence Functions, TraceIn, Representer Point Selection, Grad-Dot, and Grad-Cos.
Do We Still Need Automatic Speech Recognition for Spoken Language Understanding?
We compare learned speech features from wav2vec 2. 0, state-of-the-art ASR transcripts, and the ground truth text as input for a novel speech-based named entity recognition task, a cardiac arrest detection task on real-world emergency calls and two existing SLU benchmarks.
Ranked #7 on
Spoken Language Understanding
on Fluent Speech Commands
(using extra training data)
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+8
Exploring the Unfairness of DP-SGD Across Settings
We establish a negative, logarithmic correlation between privacy and fairness in the case of linear classification and robust deep learning.
The Impact of Differential Privacy on Group Disparity Mitigation
The performance cost of differential privacy has, for some applications, been shown to be higher for minority groups; fairness, conversely, has been shown to disproportionally compromise the privacy of members of such groups.
FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing
We present a benchmark suite of four datasets for evaluating the fairness of pre-trained language models and the techniques used to fine-tune them for downstream tasks.
Improved Multi-label Classification under Temporal Concept Drift: Rethinking Group-Robust Algorithms in a Label-Wise Setting
In document classification for, e. g., legal and biomedical text, we often deal with hundreds of classes, including very infrequent ones, as well as temporal concept drift caused by the influence of real world events, e. g., policy changes, conflicts, or pandemics.
Zero-Shot Dependency Parsing with Worst-Case Aware Automated Curriculum Learning
Large multilingual pretrained language models such as mBERT and XLM-RoBERTa have been found to be surprisingly effective for cross-lingual transfer of syntactic parsing models (Wu and Dredze 2019), but only between related languages.
Challenges and Strategies in Cross-Cultural NLP
Various efforts in the Natural Language Processing (NLP) community have been made to accommodate linguistic diversity and serve speakers of many different languages.
Word Order Does Matter (And Shuffled Language Models Know It)
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the importance of word order information.
Factual Consistency of Multilingual Pretrained Language Models
However, for that, we need to know how reliable this knowledge is, and recent work has shown that monolingual English language models lack consistency when predicting factual knowledge, that is, they fill-in-the-blank differently for paraphrases describing the same fact.
How Conservative are Language Models? Adapting to the Introduction of Gender-Neutral Pronouns
Gender-neutral pronouns have recently been introduced in many languages to a) include non-binary people and b) as a generic singular.
Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks
Logical approaches to representing language have developed and evaluated computational models of quantifier words since the 19th century, but today's NLU models still struggle to capture their semantics.
Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?
We investigate whether self-attention in large-scale pre-trained language models is as predictive of human eye fixation patterns during task-reading as classical cognitive models of human attention.
Evaluating Deep Taylor Decomposition for Reliability Assessment in the Wild
We argue that we need to evaluate model interpretability methods 'in the wild', i. e., in situations where professionals make critical decisions, and models can potentially assist them.
QLEVR: A Diagnostic Dataset for Quantificational Language and Elementary Visual Reasoning
Synthetic datasets have successfully been used to probe visual question-answering datasets for their reasoning abilities.
Ranked #1 on
Visual Question Answering (VQA)
on QLEVR
What a Creole Wants, What a Creole Needs
We demonstrate, through conversations with Creole experts and surveys of Creole-speaking communities, how the things needed from language technology can change dramatically from one language to another, even when the languages are considered to be very similar to each other, as with Creoles.
Ancestor-to-Creole Transfer is Not a Walk in the Park
We aim to learn language models for Creole languages for which large volumes of data are not readily available, and therefore explore the potential transfer from ancestor languages (the 'Ancestry Transfer Hypothesis').
Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold
Most work targeting multilinguality, for example, considers only accuracy; most work on fairness or interpretability considers only English; and so on.
Are Pretrained Multilingual Models Equally Fair Across Languages?
Pretrained multilingual language models can help bridge the digital language divide, enabling high-quality NLP models for lower resourced languages.
Multi hash embeddings in spaCy
In this technical report we lay out a bit of history and introduce the embedding methods in spaCy in detail.
Implications of the Convergence of Language and Vision Model Geometries
Large-scale pretrained language models (LMs) are said to ``lack the ability to connect [their] utterances to the world'' (Bender and Koller, 2020).
A Two-Sided Discussion of Preregistration of NLP Research
Van Miltenburg et al. (2021) suggest NLP research should adopt preregistration to prevent fishing expeditions and to promote publication of negative results.
WebQAmGaze: A Multilingual Webcam Eye-Tracking-While-Reading Dataset
We present WebQAmGaze, a multilingual low-cost eye-tracking-while-reading dataset, designed as the first webcam-based eye-tracking corpus of reading to support the development of explainable computational language processing models.
On the Independence of Association Bias and Empirical Fairness in Language Models
To this end, we first provide a thought experiment, showing how association bias and empirical fairness can be completely orthogonal.
LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development
To this end, we release a multinational English legal corpus (LeXFiles) and a legal knowledge probing benchmark (LegalLAMA) to facilitate training and detailed analysis of legal-oriented PLMs.
Private Meeting Summarization Without Performance Loss
Meeting summarization has an enormous business potential, but in addition to being a hard problem, roll-out is challenged by privacy concerns.
What does the Failure to Reason with "Respectively" in Zero/Few-Shot Settings Tell Us about Language Models?
Humans can effortlessly understand the coordinate structure of sentences such as "Niels Bohr and Kurt Cobain were born in Copenhagen and Seattle, respectively".
Being Right for Whose Right Reasons?
Explainability methods are used to benchmark the extent to which model predictions align with human rationales i. e., are 'right for the right reasons'.
Structural Similarities Between Language Models and Neural Response Measurements
Human language processing is also opaque, but neural response measurements can provide (noisy) recordings of activation during listening or reading, from which we can extract similar representations of words and phrases.
Mapping Brains with Language Models: A Survey
Over the years, many researchers have seemingly made the same observation: Brain and language model activations exhibit some structural similarities, enabling linear partial mappings between features extracted from neural recordings and computational language models.
Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models
Language models such as mBERT, XLM-R, and BLOOM aim to achieve multilingual generalization or compression to facilitate transfer to a large number of (potentially unseen) languages.
Large language models converge toward human-like concept organization
Large language models show human-like performance in knowledge extraction, reasoning and dialogue, but it remains controversial whether this performance is best explained by memorization and pattern matching, or whether it reflects human-like inferential semantics and world knowledge.
Copyright Violations and Large Language Models
Language models may memorize more than just facts, including entire chunks of texts seen during training.
CreoleVal: Multilingual Multitask Benchmarks for Creoles
Creoles represent an under-explored and marginalized group of languages, with few available resources for NLP research.
Evaluating Webcam-based Gaze Data as an Alternative for Human Rationale Annotations
Rationales in the form of manually annotated input spans usually serve as ground truth when evaluating explainability methods in NLP.
Word Order and World Knowledge
Word order is an important concept in natural language, and in this work, we study how word order affects the induction of world knowledge from raw text using language models.
Comprehensive Reassessment of Large-Scale Evaluation Outcomes in LLMs: A Multifaceted Statistical Approach
Evaluations have revealed that factors such as scaling, training types, architectures and other factors profoundly impact the performance of LLMs.
MuLan: A Study of Fact Mutability in Language Models
Facts are subject to contingencies and can be true or false in different circumstances.
Does Instruction Tuning Make LLMs More Consistent?
The purpose of instruction tuning is enabling zero-shot performance, but instruction tuning has also been shown to improve chain-of-thought reasoning and value alignment (Si et al., 2023).
The Sensitivity of Annotator Bias to Task Definitions in Argument Mining
NLP models are dependent on the data they are trained on, including how this data is annotated.
Moses and the Character-Based Random Babbling Baseline: CoAStaL at AmericasNLP 2021 Shared Task
We evaluated a range of neural machine translation techniques developed specifically for low-resource scenarios.
Common Sense Bias in Semantic Role Labeling
Large-scale language models such as ELMo and BERT have pushed the horizon of what is possible in semantic role labeling (SRL), solving the out-of-vocabulary problem and enabling end-to-end systems, but they have also introduced significant biases.
Resources and Evaluations for Danish Entity Resolution
Automatic coreference resolution is understudied in Danish even though most of the Danish Dependency Treebank (Buch-Kromann, 2003) is annotated with coreference relations.
Clustering Monolingual Vocabularies to Improve Cross-Lingual Generalization
This work explores the idea of learning multilingual language models based on clustering of monolingual segments.
A Multilingual Benchmark for Probing Negation-Awareness with Minimal Pairs
Negation is one of the most fundamental concepts in human cognition and language, and several natural language inference (NLI) probes have been designed to investigate pretrained language models’ ability to detect and reason with negation.
Word Order Does Matter and Shuffled Language Models Know It
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the importance of word order information.
Multilingual Negation Scope Resolution for Clinical Text
Negation scope resolution is key to high-quality information extraction from clinical texts, but so far, efforts to make encoders used for information extraction negation-aware have been limited to English.
Locke’s Holiday: Belief Bias in Machine Reading
I highlight a simple failure mode of state-of-the-art machine reading systems: when contexts do not align with commonly shared beliefs.
Sociolectal Analysis of Pretrained Language Models
Using data from English cloze tests, in which subjects also self-reported their gender, age, education, and race, we examine performance differences of pretrained language models across demographic groups, defined by these (protected) attributes.
The Effect of Round-Trip Translation on Fairness in Sentiment Analysis
Sentiment analysis systems have been shown to exhibit sensitivity to protected attributes.
Guideline Bias in Wizard-of-Oz Dialogues
NLP models struggle with generalization due to sampling and annotator bias.
How far can we get with one GPU in 100 hours? CoAStaL at MultiIndicMT Shared Task
This work shows that competitive translation results can be obtained in a constrained setting by incorporating the latest advances in memory and compute optimization.
