Search Results for author:
Found 31 papers, 10 papers with code
Multi-SimLex: A Large-Scale Evaluation of Multilingual and Crosslingual Lexical Semantic Similarity
We introduce Multi-SimLex, a large-scale lexical resource and evaluation benchmark covering data sets for 12 typologically diverse languages, including major languages (e. g., Mandarin Chinese, Spanish, Russian) as well as less-resourced ones (e. g., Welsh, Kiswahili).
MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer
While offering (1) improved fine-tuning efficiency (by a factor of around 50 in our experiments), (2) a smaller parameter budget, and (3) increased language coverage, MAD-G remains competitive with more expensive methods for language-specific adapter training across the board.
Model Merging by Uncertainty-Based Gradient Matching
Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail?
Distilling Efficient Language-Specific Models for Cross-Lingual Transfer
Specifically, we use a two-phase distillation approach, termed BiStil: (i) the first phase distils a general bilingual model from the MMT, while (ii) the second, task-specific phase sparsely fine-tunes the bilingual "student" model using a task-tuned variant of the original MMT as its "teacher".
Modular Deep Learning
Modular deep learning has emerged as a promising solution to these challenges.
Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders
In this work, we probe SEs for the amount of cross-lingual lexical knowledge stored in their parameters, and compare them against the original multilingual LMs.
Cross-Lingual Dialogue Dataset Creation via Outline-Based Generation
Through this process we annotate a new large-scale dataset for training and evaluation of multilingual and cross-lingual ToD systems.
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages
Our benchmark enables the evaluation of multilingual multimodal models for transfer learning, not only in a zero-shot setting, but also in newly defined few-shot learning setups.
Composable Sparse Fine-Tuning for Cross-Lingual Transfer
Both these masks can then be composed with the pretrained model.
Visually Grounded Reasoning across Languages and Cultures
The design of widespread vision-and-language datasets and pre-trained encoders directly adopts, or draws inspiration from, the concepts and images of ImageNet.
Ranked #1 on
Zero-Shot Cross-Lingual Transfer
on MaRVL
Towards Zero-shot Language Modeling
Motivated by this question, we aim at constructing an informative prior over neural weights, in order to adapt quickly to held-out languages in the task of character-level language modeling.
LexFit: Lexical Fine-Tuning of Pretrained Language Models
Inspired by prior work on semantic specialization of static word embedding (WE) models, we show that it is possible to expose and enrich lexical knowledge from the LMs, that is, to specialize them to serve as effective and universal {``}decontextualized{''} word encoders even when fed input words {``}in isolation{''} (i. e., without any context).
Modelling Latent Translations for Cross-Lingual Transfer
To remedy this, we propose a new technique that integrates both steps of the traditional pipeline (translation and classification) into a single model, by treating the intermediate translations as a latent random variable.
Minimax and Neyman-Pearson Meta-Learning for Outlier Languages
In fact, under a decision-theoretic framework, MAML can be interpreted as minimising the expected risk across training languages (with a uniform prior), which is known as Bayes criterion.
Differentiable Generative Phonology
As the main contribution of our work, we implement the phonological generative system as a neural model differentiable end-to-end, rather than as a set of rules or constraints.
Probing Pretrained Language Models for Lexical Semantics
The success of large pretrained language models (LMs) such as BERT and RoBERTa has sparked interest in probing their representations, in order to unveil what types of knowledge they implicitly capture.
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning
In order to simulate human language capacity, natural language processing systems must be able to reason about the dynamics of everyday situations, including their possible causes and effects.
Ranked #3 on
Cross-Lingual Transfer
on XCOPA
(using extra training data)
Internal and external pressures on language emergence: least effort, object constancy and frequency
In previous work, artificial agents were shown to achieve almost perfect accuracy in referential games where they have to communicate to identify images.
Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similarity
We introduce Multi-SimLex, a large-scale lexical resource and evaluation benchmark covering datasets for 12 typologically diverse languages, including major languages (e. g., Mandarin Chinese, Spanish, Russian) as well as less-resourced ones (e. g., Welsh, Kiswahili).
Cross-lingual Semantic Specialization via Lexical Relation Induction
Semantic specialization integrates structured linguistic knowledge from external resources (such as lexical relations in WordNet) into pretrained distributional vectors in the form of constraints.
Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity
In this work, we complement such distributional knowledge with external lexical knowledge, that is, we integrate the discrete knowledge on word-level semantic similarity into pretraining.
Specializing Distributional Vectors of All Words for Lexical Entailment
Semantic specialization methods fine-tune distributional word vectors using lexical knowledge from external resources (e. g. WordNet) to accentuate a particular relation between words.
On the Relation between Linguistic Typology and (Limitations of) Multilingual Language Modeling
A key challenge in cross-lingual NLP is developing general language-independent architectures that are equally applicable to any language.
Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specialization
Our adversarial post-specialization method propagates the external lexical knowledge to the full distributional space.
Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing
Linguistic typology aims to capture structural and semantic variation across the world's languages.
Isomorphic Transfer of Syntactic Structures in Cross-Lingual NLP
The transfer or share of knowledge between languages is a potential solution to resource scarcity in NLP.
Decoding Sentiment from Distributed Representations of Sentences
Distributed representations of sentences have been developed recently to represent their meaning as real-valued vectors.
Event-Related Features in Feedforward Neural Networks Contribute to Identifying Causal Relations in Discourse
Causal relations play a key role in information extraction and reasoning.
Distributed Representations of Lexical Sets and Prototypes in Causal Alternation Verbs
Lexical sets contain the words filling an argument slot of a verb, and are in part determined by selectional preferences.
Differentia compositionem facit. A Slower-Paced and Reliable Parser for Latin
The Index Thomisticus Treebank is the largest available treebank for Latin; it contains Medieval Latin texts by Thomas Aquinas.
