Search Results for author:
Found 7 papers, 4 papers with code
Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models
Recent advances in training multilingual language models on large datasets seem to have shown promising results in knowledge transfer across languages and achieve high performance on downstream tasks.
An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning
Parameter-efficient fine-tuning approaches have recently garnered a lot of attention.
Looking at the Overlooked: An Analysis on the Word-Overlap Bias in Natural Language Inference
It has been shown that NLI models are usually biased with respect to the word-overlap between premise and hypothesis; they take this feature as a primary cue for predicting the entailment label.
On the Importance of Data Size in Probing Fine-tuned Models
The analysis also reveals that larger training data mainly affects higher layers, and that the extent of this change is a factor of the number of iterations updating the model during fine-tuning rather than the diversity of the training samples.
An Isotropy Analysis in the Multilingual BERT Embedding Space
However, we observe no such dimensions in the multilingual BERT.
How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropy
It is widely accepted that fine-tuning pre-trained language models usually brings about performance improvements in downstream tasks.
A Cluster-based Approach for Improving Isotropy in Contextual Embedding Space
Based on this observation, we propose a local cluster-based method to address the degeneration issue in contextual embedding spaces.
