no code implementations • 30 Jan 2024 • Kurt Micallef, Nizar Habash, Claudia Borg, Fadhl Eryani, Houda Bouamor
Although multilingual language models exhibit impressive cross-lingual transfer capabilities on unseen languages, the performance on downstream tasks is impacted when there is a script disparity with the languages used in the multilingual model's pre-training data.
1 code implementation • DeepLo 2022 • Kurt Micallef, Albert Gatt, Marc Tanti, Lonneke van der Plas, Claudia Borg
We also present a newly created corpus for Maltese, and determine the effect that the pre-training data size and domain have on the downstream performance.