Search Results for author: Leanne Nortje

Found 9 papers, 5 papers with code

Visually Grounded Speech Models have a Mutual Exclusivity Bias

no code implementations20 Mar 2024 Leanne Nortje, Dan Oneaţă, Yevgen Matusevych, Herman Kamper

To simulate prior acoustic and visual knowledge, we experiment with several initialisation strategies using pretrained speech and vision networks.

Visually grounded few-shot word learning in low-resource settings

no code implementations20 Jun 2023 Leanne Nortje, Dan Oneata, Herman Kamper

We propose an approach that can work on natural word-image pairs but with less examples, i. e. fewer shots, and then illustrate how this approach can be applied for multimodal few-shot learning in a real low-resource language, Yor\`ub\'a.

Few-Shot Learning

Visually grounded few-shot word acquisition with fewer shots

no code implementations25 May 2023 Leanne Nortje, Benjamin van Niekerk, Herman Kamper

Our approach involves using the given word-image example pairs to mine new unsupervised word-image training pairs from large collections of unlabelled speech and images.

Towards visually prompted keyword localisation for zero-resource spoken languages

1 code implementation12 Oct 2022 Leanne Nortje, Herman Kamper

We formalise this task and call it visually prompted keyword localisation (VPKL): given an image of a keyword, detect and predict where in an utterance the keyword occurs.

Direct multimodal few-shot learning of speech and images

1 code implementation10 Dec 2020 Leanne Nortje, Herman Kamper

We propose direct multimodal few-shot models that learn a shared embedding space of spoken words and images from only a few paired examples.

Few-Shot Learning Transfer Learning

Unsupervised vs. transfer learning for multimodal one-shot matching of speech and images

1 code implementation14 Aug 2020 Leanne Nortje, Herman Kamper

Here we compare transfer learning to unsupervised models trained on unlabelled in-domain data.

Transfer Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.