Contrastive Multimodal Learning for Emergence of Graphical Sensory-Motor Communication

no code implementations • 3 Oct 2022 • Tristan Karch, Yoann Lemesle, Romain Laroche, Clément Moulin-Frier, Pierre-Yves Oudeyer

In this paper, we investigate whether artificial agents can develop a shared language in an ecological setting where communication relies on a sensory-motor channel.

Language-biased image classification: evaluation based on semantic representations

1 code implementation • 26 Jan 2022 • Yoann Lemesle, Masataka Sawayama, Guillermo Valle-Perez, Maxime Adolphe, Hélène Sauzéon, Pierre-Yves Oudeyer

This suggests that the semantic word representation in the CLIP visual processing is not shared with the image representation, although the word representation strongly dominates for word-embedded images.

Classification Image Classification

Evaluating Language-biased image classification based on semantic compositionality

no code implementations • ICLR 2022 • Yoann Lemesle, Masataka Sawayama, Guillermo Valle-Perez, Maxime Adolphe, Hélène Sauzéon, Pierre-Yves Oudeyer

This suggests that the semantic word representation in the CLIP visual processing is not shared with the image representation, although the word representation strongly dominates for word-embedded images.

Classification Image Classification