Finding Structural Knowledge in Multimodal-BERT

1 code implementation • ACL 2022 • Victor Milewski, Miryam de Lhoneux, Marie-Francine Moens

In this work, we investigate the knowledge learned in the embeddings of multimodal-BERT models.

Giving Commands to a Self-Driving Car: How to Deal with Uncertain Situations?

1 code implementation • 8 Jun 2021 • Thierry Deruyttere, Victor Milewski, Marie-Francine Moens

This paper proposes a model that detects uncertain situations when a command is given and finds the visual objects causing it.

Referring Expression Self-Driving Cars

Are scene graphs good enough to improve Image Captioning?

1 code implementation • Asian Chapter of the Association for Computational Linguistics 2020 • Victor Milewski, Marie-Francine Moens, Iacer Calixto

Overall, we find no significant difference between models that use scene graph features and models that only use object detection features across different captioning metrics, which suggests that existing scene graph generation models are still too noisy to be useful in image captioning.

Graph Attention Graph Generation +5