Search Results for author: Danny Merkx

Found 8 papers, 6 papers with code

Modelling word learning and recognition using visually grounded speech

1 code implementation14 Mar 2022 Danny Merkx, Sebastiaan Scholten, Stefan L. Frank, Mirjam Ernestus, Odette Scharenborg

We furthermore investigate whether vector quantisation, a technique for discrete representation learning, aids the model in the discovery and recognition of words.

Representation Learning speech-recognition +1

Seeing the advantage: visually grounding word embeddings to better capture human semantic knowledge

1 code implementation CMCL (ACL) 2022 Danny Merkx, Stefan L. Frank, Mirjam Ernestus

In this paper we create visually grounded word embeddings by combining English text and images and compare them to popular text-based methods, to see if visual information allows our model to better capture cognitive aspects of word meaning.

Grounded language learning Image Retrieval +4

Semantic sentence similarity: size does not always matter

1 code implementation16 Jun 2021 Danny Merkx, Stefan L. Frank, Mirjam Ernestus

This study addresses the question whether visually grounded speech recognition (VGS) models learn to capture sentence semantics without access to any prior linguistic knowledge.

Grounded language learning Image Retrieval +9

Human Sentence Processing: Recurrence or Attention?

1 code implementation NAACL (CMCL) 2021 Danny Merkx, Stefan L. Frank

Recurrent neural networks (RNNs) have long been an architecture of interest for computational models of human sentence processing.

Language Modelling Retrieval +1

Learning semantic sentence representations from visually grounded language without lexical knowledge

1 code implementation27 Mar 2019 Danny Merkx, Stefan Frank

The system achieves state-of-the-art results on several of these benchmarks, which shows that a system trained solely on multimodal data, without assuming any word representations, is able to capture sentence level semantics.

Grounded language learning Learning Semantic Representations +7

Cannot find the paper you are looking for? You can Submit a new open access paper.