Search Results for author: Ece Takmaz

Found 9 papers, 4 papers with code

Less Descriptive yet Discriminative: Quantifying the Properties of Multimodal Referring Utterances via CLIP

1 code implementation CMCL (ACL) 2022 Ece Takmaz, Sandro Pezzelle, Raquel Fernández

In this work, we use a transformer-based pre-trained multimodal model, CLIP, to shed light on the mechanisms employed by human speakers when referring to visual entities.


Team DMG at CMCL 2022 Shared Task: Transformer Adapters for the Multi- and Cross-Lingual Prediction of Human Reading Behavior

no code implementations CMCL (ACL) 2022 Ece Takmaz

In this paper, we present the details of our approaches that attained the second place in the shared task of the ACL 2022 Cognitive Modeling and Computational Linguistics Workshop.

Decoding Emotions in Abstract Art: Cognitive Plausibility of CLIP in Recognizing Color-Emotion Associations

no code implementations10 May 2024 Hanna-Sophia Widhoelzl, Ece Takmaz

We employ a dataset comprising images with associated emotion labels and textual rationales of these labels provided by human annotators.

Emotion Classification

Describing Images $\textit{Fast and Slow}$: Quantifying and Predicting the Variation in Human Signals during Visuo-Linguistic Processes

1 code implementation2 Feb 2024 Ece Takmaz, Sandro Pezzelle, Raquel Fernández

There is an intricate relation between the properties of an image and how humans behave while describing the image.

Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind

1 code implementation31 May 2023 Ece Takmaz, Nicolo' Brandizzi, Mario Giulianelli, Sandro Pezzelle, Raquel Fernández

Inspired by psycholinguistic theories, we endow our speaker with the ability to adapt its referring expressions via a simulation module that monitors the effectiveness of planned utterances from the listener's perspective.

Language Modelling Open-Ended Question Answering +1

The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue

no code implementations ACL 2019 Janosch Haber, Tim Baumgärtner, Ece Takmaz, Lieke Gelderloos, Elia Bruni, Raquel Fernández

This paper introduces the PhotoBook dataset, a large-scale collection of visually-grounded, task-oriented dialogues in English designed to investigate shared dialogue history accumulating during conversation.

Evaluating the Representational Hub of Language and Vision Models

no code implementations WS 2019 Ravi Shekhar, Ece Takmaz, Raquel Fernández, Raffaella Bernardi

The multimodal models used in the emerging field at the intersection of computational linguistics and computer vision implement the bottom-up processing of the `Hub and Spoke' architecture proposed in cognitive science to represent how the brain processes and combines multi-sensory inputs.

Question Answering Visual Question Answering

Cannot find the paper you are looking for? You can Submit a new open access paper.