Search Results for author: Raquel Fernández

Found 40 papers, 23 papers with code

Analysing Human Strategies of Information Transmission as a Function of Discourse Context

1 code implementation CoNLL (EMNLP) 2021 Mario Giulianelli, Raquel Fernández

Speakers are thought to use rational information transmission strategies for efficient communication (Genzel and Charniak, 2002; Aylett and Turk, 2004; Jaeger and Levy, 2007).


AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models

no code implementations *SEM (NAACL) 2022 Samuel Ryb, Mario Giulianelli, Arabella Sinclair, Raquel Fernández

We investigate the extent to which pre-trained language models acquire analytical and deductive logical reasoning capabilities as a side effect of learning word prediction.

Language Modelling Logical Reasoning +1

Less Descriptive yet Discriminative: Quantifying the Properties of Multimodal Referring Utterances via CLIP

1 code implementation CMCL (ACL) 2022 Ece Takmaz, Sandro Pezzelle, Raquel Fernández

In this work, we use a transformer-based pre-trained multimodal model, CLIP, to shed light on the mechanisms employed by human speakers when referring to visual entities.


Is Information Density Uniform in Task-Oriented Dialogues?

1 code implementation EMNLP 2021 Mario Giulianelli, Arabella Sinclair, Raquel Fernández

The Uniform Information Density principle states that speakers plan their utterances to reduce fluctuations in the density of the information transmitted.

Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation

no code implementations19 Jun 2024 Jirui Qi, Gabriele Sarti, Raquel Fernández, Arianna Bisazza

In this work, we present MIRAGE --Model Internals-based RAG Explanations -- a plug-and-play approach using model internals for faithful answer attribution in RAG applications.

Question Answering Retrieval

MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs

1 code implementation11 Jun 2024 Vera Neplenbroek, Arianna Bisazza, Raquel Fernández

Motivated by this, we investigate whether the social stereotypes exhibited by LLMs differ as a function of the language used to prompt them, while controlling for cultural differences and task accuracy.

Question Answering

Analysing Cross-Speaker Convergence in Face-to-Face Dialogue through the Lens of Automatically Detected Shared Linguistic Constructions

no code implementations14 May 2024 Esam Ghaleb, Marlou Rasenberg, Wim Pouw, Ivan Toni, Judith Holler, Aslı Özyürek, Raquel Fernández

Conversation requires a substantial amount of coordination between dialogue participants, from managing turn taking to negotiating mutual understanding.

Leveraging Speech for Gesture Detection in Multimodal Communication

1 code implementation23 Apr 2024 Esam Ghaleb, Ilya Burenko, Marlou Rasenberg, Wim Pouw, Ivan Toni, Peter Uhrig, Anna Wilson, Judith Holler, Aslı Özyürek, Raquel Fernández

Our findings indicate that expanding the speech buffer beyond visual time segments improves performance and that multimodal integration using cross-modal and early fusion techniques outperforms baseline methods using unimodal and late fusion methods.

Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?

no code implementations25 Feb 2024 Joris Baan, Raquel Fernández, Barbara Plank, Wilker Aziz

With the rise of increasingly powerful and user-facing NLP systems, there is growing interest in assessing whether they have a good representation of uncertainty by evaluating the quality of their predictive distribution over outcomes.


Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions

no code implementations9 Feb 2024 Alberto Testoni, Raquel Fernández

Clarification questions are an essential dialogue tool to signal misunderstanding, ambiguities, and under-specification in language use.

Describing Images $\textit{Fast and Slow}$: Quantifying and Predicting the Variation in Human Signals during Visuo-Linguistic Processes

1 code implementation2 Feb 2024 Ece Takmaz, Sandro Pezzelle, Raquel Fernández

There is an intricate relation between the properties of an image and how humans behave while describing the image.

GROOViST: A Metric for Grounding Objects in Visual Storytelling

1 code implementation26 Oct 2023 Aditya K Surikuchi, Sandro Pezzelle, Raquel Fernández

A proper evaluation of stories generated for a sequence of images -- the task commonly referred to as visual storytelling -- must consider multiple aspects, such as coherence, grammatical correctness, and visual grounding.

Visual Grounding Visual Storytelling

The BLA Benchmark: Investigating Basic Language Abilities of Pre-Trained Multimodal Models

1 code implementation23 Oct 2023 Xinyi Chen, Raquel Fernández, Sandro Pezzelle

Despite the impressive performance achieved by pre-trained language-and-vision models in downstream tasks, it remains an open question whether this reflects a proper understanding of image-text interaction.

In-Context Learning

Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives

1 code implementation20 Oct 2023 Mario Giulianelli, Sarenne Wallbridge, Raquel Fernández

We present information value, a measure which quantifies the predictability of an utterance relative to a set of plausible alternatives.

Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models

1 code implementation16 Oct 2023 Jirui Qi, Raquel Fernández, Arianna Bisazza

Finally, we conduct a case study on CLC when new factual associations are inserted in the PLMs via model editing.

Model Editing

Co-Speech Gesture Detection through Multi-Phase Sequence Labeling

1 code implementation21 Aug 2023 Esam Ghaleb, Ilya Burenko, Marlou Rasenberg, Wim Pouw, Peter Uhrig, Judith Holler, Ivan Toni, Aslı Özyürek, Raquel Fernández

Yet, the prevalent approach to automatic gesture detection treats the problem as binary classification, classifying a segment as either containing a gesture or not, thus failing to capture its inherently sequential and contextual nature.

Binary Classification

Uncertainty in Natural Language Generation: From Theory to Applications

no code implementations28 Jul 2023 Joris Baan, Nico Daheim, Evgenia Ilia, Dennis Ulmer, Haau-Sing Li, Raquel Fernández, Barbara Plank, Rico Sennrich, Chrysoula Zerva, Wilker Aziz

Recent advances of powerful Language Models have allowed Natural Language Generation (NLG) to emerge as an important technology that can not only perform traditional tasks like summarisation or translation, but also serve as a natural language interface to a variety of applications.

Active Learning Text Generation

Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind

1 code implementation31 May 2023 Ece Takmaz, Nicolo' Brandizzi, Mario Giulianelli, Sandro Pezzelle, Raquel Fernández

Inspired by psycholinguistic theories, we endow our speaker with the ability to adapt its referring expressions via a simulation module that monitors the effectiveness of planned utterances from the listener's perspective.

Language Modelling Open-Ended Question Answering +1

What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability

1 code implementation19 May 2023 Mario Giulianelli, Joris Baan, Wilker Aziz, Raquel Fernández, Barbara Plank

In Natural Language Generation (NLG) tasks, for any input, multiple communicative goals are plausible, and any goal can be put into words, or produced, in multiple ways.

Text Generation

Stop Measuring Calibration When Humans Disagree

1 code implementation28 Oct 2022 Joris Baan, Wilker Aziz, Barbara Plank, Raquel Fernández

Calibration is a popular framework to evaluate whether a classifier knows when it does not know - i. e., its predictive probabilities are a good indication of how likely a prediction is to be correct.

Construction Repetition Reduces Information Rate in Dialogue

1 code implementation15 Oct 2022 Mario Giulianelli, Arabella Sinclair, Raquel Fernández

We hypothesise that speakers use construction repetition to mitigate information rate, leading to an overall decrease in utterance information content over the course of a dialogue.

Language Modelling

Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations

1 code implementation30 Sep 2021 Arabella Sinclair, Jaap Jumelet, Willem Zuidema, Raquel Fernández

We investigate the extent to which modern, neural language models are susceptible to structural priming, the phenomenon whereby the structure of a sentence makes the same structure more probable in a follow-up sentence.

Natural Language Understanding Sentence

You Shall Know a User by the Company It Keeps: Dynamic Representations for Social Media Users in NLP

no code implementations IJCNLP 2019 Marco Del Tredici, Diego Marcheggiani, Sabine Schulte im Walde, Raquel Fernández

Information about individuals can help to better understand what they say, particularly in social media where texts are short.

Graph Attention

Is the Red Square Big? MALeViC: Modeling Adjectives Leveraging Visual Contexts

no code implementations27 Aug 2019 Sandro Pezzelle, Raquel Fernández

This work aims at modeling how the meaning of gradable adjectives of size (`big', `small') can be learned from visually-grounded contexts.

The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue

no code implementations ACL 2019 Janosch Haber, Tim Baumgärtner, Ece Takmaz, Lieke Gelderloos, Elia Bruni, Raquel Fernández

This paper introduces the PhotoBook dataset, a large-scale collection of visually-grounded, task-oriented dialogues in English designed to investigate shared dialogue history accumulating during conversation.

Evaluating the Representational Hub of Language and Vision Models

no code implementations WS 2019 Ravi Shekhar, Ece Takmaz, Raquel Fernández, Raffaella Bernardi

The multimodal models used in the emerging field at the intersection of computational linguistics and computer vision implement the bottom-up processing of the `Hub and Spoke' architecture proposed in cognitive science to represent how the brain processes and combines multi-sensory inputs.

Question Answering Visual Question Answering

Automatic Evaluation of Neural Personality-based Chatbots

no code implementations WS 2018 Yujie Xing, Raquel Fernández

Stylistic variation is critical to render the utterances generated by conversational agents natural and engaging.

Response Generation

Short-Term Meaning Shift: A Distributional Exploration

1 code implementation NAACL 2019 Marco Del Tredici, Raquel Fernández, Gemma Boleda

We present the first exploration of meaning shift over short periods of time in online communities using distributional representations.

Analysing the potential of seq-to-seq models for incremental interpretation in task-oriented dialogue

no code implementations WS 2018 Dieuwke Hupkes, Sanne Bouwmeester, Raquel Fernández

We investigate how encoder-decoder models trained on a synthetic dataset of task-oriented dialogues process disfluencies, such as hesitations and self-corrections.


The Road to Success: Assessing the Fate of Linguistic Innovations in Online Communities

no code implementations COLING 2018 Marco Del Tredici, Raquel Fernández

We investigate the birth and diffusion of lexical innovations in a large dataset of online social communities.


Semantic Variation in Online Communities of Practice

no code implementations WS 2017 Marco Del Tredici, Raquel Fernández

We introduce a framework for quantifying semantic variation of common words in Communities of Practice and in sets of topic-related communities.

Language Modelling

Examining a hate speech corpus for hate speech detection and popularity prediction

1 code implementation12 May 2018 Filip Klubička, Raquel Fernández

As research on hate speech becomes more and more relevant every day, most of it is still focused on hate speech detection.

Hate Speech Detection

Cannot find the paper you are looking for? You can Submit a new open access paper.