Search Results for author:
Found 37 papers, 3 papers with code
How Vision Affects Language: Comparing Masked Self-Attention in Uni-Modal and Multi-Modal Transformer
In this paper, we examine masked self-attention in a multi-modal transformer trained for the task of image captioning.
Fine-grained Entailment: Resources for Greek NLI and Precise Entailment
First, we extend the Greek version of the FraCaS test suite to include examples where the inference is directly linked to the syntactic/morphological properties of Greek.
Annotating anaphoric phenomena in situated dialogue
With this paper, we intend to start a discussion on the annotation of referential phenomena in situated dialogue.
When an Image Tells a Story: The Role of Visual and Semantic Information for Generating Paragraph Descriptions
Generating multi-sentence image descriptions is a challenging task, which requires a good model to produce coherent and accurate paragraphs, describing salient objects in the image.
Ranked #10 on
Image Paragraph Captioning
on Image Paragraph Captioning
Pre-trained Models or Feature Engineering: The Case of Dialectal Arabic
The usage of social media platforms has resulted in the proliferation of work on Arabic Natural Language Processing (ANLP), including the development of resources.
Reference and coreference in situated dialogue
We argue that there is still significant room for corpora that increase the complexity of both visual and linguistic domains and which capture different varieties of perceptual and conversational contexts.
Attention as Grounding: Exploring Textual and Cross-Modal Attention on Entities and Relations in Language-and-Vision Transformer
We explore how a multi-modal transformer trained for generation of longer image descriptions learns syntactic and semantic representations about entities and relations grounded in objects at the level of masked self-attention (text generation) and cross-modal attention (information fusion).
In Search of Meaning and Its Representations for Computational Linguistics
In this paper we examine different meaning representations that are commonly used in different natural language applications today and discuss their limits, both in terms of the aspects of the natural language meaning they are modelling and in terms of the aspects of the application for which they are used.
Examining the Effects of Language-and-Vision Data Augmentation for Generation of Descriptions of Human Faces
We conclude that face description generation systems are more susceptible to language rather than vision data augmentation.
Fast visual grounding in interaction: bringing few-shot learning with neural networks to an interactive robot
The major shortcomings of using neural networks with situated agents are that in incremental interaction very few learning examples are available and that their visual sensory representations are quite different from image caption datasets.
Anaphoric Phenomena in Situated dialog: A First Round of Annotations
We present a first release of 500 documents from the multimodal corpus Tell-me-more (Ilinykh et al., 2019) annotated with coreference information according to the ARRAU guidelines (Poesio et al., 2021).
Grandma Karl is 27 years old -- research agenda for pseudonymization of research data
Accessibility of research data is critical for advances in many research fields, but textual data often cannot be shared due to the personal and sensitive information which it contains, e. g names or political opinions.
Multi-task recommendation system for scientific papers with high-way networks
Finding and selecting the most relevant scientific papers from a large number of papers written in a research community is one of the key challenges for researchers these days.
We went to look for meaning and all we got were these lousy representations: aspects of meaning representation for computational semantics
In this paper we examine different meaning representations that are commonly used in different natural language applications today and discuss their limits, both in terms of the aspects of the natural language meaning they are modelling and in terms of the aspects of the application for which they are used.
Sky + Fire = Sunset. Exploring Parallels between Visually Grounded Metaphors and Image Classifiers
This work explores the differences and similarities between neural image classifiers{'} mis-categorisations and visually grounded metaphors - that we could conceive as intentional mis-categorisations.
An Arabic Tweets Sentiment Analysis Dataset (ATSAD) using Distant Supervision and Self Training
As the number of social media users increases, they express their thoughts, needs, socialise and publish their opinions reviews.
Normalising Non-standardised Orthography in Algerian Code-switched User-generated Data
We work with Algerian, an under-resourced non-standardised Arabic variety, for which we compile a new parallel corpus consisting of user-generated textual data matched with normalised and corrected human annotations following data-driven and our linguistically motivated standard.
What goes into a word: generating image descriptions with top-down spatial knowledge
The aim of this paper is to evaluate what representations facilitate generating image descriptions with spatial relations and lead to better grounded language generation.
Neural Models for Detecting Binary Semantic Textual Similarity for Algerian and MSA
We explore the extent to which neural networks can learn to identify semantically equivalent sentences from a small variable dataset using an end-to-end training.
What a neural language model tells us about spatial relations
Understanding and generating spatial descriptions requires knowledge about what objects are related, their functional interactions, and where the objects are geometrically located.
Modular Mechanistic Networks: On Bridging Mechanistic and Phenomenological Models with Deep Neural Networks in Natural Language Processing
Natural language processing (NLP) can be done using either top-down (theory driven) and bottom-up (data driven) approaches, which we call mechanistic and phenomenological respectively.
What is not where: the challenge of integrating spatial representations into deep learning architectures
This paper examines to what degree current deep learning architectures for image caption generation capture spatial language.
Improving Neural Network Performance by Injecting Background Knowledge: Detecting Code-switching and Borrowing in Algerian texts
We explore the effect of injecting background knowledge to different deep neural network (DNN) configurations in order to mitigate the problem of the scarcity of annotated data when applying these models on datasets of low-resourced languages.
A Comparison of Character Neural Language Model and Bootstrapping for Language Identification in Multilingual Noisy Texts
This paper seeks to examine the effect of including background knowledge in the form of character pre-trained neural language model (LM), and data bootstrapping to overcome the problem of unbalanced limited resources.
Exploring the Functional and Geometric Bias of Spatial Relations Using Neural Language Models
The challenge for computational models of spatial descriptions for situated dialogue systems is the integration of information from different modalities.
Identification of Languages in Algerian Arabic Multilingual Documents
This paper presents a language identification system designed to detect the language of each word, in its context, in a multilingual documents as generated in social media by bilingual/multilingual communities, in our case speakers of Algerian Arabic.
