Search Results for author:
Found 58 papers, 19 papers with code
Entity-Based Semantic Adequacy for Data-to-Text Generation
While powerful pre-trained language models have improved the fluency of text generation models, semantic adequacy -the ability to generate text that is semantically faithful to the input- remains an unsolved issue.
Unmasking Contextual Stereotypes: Measuring and Mitigating BERT’s Gender Bias
Contextualized word embeddings have been replacing standard embeddings as the representational knowledge source of choice in NLP systems.
Gradations of Error Severity in Automatic Image Descriptions
Because our manipulated descriptions form minimal pairs with the reference descriptions, we are able to assess the impact of different kinds of errors on the perceived quality of the descriptions.
Towards Harnessing Natural Language Generation to Explain Black-box Models
The opaque nature of many machine learning techniques prevents the wide adoption of powerful information processing tools for high stakes scenarios.
The Natural Language Pipeline, Neural Text Generation and Explainability
End-to-end encoder-decoder approaches to data-to-text generation are often black boxes whose predictions are difficult to explain.
Visually Grounded Interpretation of Noun-Noun Compounds in English
We find that adding visual vectors increases classification performance on our dataset in many cases.
Multi3Generation: Multitask, Multilingual, Multimodal Language Generation
This paper presents the Multitask, Multilingual, Multimodal Language Generation COST Action – Multi3Generation (CA18231), an interdisciplinary network of research groups working on different aspects of language generation.
CV-Probes: Studying the interplay of lexical and world knowledge in visually grounded verb understanding
This study investigates the ability of various vision-language (VL) models to ground context-dependent and non-context-dependent verb phrases.
Summarizing long regulatory documents with a multi-step pipeline
In this paper, we show that the effectiveness of a two-step architecture for summarizing long regulatory texts varies significantly depending on the model used.
Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices
Automatic metrics are extensively used to evaluate natural language processing systems.
Context-aware Visual Storytelling with Visual Prefix Tuning and Contrastive Learning
Visual storytelling systems generate multi-sentence stories from image sequences.
How and where does CLIP process negation?
Here we build on the existence task from the VALSE benchmark (Parcalabescu et al, 2022) which we use to test models' understanding of negation, a particularly interesting issue for multimodal models.
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks
There is an increasing trend towards evaluating NLP models with LLM-generated judgments instead of human judgments.
A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences
The reasoning abilities of Large Language Models (LLMs) are becoming a central focus of study in NLP.
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models
With the ever-increasing popularity of pretrained Video-Language Models (VidLMs), there is a pressing need to develop robust evaluation methodologies that delve deeper into their visio-linguistic capabilities.
FTFT: Efficient and Robust Fine-Tuning by Transferring Training Dynamics
Dataset cartography is a simple yet effective dual-model approach that improves the robustness of fine-tuned PLMs.
The Scenario Refiner: Grounding subjects in images at the morphological level
Derivationally related words, such as "runner" and "running", exhibit semantic differences which also elicit different visual scenarios.
Contrast Is All You Need
We focused on two finetuning objectives; SetFit (Sentence Transformer Finetuning), a contrastive learning setup, and a vanilla finetuning setup on a legal provision classification task.
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
Interpreting Vision and Language Generative Models with Semantic Visual Priors
When applied to Image-to-text models, interpretability methods often provide token-by-token explanations namely, they compute a visual explanation for each token of the generated sequence.
HL Dataset: Visually-grounded Description of Scenes, Actions and Rationales
We present the High-Level Dataset a dataset extending 14997 images from the COCO dataset, aligned with a new set of 134, 973 human-annotated (high-level) captions collected along three axes: scenes, actions, and rationales.
Understanding Cross-modal Interactions in V&L Models that Generate Scene Descriptions
Image captioning models tend to describe images in an object-centric way, emphasising visible objects.
Face2Text revisited: Improved data set and baseline results
To encourage the development of more human-focused descriptions, we developed a new data set of facial descriptions based on the CelebA image data set.
Pre-training Data Quality and Quantity for a Low-Resource Language: New Corpus and BERT Models for Maltese
We also present a newly created corpus for Maltese, and determine the effect that the pre-training data size and domain have on the downstream performance.
VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena
We propose VALSE (Vision And Language Structured Evaluation), a novel benchmark designed for testing general-purpose pretrained vision and language (V&L) models for their visio-linguistic grounding capabilities on specific linguistic phenomena.
Analysis of Data Augmentation Methods for Low-Resource Maltese ASR
Recent years have seen an increased interest in the computational speech processing of Maltese, but resources remain sparse.
What Vision-Language Models `See' when they See Scenes
Images can be described in terms of the objects they contain, or in terms of the types of scene or place that they instantiate.
On the Language-specificity of Multilingual BERT and the Impact of Fine-tuning
Recent work has shown evidence that the knowledge acquired by multilingual BERT (mBERT) has two components: a language-specific and a language-neutral one.
On the interaction of automatic evaluation and task framing in headline style transfer
An ongoing debate in the NLG community concerns the best way to evaluate systems, with human evaluation often being considered the most reliable method, compared to corpus-based metrics.
Seeing past words: Testing the cross-modal capabilities of pretrained V&L models on counting tasks
We investigate the reasoning ability of pretrained vision and language (V&L) models in two tasks that require multimodal integration: (1) discriminating a correct image-sentence pair from an incorrect one, and (2) counting entities in an image.
Datasets and Models for Authorship Attribution on Italian Personal Writings
Existing research on Authorship Attribution (AA) focuses on texts for which a lot of data is available (e. g novels), mainly in English.
Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias
Contextualized word embeddings have been replacing standard embeddings as the representational knowledge source of choice in NLP systems.
Annotating for Hate Speech: The MaNeCo Corpus and Some Input from Critical Discourse Analysis
In view of this, we suggest a multi-layer annotation scheme, which is pilot-tested against a binary +/- hate speech classification and appears to yield higher inter-annotator agreement.
MASRI-HEADSET: A Maltese Corpus for Speech Recognition
Maltese, the national language of Malta, is spoken by approximately 500, 000 people.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
On Architectures for Including Visual Information in Neural Language Models for Image Description
We also observe that the merge architecture can have its recurrent neural network pre-trained in a text-only language model (transfer learning) rather than be initialised randomly as usual.
Best practices for the human evaluation of automatically generated text
Currently, there is little agreement as to how Natural Language Generation (NLG) systems should be evaluated.
Visually grounded generation of entailments from premises
Natural Language Inference (NLI) is the task of determining the semantic relationship between a premise and a hypothesis.
Visuallly Grounded Generation of Entailments from Premises
Natural Language Inference (NLI) is the task of determining the semantic relationship between a premise and a hypothesis.
You Write Like You Eat: Stylistic variation as a predictor of social stratification
Inspired by Labov's seminal work on stylistic variation as a function of social stratification, we develop and compare neural models that predict a person's presumed socio-economic status, obtained through distant supervision, from their writing style on social media.
Transfer learning from language models to image caption generators: Better models may not transfer better
When designing a neural caption generator, a convolutional neural network can be used to extract image features.
Pre-gen metrics: Predicting caption quality metrics without generating captions
Image caption generation systems are typically evaluated against reference outputs.
Quantifying the amount of visual information used by neural caption generators
This paper addresses the sensitivity of neural image caption generators to their visual input.
Specificity measures and reference
More specifically, we study the ability of several measures of referential success to predict the success of a user in choosing the right object, given a referring expression.
Meteorologists and Students: A resource for language grounding of geographical descriptors
We present a data resource which can be useful for research purposes on language grounding tasks in the context of geographical referring expression generation.
Making effective use of healthcare data using data-to-text technology
Financial reports are produced to assess healthcare organizations on some key performance indicators to steer their healthcare delivery.
Grounded Textual Entailment
Capturing semantic relations between sentences, such as entailment, is a long-standing challenge for computational semantics.
Ranked #2 on
Natural Language Inference
on V-SNLI
Face2Text: Collecting an Annotated Image Description Corpus for the Generation of Rich Face Descriptions
To gain a better understanding of the variation we find in face description and the possible issues that this may raise, we also conducted an annotation study on a subset of the corpus.
What is the Role of Recurrent Neural Networks (RNNs) in an Image Caption Generator?
This view suggests that the RNN should only be used to encode linguistic features and that only the final representation should be `merged' with the image features at a later stage.
An Empirical Approach for Modeling Fuzzy Geographical Descriptors
We present a novel heuristic approach that defines fuzzy geographical descriptors using data gathered from a survey with human subjects.
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation
This paper surveys the current state of the art in Natural Language Generation (NLG), defined as the task of generating text or speech from non-linguistic input.
Where to put the Image in an Image Caption Generator
When a recurrent neural network language model is used for caption generation, the image information can be fed to the neural network either by directly incorporating it in the RNN -- conditioning the language model by `injecting' image features -- or in a layer following the RNN -- conditioning the language model by `merging' image features.
Morphological Analysis for the Maltese Language: The Challenges of a Hybrid System
In particular, we analyse a dataset of morphologically related word clusters to evaluate the difference in results for concatenative and nonconcatenative clusters.
Crowd-sourcing evaluation of automatically acquired, morphologically related word groupings
The automatic discovery and clustering of morphologically related words is an important problem with several practical applications.
Incorporating an Error Corpus into a Spellchecker for Maltese
This paper discusses the ongoing development of a new Maltese spell checker, highlighting the methodologies which would best suit such a language.
A Repository of Data and Evaluation Resources for Natural Language Generation
Starting in 2007, the field of natural language generation (NLG) has organised shared-task evaluation events every year, under the Generation Challenges umbrella.
