Search Results for author: Lorenzo Baraldi

Found 51 papers, 22 papers with code

Towards Retrieval-Augmented Architectures for Image Captioning

no code implementations21 May 2024 Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Alessandro Nicolosi, Rita Cucchiara

The objective of image captioning models is to bridge the gap between the visual and linguistic modalities by generating natural language descriptions that accurately reflect the content of input images.

Image Captioning Language Modelling +1

AIGeN: An Adversarial Approach for Instruction Generation in VLN

no code implementations15 Apr 2024 Niyati Rawal, Roberto Bigazzi, Lorenzo Baraldi, Rita Cucchiara

VLN is a challenging task that involves an agent following human instructions and navigating in a previously unknown environment to reach a specified goal.

Decoder Vision and Language Navigation

Mapping High-level Semantic Regions in Indoor Environments without Object Recognition

no code implementations11 Mar 2024 Roberto Bigazzi, Lorenzo Baraldi, Shreyas Kousik, Rita Cucchiara, Marco Pavone

Robots require a semantic understanding of their surroundings to operate in an efficient and explainable way in human environments.

Graph Generation Language Modelling +3

Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models

1 code implementation27 Nov 2023 Samuele Poppi, Tobia Poppi, Federico Cocchi, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

We show how this can be done by fine-tuning a CLIP model on synthetic data obtained from a large language model trained to convert between safe and unsafe sentences, and a text-to-image generator.

Cross-Modal Retrieval Image Retrieval +5

With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning

1 code implementation ICCV 2023 Manuele Barraco, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

Image captioning, like many tasks involving vision and language, currently relies on Transformer-based architectures for extracting the semantics in an image and translating it into linguistically coherent descriptions.

Decoder Image Captioning

Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation

no code implementations18 Jul 2023 Federico Betti, Jacopo Staiano, Lorenzo Baraldi, Rita Cucchiara, Nicu Sebe

Research in Image Generation has recently made significant progress, particularly boosted by the introduction of Vision-Language models which are able to produce high-quality visual content based on textual inputs.

Image Generation Question Answering +1

Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training

1 code implementation12 Jun 2023 Roberto Amoroso, Marcella Cornia, Lorenzo Baraldi, Andrea Pilzer, Rita Cucchiara

The use of self-supervised pre-training has emerged as a promising approach to enhance the performance of visual tasks such as image classification.

Image Classification

Evaluating Synthetic Pre-Training for Handwriting Processing Tasks

no code implementations4 Apr 2023 Vittorio Pippi, Silvia Cascianelli, Lorenzo Baraldi, Rita Cucchiara

In this work, we explore massive pre-training on synthetic word images for enhancing the performance on four benchmark downstream handwriting analysis tasks.

Retrieval

Multi-Class Unlearning for Image Classification via Weight Filtering

no code implementations4 Apr 2023 Samuele Poppi, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

Machine Unlearning is an emerging paradigm for selectively removing the impact of training datapoints from a network.

Classification Image Classification +1

Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images

2 code implementations2 Apr 2023 Roberto Amoroso, Davide Morelli, Marcella Cornia, Lorenzo Baraldi, Alberto del Bimbo, Rita Cucchiara

Recent advancements in diffusion models have enabled the generation of realistic deepfakes from textual prompts in natural language.

DeepFake Detection Face Swapping +2

Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation

1 code implementation CVPR 2023 Sara Sarto, Manuele Barraco, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

The CLIP model has been recently proven to be very effective for a variety of cross-modal tasks, including the evaluation of captions generated from vision-and-language architectures.

Contrastive Learning Image Captioning +1

Embodied Agents for Efficient Exploration and Smart Scene Description

no code implementations17 Jan 2023 Roberto Bigazzi, Marcella Cornia, Silvia Cascianelli, Lorenzo Baraldi, Rita Cucchiara

The development of embodied agents that can communicate with humans in natural language has gained increasing interest over the last years, as it facilitates the diffusion of robotic platforms in human-populated environments.

Efficient Exploration Image Captioning +1

Boosting Modern and Historical Handwritten Text Recognition with Deformable Convolutions

no code implementations17 Aug 2022 Silvia Cascianelli, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

Handwritten Text Recognition (HTR) in free-layout pages is a challenging image understanding task that can provide a relevant boost to the digitization of handwritten documents and reuse of their content.

Handwritten Text Recognition HTR

The LAM Dataset: A Novel Benchmark for Line-Level Handwritten Text Recognition

no code implementations16 Aug 2022 Silvia Cascianelli, Vittorio Pippi, Martin Maarand, Marcella Cornia, Lorenzo Baraldi, Christopher Kermorvant, Rita Cucchiara

With the aim of fostering the research on this topic, in this paper we present the Ludovico Antonio Muratori (LAM) dataset, a large line-level HTR dataset of Italian ancient manuscripts edited by a single author over 60 years.

Handwritten Text Recognition HTR

ALADIN: Distilling Fine-grained Alignment Scores for Efficient Image-Text Matching and Retrieval

1 code implementation29 Jul 2022 Nicola Messina, Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Fabrizio Falchi, Giuseppe Amato, Rita Cucchiara

In literature, this task is often used as a pre-training objective to forge architectures able to jointly deal with images and texts.

Ranked #22 on Cross-Modal Retrieval on COCO 2014 (using extra training data)

Image-text matching Retrieval +1

Retrieval-Augmented Transformer for Image Captioning

no code implementations26 Jul 2022 Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

In this paper, we investigate the development of an image captioning approach with a kNN memory, with which knowledge can be retrieved from an external corpus to aid the generation process.

Image Captioning Retrieval

Embodied Navigation at the Art Gallery

no code implementations19 Apr 2022 Roberto Bigazzi, Federico Landi, Silvia Cascianelli, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

This feature is challenging for occupancy-based agents which are usually trained in crowded domestic environments with plenty of occupancy information.

Navigate PointGoal Navigation

Spot the Difference: A Novel Task for Embodied Agents in Changing Environments

no code implementations18 Apr 2022 Federico Landi, Roberto Bigazzi, Marcella Cornia, Silvia Cascianelli, Lorenzo Baraldi, Rita Cucchiara

To make a step towards this setting, we propose Spot the Difference: a novel task for Embodied AI where the agent has access to an outdated map of the environment and needs to recover the correct layout in a fixed time budget.

CaMEL: Mean Teacher Learning for Image Captioning

1 code implementation21 Feb 2022 Manuele Barraco, Matteo Stefanini, Marcella Cornia, Silvia Cascianelli, Lorenzo Baraldi, Rita Cucchiara

Describing images in natural language is a fundamental step towards the automatic modeling of connections between the visual and textual modalities.

Image Captioning Knowledge Distillation

Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets

no code implementations24 Nov 2021 Marcella Cornia, Lorenzo Baraldi, Giuseppe Fiameni, Rita Cucchiara

This paper addresses the task of generating fluent descriptions by training on a non-uniform combination of data sources, containing both human-annotated and web-collected captions.

Descriptive Image Captioning +2

Focus on Impact: Indoor Exploration with Intrinsic Motivation

1 code implementation14 Sep 2021 Roberto Bigazzi, Federico Landi, Silvia Cascianelli, Lorenzo Baraldi, Marcella Cornia, Rita Cucchiara

The proposed exploration approach outperforms DRL-based competitors relying on intrinsic rewards and surpasses the agents trained with a dense extrinsic reward computed with the environment layouts.

Working Memory Connections for LSTM

no code implementations31 Aug 2021 Federico Landi, Lorenzo Baraldi, Marcella Cornia, Rita Cucchiara

Numerical results suggest that the cell state contains useful information that is worth including in the gate structure.

From Show to Tell: A Survey on Deep Learning-based Image Captioning

no code implementations14 Jul 2021 Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Silvia Cascianelli, Giuseppe Fiameni, Rita Cucchiara

Starting from 2015 the task has generally been addressed with pipelines composed of a visual encoder and a language model for text generation.

Image Captioning Language Modelling +1

Learning to Select: A Fully Attentive Approach for Novel Object Captioning

no code implementations2 Jun 2021 Marco Cagrandi, Marcella Cornia, Matteo Stefanini, Lorenzo Baraldi, Rita Cucchiara

In this paper, we present a novel approach for NOC that learns to select the most relevant objects of an image, regardless of their adherence to the training set, and to constrain the generative process of a language model accordingly.

Image Captioning Language Modelling

Inter-Homines: Distance-Based Risk Estimation for Human Safety

no code implementations20 Jul 2020 Matteo Fabbri, Fabio Lanzi, Riccardo Gasparini, Simone Calderara, Lorenzo Baraldi, Rita Cucchiara

In this document, we report our proposal for modeling the risk of possible contagiousity in a given area monitored by RGB cameras where people freely move and interact.

Explore and Explain: Self-supervised Navigation and Recounting

no code implementations14 Jul 2020 Roberto Bigazzi, Federico Landi, Marcella Cornia, Silvia Cascianelli, Lorenzo Baraldi, Rita Cucchiara

In this paper, we devise a novel embodied setting in which an agent needs to explore a previously unknown environment while recounting what it sees during the path.

Navigate

A Novel Attention-based Aggregation Function to Combine Vision and Language

no code implementations27 Apr 2020 Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

The joint understanding of vision and language has been recently gaining a lot of attention in both the Computer Vision and Natural Language Processing communities, with the emergence of tasks such as image captioning, image-text matching, and visual question answering.

General Classification Image Captioning +4

Meshed-Memory Transformer for Image Captioning

2 code implementations CVPR 2020 Marcella Cornia, Matteo Stefanini, Lorenzo Baraldi, Rita Cucchiara

Transformer-based architectures represent the state of the art in sequence modeling tasks like machine translation and language understanding.

Image Captioning Machine Translation +2

Multimodal Attention Networks for Low-Level Vision-and-Language Navigation

1 code implementation27 Nov 2019 Federico Landi, Lorenzo Baraldi, Marcella Cornia, Massimiliano Corsini, Rita Cucchiara

Vision-and-Language Navigation (VLN) is a challenging task in which an agent needs to follow a language-specified path to reach a target destination.

Vision and Language Navigation

SMArT: Training Shallow Memory-aware Transformers for Robotic Explainability

no code implementations7 Oct 2019 Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

The ability to generate natural language explanations conditioned on the visual perception is a crucial step towards autonomous agents which can explain themselves and communicate with humans.

Text Generation Video Captioning

Artpedia

no code implementations International Conference on Image Analysis and Processing 2019 Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Massimiliano Corsini, and Rita Cucchiara

As vision and language techniques are widely applied to realistic images , there is a growing interest in designing visual-semantic models suitable for more complex and challenging scenarios.

Cross-Modal Retrieval Retrieval

Embodied Vision-and-Language Navigation with Dynamic Convolutional Filters

1 code implementation5 Jul 2019 Federico Landi, Lorenzo Baraldi, Massimiliano Corsini, Rita Cucchiara

In Vision-and-Language Navigation (VLN), an embodied agent needs to reach a target destination with the only guidance of a natural language instruction.

Vision and Language Navigation

A Deep Learning based approach to VM behavior identification in cloud systems

1 code implementation5 Mar 2019 Matteo Stefanini, Riccardo Lancellotti, Lorenzo Baraldi, Simone Calderara

The experiments compare our proposal with state-of-the-art solutions available in literature, demonstrating that our proposal achieve better performance.

Cloud Computing Clustering +1

M-VAD Names: a Dataset for Video Captioning with Naming

1 code implementation4 Mar 2019 Stefano Pini, Marcella Cornia, Federico Bolelli, Lorenzo Baraldi, Rita Cucchiara

Current movie captioning architectures are not capable of mentioning characters with their proper name, replacing them with a generic "someone" tag.

TAG Video Captioning

Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translation

1 code implementation CVPR 2019 Matteo Tomei, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

The applicability of computer vision to real paintings and artworks has been rarely investigated, even though a vast heritage would greatly benefit from techniques which can understand and process data from the artistic domain.

Image-to-Image Translation Translation

Show, Control and Tell: A Framework for Generating Controllable and Grounded Captions

1 code implementation CVPR 2019 Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

Current captioning approaches can describe images using black-box architectures whose behavior is hardly controllable and explainable from the exterior.

controllable image captioning Diversity

Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention

no code implementations26 Jun 2017 Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara

Image captioning has been recently gaining a lot of attention thanks to the impressive achievements shown by deep captioning architectures, which combine Convolutional Neural Networks to extract image representations, and Recurrent Neural Networks to generate the corresponding captions.

Ranked #2 on Image Captioning on Flickr30k Captions test (using extra training data)

Image Captioning Saliency Prediction

Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model

2 code implementations29 Nov 2016 Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara

Data-driven saliency has recently gained a lot of attention thanks to the use of Convolutional Neural Networks for predicting gaze fixations.

Saliency Prediction

Hierarchical Boundary-Aware Neural Encoder for Video Captioning

no code implementations CVPR 2017 Lorenzo Baraldi, Costantino Grana, Rita Cucchiara

The use of Recurrent Neural Networks for video captioning has recently gained a lot of attention, since they can be used both to encode the input video and to generate the corresponding description.

Decoder Video Captioning +1

Recognizing and Presenting the Storytelling Video Structure with Deep Multimodal Networks

no code implementations5 Oct 2016 Lorenzo Baraldi, Costantino Grana, Rita Cucchiara

This paper presents a novel approach for temporal and semantic segmentation of edited videos into meaningful segments, from the point of view of the storytelling structure.

Change Detection Retrieval +1

A Deep Multi-Level Network for Saliency Prediction

2 code implementations5 Sep 2016 Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara

Current state of the art models for saliency prediction employ Fully Convolutional networks that perform a non-linear combination of features extracted from the last convolutional layer to predict saliency maps.

Saliency Prediction

Scene-driven Retrieval in Edited Videos using Aesthetic and Semantic Deep Features

no code implementations9 Apr 2016 Lorenzo Baraldi, Costantino Grana, Rita Cucchiara

This paper presents a novel retrieval pipeline for video collections, which aims to retrieve the most significant parts of an edited video for a given query, and represent them with thumbnails which are at the same time semantically meaningful and aesthetically remarkable.

Retrieval

A Deep Siamese Network for Scene Detection in Broadcast Videos

1 code implementation29 Oct 2015 Lorenzo Baraldi, Costantino Grana, Rita Cucchiara

We present a model that automatically divides broadcast videos into coherent scenes by learning a distance measure between shots.

Scene Segmentation

Cannot find the paper you are looking for? You can Submit a new open access paper.