Search Results for author:
Found 104 papers, 51 papers with code
Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMs
Multimodal LLMs are the natural evolution of LLMs, and enlarge their capabilities so as to work beyond the pure textual modality.
AIGeN: An Adversarial Approach for Instruction Generation in VLN
VLN is a challenging task that involves an agent following human instructions and navigating in a previously unknown environment to reach a specified goal.
Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation
Open-vocabulary semantic segmentation aims at segmenting arbitrary categories expressed in textual form.
Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing
Fashion illustration is a crucial medium for designers to convey their creative vision and transform design concepts into tangible representations that showcase the interplay between clothing and the human body.
Unveiling the Truth: Exploring Human Gaze Patterns in Fake Images
Creating high-quality and realistic images is now possible thanks to the impressive advancements in image generation.
Mapping High-level Semantic Regions in Indoor Environments without Object Recognition
Robots require a semantic understanding of their surroundings to operate in an efficient and explainable way in human environments.
Trends, Applications, and Challenges in Human Attention Modelling
Human attention modelling has proven, in recent years, to be particularly useful not only for understanding the cognitive processes underlying visual exploration, but also for providing support to artificial intelligence models that aim to solve problems in various domains, including image and video processing, vision-and-language applications, and language modelling.
The (R)Evolution of Multimodal Large Language Models: A Survey
Connecting text and visual modalities plays an essential role in generative intelligence.
VATr++: Choose Your Words Wisely for Handwritten Text Generation
Styled Handwritten Text Generation (HTG) has received significant attention in recent years, propelled by the success of learning-based solutions employing GANs, Transformers, and, preliminarily, Diffusion Models.
Key-Graph Transformer for Image Restoration
While it is crucial to capture global information for effective image restoration (IR), integrating such cues into transformer-based methods becomes computationally expensive, especially with high input resolution.
DistFormer: Enhancing Local and Global Features for Monocular Per-Object Distance Estimation
Existing approaches rely on two scales: local information (i. e., the bounding box proportions) or global information, which encodes the semantics of the scene as well as the spatial relations with neighboring objects.
Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models
We show how this can be done by fine-tuning a CLIP model on synthetic data obtained from a large language model trained to convert between safe and unsafe sentences, and a text-to-image generator.
HWD: A Novel Evaluation Score for Styled Handwritten Text Generation
Through extensive experimental evaluation on different word-level and line-level datasets of handwritten text images, we demonstrate the suitability of the proposed HWD as a score for Styled HTG.
OpenFashionCLIP: Vision-and-Language Contrastive Learning with Open-Source Fashion Data
The inexorable growth of online shopping and e-commerce demands scalable and robust machine learning-based solutions to accommodate customer requirements.
With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning
Image captioning, like many tasks involving vision and language, currently relies on Transformer-based architectures for extracting the semantics in an image and translating it into linguistically coherent descriptions.
TrackFlow: Multi-Object Tracking with Normalizing Flows
The field of multi-object tracking has recently seen a renewed interest in the good old schema of tracking-by-detection, as its simplicity and strong priors spare it from the complex design and painful babysitting of tracking-by-attention approaches.
Volumetric Fast Fourier Convolution for Detecting Ink on the Carbonized Herculaneum Papyri
Recent advancements in Digital Document Restoration (DDR) have led to significant breakthroughs in analyzing highly damaged written artifacts.
CarPatch: A Synthetic Benchmark for Radiance Field Evaluation on Vehicle Components
Neural Radiance Fields (NeRFs) have gained widespread recognition as a highly effective technique for representing 3D reconstructions of objects and scenes derived from sets of images.
Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation
Research in Image Generation has recently made significant progress, particularly boosted by the introduction of Vision-Language models which are able to produce high-quality visual content based on textual inputs.
Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training
The use of self-supervised pre-training has emerged as a promising approach to enhance the performance of visual tasks such as image classification.
LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-On
In this context, image-based virtual try-on, which consists in generating a novel image of a target model wearing a given in-shop garment, has yet to capitalize on the potential of these powerful generative solutions.
How to Choose Pretrained Handwriting Recognition Models for Single Writer Fine-Tuning
Recent advancements in Deep Learning-based Handwritten Text Recognition (HTR) have led to models with remarkable performance on both modern and historical manuscripts in large benchmark datasets.
Evaluating Synthetic Pre-Training for Handwriting Processing Tasks
In this work, we explore massive pre-training on synthetic word images for enhancing the performance on four benchmark downstream handwriting analysis tasks.
Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing
Given the lack of existing datasets suitable for the task, we also extend two existing fashion datasets, namely Dress Code and VITON-HD, with multimodal annotations collected in a semi-automatic manner.
Multi-Class Explainable Unlearning for Image Classification via Weight Filtering
Machine Unlearning has recently been emerging as a paradigm for selectively removing the impact of training datapoints from a network.
Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images
Recent advancements in diffusion models have enabled the generation of realistic deepfakes by writing textual prompts in natural language.
Handwritten Text Generation from Visual Archetypes
Generating synthetic images of handwritten text in a writer-specific style is a challenging task, especially in the case of unseen styles and new words, and even more when these latter contain characters that are rarely encountered during training.
Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation
The CLIP model has been recently proven to be very effective for a variety of cross-modal tasks, including the evaluation of captions generated from vision-and-language architectures.
One Transformer for All Time Series: Representing and Training with Time-Dependent Heterogeneous Tabular Data
There is a recent growing interest in applying Deep Learning techniques to tabular data, in order to replicate the success of other Artificial Intelligence areas in this structured domain.
Input Perturbation Reduces Exposure Bias in Diffusion Models
Denoising Diffusion Probabilistic Models have shown an impressive generation quality, although their long sampling chain leads to high computational costs.
Ranked #1 on
Image Generation
on FFHQ 128 x 128
Embodied Agents for Efficient Exploration and Smart Scene Description
The development of embodied agents that can communicate with humans in natural language has gained increasing interest over the last years, as it facilitates the diffusion of robotic platforms in human-populated environments.
Boosting Modern and Historical Handwritten Text Recognition with Deformable Convolutions
Handwritten Text Recognition (HTR) in free-layout pages is a challenging image understanding task that can provide a relevant boost to the digitization of handwritten documents and reuse of their content.
The LAM Dataset: A Novel Benchmark for Line-Level Handwritten Text Recognition
With the aim of fostering the research on this topic, in this paper we present the Ludovico Antonio Muratori (LAM) dataset, a large line-level HTR dataset of Italian ancient manuscripts edited by a single author over 60 years.
Consistency-based Self-supervised Learning for Temporal Anomaly Localization
This work tackles Weakly Supervised Anomaly detection, in which a predictor is allowed to learn not only from normal examples but also from a few labeled anomalies made available during training.
Ranked #11 on
Anomaly Detection In Surveillance Videos
on XD-Violence
Anomaly Detection In Surveillance Videos
Self-Supervised Learning
+3
ALADIN: Distilling Fine-grained Alignment Scores for Efficient Image-Text Matching and Retrieval
In literature, this task is often used as a pre-training objective to forge architectures able to jointly deal with images and texts.
Ranked #22 on
Cross-Modal Retrieval
on COCO 2014
Retrieval-Augmented Transformer for Image Captioning
In this paper, we investigate the development of an image captioning approach with a kNN memory, with which knowledge can be retrieved from an external corpus to aid the generation process.
Maximum Class Separation as Inductive Bias in One Matrix
This paper proposes a simple alternative: encoding maximum separation as an inductive bias in the network by adding one fixed matrix multiplication before computing the softmax activations.
Masked Jigsaw Puzzle: A Versatile Position Embedding for Vision Transformers
In particular, MJP first shuffles the selected patches via our block-wise random jigsaw puzzle shuffle algorithm, and their corresponding PEs are occluded.
SeeFar: Vehicle Speed Estimation and Flow Analysis from a Moving UAV
In this paper, we introduce SeeFar to achieve vehicle speed estimation and traffic flow analysis based on YOLOv5 and DeepSORT from a moving drone.
Goal-driven Self-Attentive Recurrent Networks for Trajectory Prediction
Human trajectory forecasting is a key component of autonomous vehicles, social-aware robots and advanced video-surveillance applications.
Embodied Navigation at the Art Gallery
This feature is challenging for occupancy-based agents which are usually trained in crowded domestic environments with plenty of occupancy information.
Dress Code: High-Resolution Multi-Category Virtual Try-On
Dress Code is more than 3x larger than publicly available datasets for image-based virtual try-on and features high-resolution paired images (1024x768) with front-view, full-body reference models.
Ranked #5 on
Virtual Try-on
on VITON
Spot the Difference: A Novel Task for Embodied Agents in Changing Environments
To make a step towards this setting, we propose Spot the Difference: a novel task for Embodied AI where the agent has access to an outdated map of the environment and needs to recover the correct layout in a fixed time budget.
How many Observations are Enough? Knowledge Distillation for Trajectory Forecasting
To this end, we conceive a novel distillation strategy that allows a knowledge transfer from a teacher network to a student one, the latter fed with fewer observations (just two ones).
CaMEL: Mean Teacher Learning for Image Captioning
Describing images in natural language is a fundamental step towards the automatic modeling of connections between the visual and textual modalities.
Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets
This paper addresses the task of generating fluent descriptions by training on a non-uniform combination of data sources, containing both human-annotated and web-collected captions.
Multi-Category Mesh Reconstruction From Image Collections
Recently, learning frameworks have shown the capability of inferring the accurate shape, pose, and texture of an object from a single RGB image.
Focus on Impact: Indoor Exploration with Intrinsic Motivation
The proposed exploration approach outperforms DRL-based competitors relying on intrinsic rewards and surpasses the agents trained with a dense extrinsic reward computed with the environment layouts.
Working Memory Connections for LSTM
Numerical results suggest that the cell state contains useful information that is worth including in the gate structure.
MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?
Deep learning-based methods for video pedestrian detection and tracking require large volumes of training data to achieve good performance.
From Show to Tell: A Survey on Deep Learning-based Image Captioning
Starting from 2015 the task has generally been addressed with pipelines composed of a visual encoder and a language model for text generation.
SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild
Gesture recognition is a fundamental tool to enable novel interaction paradigms in a variety of application scenarios like Mixed Reality environments, touchless public kiosks, entertainment systems, and more.
Learning to Select: A Fully Attentive Approach for Novel Object Captioning
In this paper, we present a novel approach for NOC that learns to select the most relevant objects of an image, regardless of their adherence to the training set, and to constrain the generative process of a language model accordingly.
Out of the Box: Embodied Navigation in the Real World
In this work, we detail how to transfer the knowledge acquired in simulation into the real world.
Revisiting The Evaluation of Class Activation Mapping for Explainability: A Novel Metric and Experimental Analysis
As the request for deep learning solutions increases, the need for explainability is even more fundamental.
RMS-Net: Regression and Masking for Soccer Event Spotting
The recently proposed action spotting task consists in finding the exact timestamp in which an event occurs.
Ranked #1 on
Action Spotting
on SoccerNet
Inter-Homines: Distance-Based Risk Estimation for Human Safety
In this document, we report our proposal for modeling the risk of possible contagiousity in a given area monitored by RGB cameras where people freely move and interact.
Explore and Explain: Self-supervised Navigation and Recounting
In this paper, we devise a novel embodied setting in which an agent needs to explore a previously unknown environment while recounting what it sees during the path.
Future Urban Scenes Generation Through Vehicles Synthesis
In this work we propose a deep learning pipeline to predict the visual future appearance of an urban scene.
DAG-Net: Double Attentive Graph Neural Network for Trajectory Forecasting
Understanding human motion behaviour is a critical task for several possible applications like self-driving cars or social robots, and in general for all those settings where an autonomous agent has to navigate inside a human-centric environment.
Ranked #1 on
Trajectory Prediction
on STATS SportVu NBA [ATK]
Human motion prediction
Multi-future Trajectory Prediction
+3
AC-VRNN: Attentive Conditional-VRNN for Multi-Future Trajectory Prediction
Anticipating human motion in crowded scenarios is essential for developing intelligent transportation systems, social-aware robots and advanced video surveillance applications.
A Novel Attention-based Aggregation Function to Combine Vision and Language
The joint understanding of vision and language has been recently gaining a lot of attention in both the Computer Vision and Natural Language Processing communities, with the emergence of tasks such as image captioning, image-text matching, and visual question answering.
Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimation
At the core of the proposed method lies our Volumetric Heatmap Autoencoder, a fully-convolutional network tasked with the compression of ground-truth heatmaps into a dense intermediate representation.
Ranked #6 on
3D Human Pose Estimation
on Panoptic
(using extra training data)
Conditional Channel Gated Networks for Task-Aware Continual Learning
Therefore, we additionally introduce a task classifier that predicts the task label of each example, to deal with settings in which a task oracle is not available.
Ranked #3 on
Continual Learning
on ImageNet-50 (5 tasks)
Meshed-Memory Transformer for Image Captioning
Transformer-based architectures represent the state of the art in sequence modeling tasks like machine translation and language understanding.
Ranked #2 on
Image Captioning
on MS COCO
Video action detection by learning graph-based spatio-temporal interactions
Action Detection is a complex task that aims to detect and classify human actions in video clips.
Multimodal Attention Networks for Low-Level Vision-and-Language Navigation
Vision-and-Language Navigation (VLN) is a challenging task in which an agent needs to follow a language-specified path to reach a target destination.
SMArT: Training Shallow Memory-aware Transformers for Robotic Explainability
The ability to generate natural language explanations conditioned on the visual perception is a crucial step towards autonomous agents which can explain themselves and communicate with humans.
Warp and Learn: Novel Views Generation for Vehicles and Other Objects
An Image Completion Network (ICN) is then trained to generate a realistic image starting from this geometric guidance.
Embodied Vision-and-Language Navigation with Dynamic Convolutional Filters
In Vision-and-Language Navigation (VLN), an embodied agent needs to reach a target destination with the only guidance of a natural language instruction.
M-VAD Names: a Dataset for Video Captioning with Naming
Current movie captioning architectures are not capable of mentioning characters with their proper name, replacing them with a generic "someone" tag.
Classifying Signals on Irregular Domains via Convolutional Cluster Pooling
We present a novel and hierarchical approach for supervised classification of signals spanning over a fixed graph, reflecting shared properties of the dataset.
Anomaly Locality in Video Surveillance
This paper strives for the detection of real-world anomalies such as burglaries and assaults in surveillance videos.
Can Adversarial Networks Hallucinate Occluded People With a Plausible Aspect?
When you see a person in a crowd, occluded by other persons, you miss visual information that can be used to recognize, re-identify or simply classify him or her.
Domain Translation with Conditional GANs: from Depth to RGB Face-to-Face
Can faces acquired by low-cost depth sensors be useful to catch some characteristic details of the face?
Show, Control and Tell: A Framework for Generating Controllable and Grounded Captions
Current captioning approaches can describe images using black-box architectures whose behavior is hardly controllable and explainable from the exterior.
Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translation
The applicability of computer vision to real paintings and artworks has been rarely investigated, even though a vast heritage would greatly benefit from techniques which can understand and process data from the artistic domain.
Latent Space Autoregression for Novelty Detection
Novelty detection is commonly referred to as the discrimination of observations that do not conform to a learned model of regularity.
A Graph Transduction Game for Multi-target Tracking
Semi-supervised learning is a popular class of techniques to learn from labeled and unlabeled data.
LAMV: Learning to Align and Match Videos With Kernelized Temporal Layers
This paper considers a learnable approach for comparing and aligning videos.
Learning to Generate Facial Depth Maps
In this paper, an adversarial architecture for facial depth map estimation from monocular intensity images is presented.
Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World
Multi-People Tracking in an open-world setting requires a special effort in precise detection.
Face-from-Depth for Head Pose Estimation on Depth Images
Therefore, we propose a complete framework for the estimation of the head and shoulder pose based on depth images only.
Head Detection with Depth Images in the Wild
Two public datasets have been exploited: the first one, called Pandora, is used to train a deep binary classifier with face and non-face images.
Generative Adversarial Models for People Attribute Recognition in Surveillance
In this paper we propose a deep architecture for detecting people attributes (e. g. gender, race, clothing ...) in surveillance contexts.
Learning to Map Vehicles into Bird's Eye View
Awareness of the road scene is an essential component for both autonomous vehicles and Advances Driver Assistance Systems and is gaining importance both for the academia and car companies.
Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention
Image captioning has been recently gaining a lot of attention thanks to the impressive achievements shown by deep captioning architectures, which combine Convolutional Neural Networks to extract image representations, and Recurrent Neural Networks to generate the corresponding captions.
Ranked #2 on
Image Captioning
on Flickr30k Captions test
(using extra training data)
TransFlow: Unsupervised Motion Flow by Joint Geometric and Pixel-level Estimation
We address unsupervised optical flow estimation for ego-centric motion.
Predicting the Driver's Focus of Attention: the DR(eye)VE Project
In this work we aim to predict the driver's focus of attention.
From Depth Data to Head Pose Estimation: a Siamese approach
In this paper, we tackle the pose estimation problem through a deep learning network working in regression manner.
Fast Gesture Recognition with Multiple Stream Discrete HMMs on 3D Skeletons
HMMs are widely used in action and gesture recognition due to their implementation simplicity, low computational requirement, scalability and high parallelism.
Deep Head Pose Estimation from Depth Data for In-car Automotive Applications
Recently, deep learning approaches have achieved promising results in various fields of computer vision.
POSEidon: Face-from-Depth for Driver Pose Estimation
In this work, we present a new deep learning framework for head localization and pose estimation on depth images.
Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model
Data-driven saliency has recently gained a lot of attention thanks to the use of Convolutional Neural Networks for predicting gaze fixations.
Hierarchical Boundary-Aware Neural Encoder for Video Captioning
The use of Recurrent Neural Networks for video captioning has recently gained a lot of attention, since they can be used both to encode the input video and to generate the corresponding description.
Learning Where to Attend Like a Human Driver
Despite the advent of autonomous cars, it's likely - at least in the near future - that human attention will still maintain a central role as a guarantee in terms of legal responsibility during the driving task.
Recognizing and Presenting the Storytelling Video Structure with Deep Multimodal Networks
This paper presents a novel approach for temporal and semantic segmentation of edited videos into meaningful segments, from the point of view of the storytelling structure.
Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking
To help accelerate progress in multi-target, multi-camera tracking systems, we present (i) a new pair of precision-recall measures of performance that treats errors of all types uniformly and emphasizes correct identification over sources of error; (ii) the largest fully-annotated and calibrated data set to date with more than 2 million frames of 1080p, 60fps video taken by 8 cameras observing more than 2, 700 identities over 85 minutes; and (iii) a reference software system as a comparison baseline.
A Deep Multi-Level Network for Saliency Prediction
Current state of the art models for saliency prediction employ Fully Convolutional networks that perform a non-linear combination of features extracted from the last convolutional layer to predict saliency maps.
Video Registration in Egocentric Vision under Day and Night Illumination Changes
To effectively register an egocentric video sequence under these conditions, we propose to tackle the source of the problem: the matching process.
Scene-driven Retrieval in Edited Videos using Aesthetic and Semantic Deep Features
This paper presents a novel retrieval pipeline for video collections, which aims to retrieve the most significant parts of an edited video for a given query, and represent them with thumbnails which are at the same time semantically meaningful and aesthetically remarkable.
A Deep Siamese Network for Scene Detection in Broadcast Videos
We present a model that automatically divides broadcast videos into coherent scenes by learning a distance measure between shots.
Learning to Divide and Conquer for Online Multi-Target Tracking
Online Multiple Target Tracking (MTT) is often addressed within the tracking-by-detection paradigm.
Socially Constrained Structural Learning for Groups Detection in Crowd
Modern crowd theories agree that collective behavior is the result of the underlying interactions among small groups of individuals.
