Search Results for author:
Found 47 papers, 34 papers with code
Adversarial Learning for Feature Shift Detection and Correction
Data shift is a phenomenon present in many real-world applications, and while there are multiple methods attempting to detect shifts, the task of localizing and correcting the features originating such shifts has not been studied in depth.
Towards Robust Image-in-Audio Deep Steganography
The field of steganography has experienced a surge of interest due to the recent advancements in AI-powered techniques, particularly in the context of multimodal setups that enable the concealment of signals within signals of a different nature.
Ranked #1 on
Image Reconstruction
on Audio Set
SIRA: Relightable Avatars from a Single Image
Our key ingredients are two data-driven statistical models based on neural fields that resolve the ambiguities of single-view 3D surface reconstruction and appearance factorization.
Topic Detection in Continuous Sign Language Videos
Significant progress has been made recently on challenging tasks in automatic sign language understanding, such as sign language recognition, translation and production.
H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction
In this paper, we tackle these limitations for the specific problem of few-shot full 3D head reconstruction, by endowing coordinate-based representations with a probabilistic shape prior that enables faster convergence and better generalization when using few input images (down to three).
Unsupervised Skill-Discovery and Skill-Learning in Minecraft
Pre-training Reinforcement Learning agents in a task-agnostic manner has shown promising results.
SynthRef: Generation of Synthetic Referring Expressions for Object Segmentation
Recent advances in deep learning have brought significant progress in visual grounding tasks such as language-guided video object segmentation.
Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data
Transfer learning approaches can reduce the data requirements of deep learning algorithms.
Ranked #4 on
Change Detection
on OSCD - 13ch
(using extra training data)
Can Everybody Sign Now? Exploring Sign Language Video Generation from 2D Poses
Recent work have addressed the generation of human poses represented by 2D/3D coordinates of human joints for sign language.
RefVOS: A Closer Look at Referring Expressions for Video Object Segmentation
The task of video object segmentation with referring expressions (language-guided VOS) is to, given a linguistic phrase and a video, generate binary masks for the object to which the phrase refers.
Ranked #1 on
Referring Expression Segmentation
on A2Dre test
Mask-guided sample selection for Semi-Supervised Instance Segmentation
Our method consists in first predicting pseudo-masks for the unlabeled pool of samples, together with a score predicting the quality of the mask.
How2Sign: A Large-scale Multimodal Dataset for Continuous American Sign Language
Towards this end, we introduce How2Sign, a multimodal and multiview continuous American Sign Language (ASL) dataset, consisting of a parallel corpus of more than 80 hours of sign language videos and a set of corresponding modalities including speech, English transcripts, and depth.
Transcription-Enriched Joint Embeddings for Spoken Descriptions of Images and Videos
In this work, we propose an effective approach for training unique embedding representations by combining three simultaneous modalities: image and spoken and textual narratives.
Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills
We perform an extensive evaluation of skill discovery methods on controlled environments and show that EDL offers significant advantages, such as overcoming the coverage problem, reducing the dependence of learned skills on the initial state, and allowing the user to define a prior over which behaviors should be learned.
Recurrent Instance Segmentation using Sequences of Referring Expressions
The goal of this work is to segment the objects in an image that are referred to by a sequence of linguistic descriptions (referring expressions).
Automatic Reminiscence Therapy for Dementia
With people living longer than ever, the number of cases with dementia such as Alzheimer's disease increases steadily.
Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation
This work addresses the challenge of hate speech detection in Internet memes, and attempts using visual information to automatically detect hate speech, unlike any previous work of our knowledge.
Simple vs complex temporal recurrences for video saliency prediction
This paper investigates modifying an existing neural network architecture for static saliency prediction using two types of recurrences that integrate information from the temporal domain.
Budget-aware Semi-Supervised Semantic and Instance Segmentation
Methods that move towards less supervised scenarios are key for image segmentation, as dense labels demand significant human intervention.
Wav2Pix: Speech-conditioned Face Generation using Generative Adversarial Networks
Speech is a rich biometric signal that contains information about the identity, gender and emotional state of the speaker.
RVOS: End-to-End Recurrent Network for Video Object Segmentation
Multiple object video object segmentation is a challenging task, specially for the zero-shot case, when no object mask is given at the initial frame and the model has to find the objects to be segmented along the sequence.
Ranked #1 on
One-shot visual object segmentation
on YouTube-VOS
Inverse Cooking: Recipe Generation from Food Images
Our system predicts ingredients as sets by means of a novel architecture, modeling their dependencies without imposing any order, and then generates cooking instructions by attending to both image and its inferred ingredients simultaneously.
Ranked #1 on
Recipe Generation
on Recipe1M
Importance Weighted Evolution Strategies
Evolution Strategies (ES) emerged as a scalable alternative to popular Reinforcement Learning (RL) techniques, providing an almost perfect speedup when distributed across hundreds of CPU cores thanks to a reduced communication overhead.
PathGAN: Visual Scanpath Prediction with Generative Adversarial Networks
We introduce PathGAN, a deep neural network for visual scanpath prediction trained on adversarial examples.
Temporal Saliency Adaptation in Egocentric Videos
This work adapts a deep neural model for image saliency prediction to the temporal domain of egocentric video.
Comparing Fixed and Adaptive Computation Time for Recurrent Neural Networks
Adaptive Computation Time for Recurrent Neural Networks (ACT) is one of the most promising architectures for variable computation.
Online Detection of Action Start in Untrimmed, Streaming Videos
We aim to tackle a novel task in action detection - Online Detection of Action Start (ODAS) in untrimmed, streaming videos.
Recurrent Neural Networks for Semantic Instance Segmentation
We present a recurrent model for semantic instance segmentation that sequentially generates binary masks and their associated class probabilities for every object in an image.
Detection-aided liver lesion segmentation using deep learning
A fully automatic technique for segmenting the liver and localizing its unhealthy tissues is a convenient tool in order to diagnose hepatic diseases and assess the response to the according treatments.
Saliency Weighted Convolutional Features for Instance Search
This work explores attention models to weight the contribution of local convolutional representations for the instance search task.
Cost-Effective Active Learning for Melanoma Segmentation
We propose a novel Active Learning framework capable to train effectively a convolutional neural network for semantic segmentation of medical imaging, with a limited amount of training labeled data.
Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
We introduce the Skip RNN model which extends existing RNN models by learning to skip state updates and shortens the effective size of the computational graph.
More cat than cute? Interpretable Prediction of Adjective-Noun Pairs
This work aims at disentangling the contributions of the `adjectives' and `nouns' in the visual prediction of ANPs.
Disentangling Motion, Foreground and Background Features in Videos
This paper introduces an unsupervised framework to extract semantically rich features for video representation.
SaltiNet: Scan-path Prediction on 360 Degree Images using Saliency Volumes
The first part of the network consists of a model trained to generate saliency volumes, whose parameters are fit by back-propagation computed from a binary cross entropy (BCE) loss over downsampled versions of the saliency volumes.
Class-Weighted Convolutional Features for Visual Instance Search
In this paper, we go beyond this spatial information and propose a local-aware encoding of convolutional features based on semantic information predicted in the target image.
SalGAN: Visual Saliency Prediction with Generative Adversarial Networks
We introduce SalGAN, a deep convolutional neural network for visual saliency prediction trained with adversarial examples.
Hierarchical Object Detection with Deep Reinforcement Learning
We argue that, while this loss seems unavoidable when working with large amounts of object candidates, the much more reduced amount of region proposals generated by our reinforcement learning agent allows considering to extract features for each location without sharing convolutional computation among regions.
Open-Ended Visual Question-Answering
This thesis report studies methods to solve Visual Question-Answering (VQA) tasks with a Deep Learning framework.
Where is my Phone ? Personal Object Retrieval from Egocentric Images
This work presents a retrieval pipeline and evaluation scheme for the problem of finding the last appearance of personal objects in a large dataset of images captured from a wearable camera.
Temporal Activity Detection in Untrimmed Videos with Recurrent Neural Networks
This thesis explore different approaches using Convolutional and Recurrent Neural Networks to classify and temporally localize activities on videos, furthermore an implementation to achieve it has been proposed.
Faster R-CNN Features for Instance Search
This work explores the suitability for instance retrieval of image- and region-wise representations pooled from an object detection CNN such as Faster R-CNN.
Bags of Local Convolutional Features for Scalable Instance Search
This work proposes a simple instance retrieval pipeline based on encoding the convolutional features of CNN using the bag of words aggregation scheme (BoW).
From Pixels to Sentiment: Fine-tuning CNNs for Visual Sentiment Prediction
Visual multimedia have become an inseparable part of our digital social lives, and they often capture moments tied with deep affections.
Shallow and Deep Convolutional Networks for Saliency Prediction
The prediction of salient areas in images has been traditionally addressed with hand-crafted features based on neuroscience principles.
Cultural Event Recognition with Visual ConvNets and Temporal Models
Our solution is based on the combination of visual features extracted from convolutional neural networks with temporal information using a hierarchical classifier scheme.
