Search Results for author: Nicu Sebe

Found 212 papers, 132 papers with code

Weakly-Supervised Crowd Counting Learns from Sorting rather than Locations

no code implementations ECCV 2020 Yifan Yang, Guorong Li, Zhe Wu, Li Su, Qingming Huang, Nicu Sebe

We propose a soft-label sorting network along with the counting network, which sorts the given images by their crowd numbers.

Crowd Counting

A Lie Group Approach to Riemannian Batch Normalization

1 code implementation17 Mar 2024 Ziheng Chen, Yue Song, Yunmei Liu, Nicu Sebe

Using the deformation concept, we generalize the existing Lie groups on SPD manifolds into three families of parameterized Lie groups.

Action Recognition EEG +1

Textual Knowledge Matters: Cross-Modality Co-Teaching for Generalized Visual Class Discovery

no code implementations12 Mar 2024 Haiyang Zheng, Nan Pu, Wenjing Li, Nicu Sebe, Zhun Zhong

In this paper, we study the problem of Generalized Category Discovery (GCD), which aims to cluster unlabeled data from both known and unknown categories using the knowledge of labeled data from known categories.

Descriptive Retrieval +1

Uncertainty-Aware Testing-Time Optimization for 3D Human Pose Estimation

no code implementations4 Feb 2024 Ti Wang, Mengyuan Liu, Hong Liu, Bin Ren, Yingxuan You, Wenhao Li, Nicu Sebe, Xia Li

We observe that previous optimization-based methods commonly rely on projection constraint, which only ensures alignment in 2D space, potentially leading to the overfitting problem.

3D Human Pose Estimation

Key-Graph Transformer for Image Restoration

no code implementations4 Feb 2024 Bin Ren, Yawei Li, Jingyun Liang, Rakesh Ranjan, Mengyuan Liu, Rita Cucchiara, Luc van Gool, Nicu Sebe

While it is crucial to capture global information for effective image restoration (IR), integrating such cues into transformer-based methods becomes computationally expensive, especially with high input resolution.

Graph Attention Image Restoration

Democratizing Fine-grained Visual Recognition with Large Language Models

no code implementations24 Jan 2024 Mingxuan Liu, Subhankar Roy, Wenjing Li, Zhun Zhong, Nicu Sebe, Elisa Ricci

Identifying subordinate-level categories from images is a longstanding task in computer vision and is referred to as fine-grained visual recognition (FGVR).

Fine-Grained Visual Recognition World Knowledge

Bilateral Reference for High-Resolution Dichotomous Image Segmentation

1 code implementation7 Jan 2024 Peng Zheng, Dehong Gao, Deng-Ping Fan, Li Liu, Jorma Laaksonen, Wanli Ouyang, Nicu Sebe

It comprises two essential components: the localization module (LM) and the reconstruction module (RM) with our proposed bilateral reference (BiRef).

 Ranked #1 on RGB Salient Object Detection on HRSOD (using extra training data)

Camouflaged Object Segmentation Dichotomous Image Segmentation +3

VASE: Object-Centric Appearance and Shape Manipulation of Real Videos

no code implementations4 Jan 2024 Elia Peruzzo, Vidit Goel, Dejia Xu, Xingqian Xu, Yifan Jiang, Zhangyang Wang, Humphrey Shi, Nicu Sebe

Recently, several works tackled the video editing task fostered by the success of large-scale text-to-image generative models.

Video Editing

Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation

1 code implementation11 Dec 2023 Dong Zhao, Ruizhi Yang, Shuang Wang, Qi Zang, Yang Hu, Licheng Jiao, Nicu Sebe, Zhun Zhong

This approach formulates pseudo-labels at the connectivity level and thus can facilitate learning structured and low-noise semantics.

Domain Adaptation Semantic Segmentation

Zero-Shot Point Cloud Registration

no code implementations5 Dec 2023 Weijie Wang, Guofeng Mei, Bin Ren, Xiaoshui Huang, Fabio Poiesi, Luc van Gool, Nicu Sebe, Bruno Lepri

The cornerstone of ZeroReg is the novel transfer of image features from keypoints to the point cloud, enriched by aggregating information from 3D geometric neighborhoods.

Point Cloud Registration

RankFeat&RankWeight: Rank-1 Feature/Weight Removal for Out-of-distribution Detection

1 code implementation23 Nov 2023 Yue Song, Nicu Sebe, Wei Wang

This observation motivates us to propose \texttt{RankFeat}, a simple yet effective \emph{post hoc} approach for OOD detection by removing the rank-1 matrix composed of the largest singular value and the associated singular vectors from the high-level feature.

Out-of-Distribution Detection Out of Distribution (OOD) Detection

Hourglass Tokenizer for Efficient Transformer-Based 3D Human Pose Estimation

1 code implementation20 Nov 2023 Wenhao Li, Mengyuan Liu, Hong Liu, Pichao Wang, Jialun Cai, Nicu Sebe

Transformers have been successfully applied in the field of video-based 3D human pose estimation.

3D Human Pose Estimation

Flow Factorized Representation Learning

1 code implementation NeurIPS 2023 Yue Song, T. Anderson Keller, Nicu Sebe, Max Welling

A prominent goal of representation learning research is to achieve representations which are factorized in a useful manner with respect to the ground truth factors of variation.

Disentanglement

CNNs for JPEGs: A Study in Computational Cost

no code implementations20 Sep 2023 Samuel Felipe dos Santos, Nicu Sebe, Jurandy Almeida

In this paper, we propose a further study of the computational cost of deep models designed for the frequency domain, evaluating the cost of decoding and passing the images through the network.

Tightening Classification Boundaries in Open Set Domain Adaptation through Unknown Exploitation

no code implementations16 Sep 2023 Lucas Fernando Alvarenga e Silva, Nicu Sebe, Jurandy Almeida

Convolutional Neural Networks (CNNs) have brought revolutionary advances to many research areas due to their capacity of learning from raw data.

Data Augmentation Domain Adaptation

Turn Fake into Real: Adversarial Head Turn Attacks Against Deepfake Detection

no code implementations3 Sep 2023 Weijie Wang, Zhengyu Zhao, Nicu Sebe, Bruno Lepri

Although effective deepfake detectors have been proposed, they are substantially vulnerable to adversarial attacks.

DeepFake Detection Face Swapping

Compositional Semantic Mix for Domain Adaptation in Point Cloud Segmentation

1 code implementation28 Aug 2023 Cristiano Saltori, Fabio Galasso, Giuseppe Fiameni, Nicu Sebe, Fabio Poiesi, Elisa Ricci

In this study, we introduce compositional semantic mixing for point cloud domain adaptation, representing the first unsupervised domain adaptation technique for point cloud segmentation based on semantic and geometric sample mixing.

Point Cloud Completion Point Cloud Segmentation +2

Interactive Neural Painting

no code implementations31 Jul 2023 Elia Peruzzo, Willi Menapace, Vidit Goel, Federica Arrigoni, Hao Tang, Xingqian Xu, Arman Chopikyan, Nikita Orlov, Yuxiao Hu, Humphrey Shi, Nicu Sebe, Elisa Ricci

This paper advances the state of the art in this emerging research domain by proposing the first approach for Interactive NP.

Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesis

1 code implementation22 Jul 2023 Hao Tang, Guolei Sun, Nicu Sebe, Luc van Gool

To tackle 2), we design an effective module to selectively highlight class-dependent feature maps according to the original semantic layout to preserve the semantic information.

Contrastive Learning Image Generation

Let's ViCE! Mimicking Human Cognitive Behavior in Image Generation Evaluation

no code implementations18 Jul 2023 Federico Betti, Jacopo Staiano, Lorenzo Baraldi, Rita Cucchiara, Nicu Sebe

Research in Image Generation has recently made significant progress, particularly boosted by the introduction of Vision-Language models which are able to produce high-quality visual content based on textual inputs.

Image Generation Question Answering +1

Householder Projector for Unsupervised Latent Semantics Discovery

1 code implementation ICCV 2023 Yue Song, Jichao Zhang, Nicu Sebe, Wei Wang

Generative Adversarial Networks (GANs), especially the recent style-based generators (StyleGANs), have versatile semantics in the structured latent space.

T2TD: Text-3D Generation Model based on Prior Knowledge Guidance

no code implementations25 May 2023 Weizhi Nie, Ruidong Chen, Weijie Wang, Bruno Lepri, Nicu Sebe

Meanwhile, to effectively integrate multi-modal prior knowledge into textual information, we adopt a novel multi-layer transformer structure to progressively fuse related shape and textual information, which can effectively compensate for the lack of structural information in the text and enhance the final performance of the 3D generation model.

3D Reconstruction Causal Inference

Federated Generalized Category Discovery

no code implementations23 May 2023 Nan Pu, Zhun Zhong, Xinyuan Ji, Nicu Sebe

On each client, GCL builds class-level contrastive learning with both local and global GMMs.

Contrastive Learning

Riemannian Multinomial Logistics Regression for SPD Neural Networks

1 code implementation18 May 2023 Ziheng Chen, Yue Song, Gaowen Liu, Ramana Rao Kompella, XiaoJun Wu, Nicu Sebe

Besides, our framework offers a novel intrinsic explanation for the most popular LogEig classifier in existing SPD networks.

Action Recognition EEG +2

Latent Traversals in Generative Models as Potential Flows

1 code implementation25 Apr 2023 Yue Song, T. Anderson Keller, Nicu Sebe, Max Welling

In this work, we instead propose to model latent structures with a learned dynamic potential landscape, thereby performing latent traversals as the flow of samples down the landscape's gradient.

Disentanglement Inductive Bias

PAIR-Diffusion: A Comprehensive Multimodal Object-Level Image Editor

1 code implementation30 Mar 2023 Vidit Goel, Elia Peruzzo, Yifan Jiang, Dejia Xu, Xingqian Xu, Nicu Sebe, Trevor Darrell, Zhangyang Wang, Humphrey Shi

We propose \textbf{PAIR} Diffusion, a generic framework that can enable a diffusion model to control the structure and appearance properties of each object in the image.

Object

Dynamic Conceptional Contrastive Learning for Generalized Category Discovery

1 code implementation CVPR 2023 Nan Pu, Zhun Zhong, Nicu Sebe

This leads traditional novel category discovery (NCD) methods to be incapacitated for GCD, due to their assumption of unlabeled data are only from novel categories.

Contrastive Learning Fine-Grained Visual Recognition +2

Large-scale Pre-trained Models are Surprisingly Strong in Incremental Novel Class Discovery

1 code implementation28 Mar 2023 Mingxuan Liu, Subhankar Roy, Zhun Zhong, Nicu Sebe, Elisa Ricci

Discovering novel concepts from unlabelled data and in a continuous manner is an important desideratum of lifelong learners.

Novel Class Discovery Novel Concepts

Adaptive Riemannian Metrics on SPD Manifolds

no code implementations26 Mar 2023 Ziheng Chen, Yue Song, Tianyang Xu, Zhiwu Huang, Xiao-Jun Wu, Nicu Sebe

Symmetric Positive Definite (SPD) matrices have received wide attention in machine learning due to their intrinsic capacity of encoding underlying structural correlation in data.

Attribute-preserving Face Dataset Anonymization via Latent Code Optimization

1 code implementation CVPR 2023 Simone Barattin, Christos Tzelepis, Ioannis Patras, Nicu Sebe

By optimizing the latent codes directly, we ensure both that the identity is of a desired distance away from the original (with an identity obfuscation loss), whilst preserving the facial attributes (using a novel feature-matching loss in FaRL's deep feature space).

Attribute

SpectralCLIP: Preventing Artifacts in Text-Guided Style Transfer from a Spectral Perspective

1 code implementation16 Mar 2023 Zipeng Xu, Songlong Xing, Enver Sangineto, Nicu Sebe

However, directly using CLIP to guide style transfer leads to undesirable artifacts (mainly written words and unrelated visual entities) spread over the image.

Image Generation Style Transfer

StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized Tokenizer of a Large-Scale Generative Model

1 code implementation ICCV 2023 Zipeng Xu, Enver Sangineto, Nicu Sebe

Despite the progress made in the style transfer task, most previous work focus on transferring only relatively simple features like color or texture, while missing more abstract concepts such as overall art expression or painter-specific traits.

Style Transfer

Graph Transformer GANs for Graph-Constrained House Generation

no code implementations CVPR 2023 Hao Tang, Zhenyu Zhang, Humphrey Shi, Bo Li, Ling Shao, Nicu Sebe, Radu Timofte, Luc van Gool

We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for the challenging graph-constrained house generation task.

Generative Adversarial Network House Generation +1

Logit Margin Matters: Improving Transferable Targeted Adversarial Attack by Logit Calibration

2 code implementations7 Mar 2023 Juanjuan Weng, Zhiming Luo, Zhun Zhong, Shaozi Li, Nicu Sebe

In this work, we provide a comprehensive investigation of the CE loss function and find that the logit margin between the targeted and untargeted classes will quickly obtain saturation in CE, which largely limits the transferability.

Adversarial Attack

Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models

no code implementations10 Jan 2023 Mengyi Zhao, Mengyuan Liu, Bin Ren, Shuling Dai, Nicu Sebe

Diffusion-based generative models have recently emerged as powerful solutions for high-quality synthesis in multiple domains.

Denoising

Dynamically Instance-Guided Adaptation: A Backward-Free Approach for Test-Time Domain Adaptive Semantic Segmentation

1 code implementation CVPR 2023 Wei Wang, Zhun Zhong, Weijie Wang, Xi Chen, Charles Ling, Boyu Wang, Nicu Sebe

In this paper, we study the application of Test-time domain adaptation in semantic segmentation (TTDA-Seg) where both efficiency and effectiveness are crucial.

Domain Adaptation Semantic Segmentation

Style-Hallucinated Dual Consistency Learning: A Unified Framework for Visual Domain Generalization

1 code implementation18 Dec 2022 Yuyang Zhao, Zhun Zhong, Na Zhao, Nicu Sebe, Gim Hee Lee

Furthermore, we present a novel style hallucination module (SHM) to generate style-diversified samples that are essential to consistency learning.

Domain Generalization Hallucination +4

Orthogonal SVD Covariance Conditioning and Latent Disentanglement

1 code implementation11 Dec 2022 Yue Song, Nicu Sebe, Wei Wang

Extensive experiments on visual recognition demonstrate that our methods can simultaneously improve covariance conditioning and generalization.

Disentanglement

A Structure-Guided Diffusion Model for Large-Hole Image Completion

1 code implementation18 Nov 2022 Daichi Horita, Jiaolong Yang, Dong Chen, Yuki Koyama, Kiyoharu Aizawa, Nicu Sebe

The structure generator generates an edge image representing plausible structures within the holes, which is then used for guiding the texture generation process.

Denoising Texture Synthesis

Bipartite Graph Reasoning GANs for Person Pose and Facial Image Synthesis

1 code implementation12 Nov 2022 Hao Tang, Ling Shao, Philip H. S. Torr, Nicu Sebe

To further capture the change in pose of each part more precisely, we propose a novel part-aware bipartite graph reasoning (PBGR) block to decompose the task of reasoning the global structure transformation with a bipartite graph into learning different local transformations for different semantic body/face parts.

Generative Adversarial Network Image Generation

Deep Unsupervised Key Frame Extraction for Efficient Video Classification

no code implementations12 Nov 2022 Hao Tang, Lei Ding, Songsong Wu, Bin Ren, Nicu Sebe, Paolo Rota

The proposed TSDPC is a generic and powerful framework and it has two advantages compared with previous works, one is that it can calculate the number of key frames automatically.

Classification Video Classification

Overlap-guided Gaussian Mixture Models for Point Cloud Registration

1 code implementation17 Oct 2022 Guofeng Mei, Fabio Poiesi, Cristiano Saltori, Jian Zhang, Elisa Ricci, Nicu Sebe

Probabilistic 3D point cloud registration methods have shown competitive performance in overcoming noise, outliers, and density variations.

Point Cloud Registration

Budget-Aware Pruning for Multi-Domain Learning

no code implementations14 Oct 2022 Samuel Felipe dos Santos, Rodrigo Berriel, Thiago Oliveira-Santos, Nicu Sebe, Jurandy Almeida

Nevertheless, the models are usually larger than the baseline for a single domain.

Data Augmentation-free Unsupervised Learning for 3D Point Cloud Understanding

1 code implementation6 Oct 2022 Guofeng Mei, Cristiano Saltori, Fabio Poiesi, Jian Zhang, Elisa Ricci, Nicu Sebe, Qiang Wu

Unsupervised learning on 3D point clouds has undergone a rapid evolution, especially thanks to data augmentation-based contrastive methods.

3D Object Classification Contrastive Learning +3

Vision+X: A Survey on Multimodal Learning in the Light of Data

no code implementations5 Oct 2022 Ye Zhu, Yu Wu, Nicu Sebe, Yan Yan

We are perceiving and communicating with the world in a multisensory manner, where different information sources are sophisticatedly processed and interpreted by separate parts of the human brain to constitute a complex, yet harmonious and unified sensing system.

Representation Learning

Rethinking the Learning Paradigm for Facial Expression Recognition

no code implementations30 Sep 2022 Weijie Wang, Nicu Sebe, Bruno Lepri

Due to the subjective crowdsourcing annotations and the inherent inter-class similarity of facial expressions, the real-world Facial Expression Recognition (FER) datasets usually exhibit ambiguous annotation.

Facial Expression Recognition Facial Expression Recognition (FER)

RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection

1 code implementation18 Sep 2022 Yue Song, Nicu Sebe, Wei Wang

The task of out-of-distribution (OOD) detection is crucial for deploying machine learning models in real-world settings.

Out-of-Distribution Detection

Facial Expression Translation using Landmark Guided GANs

1 code implementation5 Sep 2022 Hao Tang, Nicu Sebe

We propose a simple yet powerful Landmark guided Generative Adversarial Network (LandmarkGAN) for the facial expression-to-expression translation using a single image, which is an important and challenging task in computer vision since the expression-to-expression translation is a non-linear and non-aligned problem.

Facial Expression Translation Generative Adversarial Network +1

Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generation

1 code implementation26 Aug 2022 Jichao Zhang, Aliaksandr Siarohin, Yahui Liu, Hao Tang, Nicu Sebe, Wei Wang

Generative Neural Radiance Fields (GNeRF) based 3D-aware GANs have demonstrated remarkable capabilities in generating high-quality images while maintaining strong 3D consistency.

Attribute Disentanglement +2

Uncertainty-guided Source-free Domain Adaptation

1 code implementation16 Aug 2022 Subhankar Roy, Martin Trapp, Andrea Pilzer, Juho Kannala, Nicu Sebe, Elisa Ricci, Arno Solin

Source-free domain adaptation (SFDA) aims to adapt a classifier to an unlabelled target data set by only using a pre-trained source model.

Source-Free Domain Adaptation

CoSMix: Compositional Semantic Mix for Domain Adaptation in 3D LiDAR Segmentation

2 code implementations20 Jul 2022 Cristiano Saltori, Fabio Galasso, Giuseppe Fiameni, Nicu Sebe, Elisa Ricci, Fabio Poiesi

We propose a new approach of sample mixing for point cloud UDA, namely Compositional Semantic Mix (CoSMix), the first UDA approach for point cloud segmentation based on sample mixing.

3D Unsupervised Domain Adaptation Autonomous Driving +5

Class-incremental Novel Class Discovery

1 code implementation18 Jul 2022 Subhankar Roy, Mingxuan Liu, Zhun Zhong, Nicu Sebe, Elisa Ricci

We study the new task of class-incremental Novel Class Discovery (class-iNCD), which refers to the problem of discovering novel categories in an unlabelled data set by leveraging a pre-trained model that has been trained on a labelled data set containing disjoint yet related categories.

Incremental Learning Knowledge Distillation +1

Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation

1 code implementation11 Jul 2022 Zhun Zhong, Yuyang Zhao, Gim Hee Lee, Nicu Sebe

Experiments on two synthetic-to-real semantic segmentation benchmarks demonstrate that AdvStyle can significantly improve the model performance on unseen real domains and show that we can achieve the state of the art.

Domain Generalization Image Classification +1

Batch-efficient EigenDecomposition for Small and Medium Matrices

1 code implementation9 Jul 2022 Yue Song, Nicu Sebe, Wei Wang

EigenDecomposition (ED) is at the heart of many computer vision algorithms and applications.

Image Generation

PI-Trans: Parallel-ConvMLP and Implicit-Transformation Based GAN for Cross-View Image Translation

1 code implementation9 Jul 2022 Bin Ren, Hao Tang, Yiming Wang, Xia Li, Wei Wang, Nicu Sebe

For semantic-guided cross-view image translation, it is crucial to learn where to sample pixels from the source view image and where to reallocate them guided by the target view semantic map, especially when there is little overlap or drastic view difference between the source and target images.

Generative Adversarial Network

Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality

1 code implementation5 Jul 2022 Yue Song, Nicu Sebe, Wei Wang

Inserting an SVD meta-layer into neural networks is prone to make the covariance ill-conditioned, which could harm the model in the training stability and generalization abilities.

Interaction Transformer for Human Reaction Generation

1 code implementation4 Jul 2022 Baptiste Chopin, Hao Tang, Naima Otberdout, Mohamed Daoudi, Nicu Sebe

To address this limitation, we propose a novel interaction Transformer (InterFormer) consisting of a Transformer network with both temporal and spatial attention.

Unsupervised High-Resolution Portrait Gaze Correction and Animation

1 code implementation1 Jul 2022 Jichao Zhang, Jingjing Chen, Hao Tang, Enver Sangineto, Peng Wu, Yan Yan, Nicu Sebe, Wei Wang

Solving this problem using an unsupervised method remains an open problem, especially for high-resolution face images in the wild, which are not easy to annotate with gaze and head pose labels.

Image Inpainting Vocal Bursts Intensity Prediction

Spatial Entropy as an Inductive Bias for Vision Transformers

1 code implementation9 Jun 2022 Elia Peruzzo, Enver Sangineto, Yahui Liu, Marco De Nadai, Wei Bi, Bruno Lepri, Nicu Sebe

In this work, we propose a different and complementary direction, in which a local bias is introduced using an auxiliary self-supervised task, performed jointly with standard supervised training.

Inductive Bias Semantic Segmentation

Masked Jigsaw Puzzle: A Versatile Position Embedding for Vision Transformers

1 code implementation CVPR 2023 Bin Ren, Yahui Liu, Yue Song, Wei Bi, Rita Cucchiara, Nicu Sebe, Wei Wang

In particular, MJP first shuffles the selected patches via our block-wise random jigsaw puzzle shuffle algorithm, and their corresponding PEs are occluded.

Federated Learning Position

LeRaC: Learning Rate Curriculum

no code implementations18 May 2022 Florinel-Alin Croitoru, Nicolae-Catalin Ristea, Radu Tudor Ionescu, Nicu Sebe

In this work, we propose a novel curriculum learning approach termed Learning Rate Curriculum (LeRaC), which leverages the use of a different learning rate for each layer of a neural network to create a data-free curriculum during the initial training epochs.

Audio Classification QNLI +2

Joint Representation Learning and Keypoint Detection for Cross-view Geo-localization

1 code implementation IEEE Transactions on Image Processing (TIP) 2022 Jinliang Lin, Zhedong Zheng, Zhun Zhong, Zhiming Luo, Shaozi Li, Yi Yang, Nicu Sebe

Inspired by the human visual system for mining local patterns, we propose a new framework called RK-Net to jointly learn the discriminative Representation and detect salient Keypoints with a single Network.

Drone navigation Drone-view target localization +3

Temporal Alignment for History Representation in Reinforcement Learning

1 code implementation7 Apr 2022 Aleksandr Ermolov, Enver Sangineto, Nicu Sebe

Inspired by human memory, we propose to represent history with only important changes in the environment and, in our approach, to obtain automatically this representation using self-supervision.

Atari Games reinforcement-learning +1

Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation

2 code implementations6 Apr 2022 Yuyang Zhao, Zhun Zhong, Na Zhao, Nicu Sebe, Gim Hee Lee

Furthermore, we present a novel style hallucination module (SHM) to generate style-diversified samples that are essential to consistency learning.

Domain Generalization Hallucination +3

Cross-View Panorama Image Synthesis

1 code implementation22 Mar 2022 Songsong Wu, Hao Tang, Xiao-Yuan Jing, Haifeng Zhao, Jianjun Qian, Nicu Sebe, Yan Yan

In this paper, we tackle the problem of synthesizing a ground-view panorama image conditioned on a top-view aerial image, which is a challenging problem due to the large gap between the two image domains with different view-points.

Image Generation

Federated and Generalized Person Re-identification through Domain and Feature Hallucinating

no code implementations5 Mar 2022 Fengxiang Yang, Zhun Zhong, Zhiming Luo, Shaozi Li, Nicu Sebe

During local training, the DFS are used to synthesize novel domain statistics with the proposed domain hallucinating, which is achieved by re-weighting DFS with random weights.

Domain Generalization Person Re-Identification

Cross-Modality Earth Mover's Distance for Visible Thermal Person Re-Identification

no code implementations3 Mar 2022 Yongguo Ling, Zhun Zhong, Donglin Cao, Zhiming Luo, Yaojin Lin, Shaozi Li, Nicu Sebe

In this manner, the model will focus on reducing the inter-modality discrepancy while paying less attention to intra-identity variations, leading to a more effective modality alignment.

Person Re-Identification

Local and Global GANs with Semantic-Aware Upsampling for Image Generation

1 code implementation28 Feb 2022 Hao Tang, Ling Shao, Philip H. S. Torr, Nicu Sebe

To learn more discriminative class-specific feature representations for the local generation, we also propose a novel classification module.

Feature Upsampling Image Generation

Relation Regularized Scene Graph Generation

no code implementations22 Feb 2022 Yuyu Guo, Lianli Gao, Jingkuan Song, Peng Wang, Nicu Sebe, Heng Tao Shen, Xuelong Li

Inspired by this observation, in this article, we propose a relation regularized network (R2-Net), which can predict whether there is a relationship between two objects and encode this relation into object feature refinement and better SGG.

Graph Classification Graph Generation +6

Disentangle Saliency Detection into Cascaded Detail Modeling and Body Filling

no code implementations8 Feb 2022 Yue Song, Hao Tang, Nicu Sebe, Wei Wang

Specifically, the detail modeling focuses on capturing the object edges by supervision of explicitly decomposed detail label that consists of the pixels that are nested on the edge and near the edge.

object-detection Object Detection +2

Fast Differentiable Matrix Square Root and Inverse Square Root

1 code implementation29 Jan 2022 Yue Song, Nicu Sebe, Wei Wang

Computing the matrix square root and its inverse in a differentiable manner is important in a variety of computer vision tasks.

Style Transfer Video Recognition

Fast Differentiable Matrix Square Root

1 code implementation ICLR 2022 Yue Song, Nicu Sebe, Wei Wang

Previous methods either adopt the Singular Value Decomposition (SVD) to explicitly factorize the matrix or use the Newton-Schulz iteration (NS iteration) to derive the approximate solution.

Geometry-Contrastive Transformer for Generalized 3D Pose Transfer

1 code implementation14 Dec 2021 Haoyu Chen, Hao Tang, Zitong Yu, Nicu Sebe, Guoying Zhao

Specifically, we propose a novel geometry-contrastive Transformer that has an efficient 3D structured perceiving ability to the global geometric inconsistencies across the given meshes.

Pose Transfer

Novel Class Discovery in Semantic Segmentation

1 code implementation CVPR 2022 Yuyang Zhao, Zhun Zhong, Nicu Sebe, Gim Hee Lee

We introduce a new setting of Novel Class Discovery in Semantic Segmentation (NCDSS), which aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes.

Image Classification Novel Class Discovery +3

3D-Aware Semantic-Guided Generative Model for Human Synthesis

1 code implementation2 Dec 2021 Jichao Zhang, Enver Sangineto, Hao Tang, Aliaksandr Siarohin, Zhun Zhong, Nicu Sebe, Wei Wang

However, they usually struggle to generate high-quality images representing non-rigid objects, such as the human body, which is of a great interest for many computer graphics applications.

3D-Aware Image Synthesis

Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Model

1 code implementation CVPR 2022 Zipeng Xu, Tianwei Lin, Hao Tang, Fu Li, Dongliang He, Nicu Sebe, Radu Timofte, Luc van Gool, Errui Ding

We propose a novel framework, i. e., Predict, Prevent, and Evaluate (PPE), for disentangled text-driven image manipulation that requires little manual annotation while being applicable to a wide variety of manipulations.

Image Manipulation Language Modelling

Bi-Mix: Bidirectional Mixing for Domain Adaptive Nighttime Semantic Segmentation

1 code implementation19 Nov 2021 Guanglei Yang, Zhun Zhong, Hao Tang, Mingli Ding, Nicu Sebe, Elisa Ricci

Specifically, in the image translation stage, Bi-Mix leverages the knowledge of day-night image pairs to improve the quality of nighttime image relighting.

Autonomous Driving Image Relighting +3

AniFormer: Data-driven 3D Animation with Transformer

1 code implementation20 Oct 2021 Haoyu Chen, Hao Tang, Nicu Sebe, Guoying Zhao

Instead, we introduce AniFormer, a novel Transformer-based architecture, that generates animated 3D sequences by directly taking the raw driving sequences and arbitrary same-type target meshes as inputs.

regression

Cascaded Cross MLP-Mixer GANs for Cross-View Image Translation

1 code implementation19 Oct 2021 Bin Ren, Hao Tang, Nicu Sebe

To ease this problem, we propose a novel two-stage framework with a new Cascaded Cross MLP-Mixer (CrossMLP) sub-network in the first stage and one refined pixel-level loss in the second stage.

Translation

ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation

1 code implementation26 Sep 2021 Yahui Liu, Yajing Chen, Linchao Bao, Nicu Sebe, Bruno Lepri, Marco De Nadai

The ISF manipulates the semantics of an input latent code to make the image generated from it lying in the desired visual domain.

Image-to-Image Translation Translation

Layout-to-Image Translation with Double Pooling Generative Adversarial Networks

1 code implementation29 Aug 2021 Hao Tang, Nicu Sebe

In this paper, we address the task of layout-to-image translation, which aims to translate an input semantic layout to a realistic image.

Translation

Total Generate: Cycle in Cycle Generative Adversarial Networks for Generating Human Faces, Hands, Bodies, and Natural Scenes

1 code implementation21 Jun 2021 Hao Tang, Nicu Sebe

Both generators are mutually connected and trained in an end-to-end fashion and explicitly form three cycled subnets, i. e., one image generation cycle and two guidance generation cycles.

Generative Adversarial Network Image-to-Image Translation +1

Neighborhood Contrastive Learning for Novel Class Discovery

1 code implementation CVPR 2021 Zhun Zhong, Enrico Fini, Subhankar Roy, Zhiming Luo, Elisa Ricci, Nicu Sebe

In this paper, we address Novel Class Discovery (NCD), the task of unveiling new classes in a set of unlabeled samples given a labeled dataset with known classes.

Clustering Contrastive Learning +1

Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation

no code implementations CVPR 2021 Yahui Liu, Enver Sangineto, Yajing Chen, Linchao Bao, Haoxian Zhang, Nicu Sebe, Bruno Lepri, Wei Wang, Marco De Nadai

In this paper, we propose a new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space in which: 1) Both intra- and inter-domain interpolations correspond to gradual changes in the generated images and 2) The content of the source image is better preserved during the translation.

Translation Unsupervised Image-To-Image Translation

Efficient Training of Visual Transformers with Small Datasets

1 code implementation NeurIPS 2021 Yahui Liu, Enver Sangineto, Wei Bi, Nicu Sebe, Bruno Lepri, Marco De Nadai

This task encourages the VTs to learn spatial relations within an image and makes the VT training much more robust when training data are scarce.

Inductive Bias

Source-Free Open Compound Domain Adaptation in Semantic Segmentation

1 code implementation7 Jun 2021 Yuyang Zhao, Zhun Zhong, Zhiming Luo, Gim Hee Lee, Nicu Sebe

Second, CPSS can reduce the influence of noisy pseudo-labels and also avoid the model overfitting to the target domain during self-supervised learning, consistently boosting the performance on the target and open domains.

Domain Generalization Self-Supervised Learning +1

Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization

1 code implementation31 May 2021 Jichao Zhang, Aliaksandr Siarohin, Hao Tang, Enver Sangineto, Wei Wang, Humphrey Sh, Nicu Sebe

Moreover, we propose a novel Self-Training Part Replacement (STPR) strategy to refine the model for the texture-transfer task, which improves the quality of the generated clothes and the preservation ability of non-target regions.

Image-to-Image Translation Pose Transfer +1

Transformer-Based Source-Free Domain Adaptation

1 code implementation28 May 2021 Guanglei Yang, Hao Tang, Zhun Zhong, Mingli Ding, Ling Shao, Nicu Sebe, Elisa Ricci

In this paper, we study the task of source-free domain adaptation (SFDA), where the source data are not available during target adaptation.

Knowledge Distillation Source-Free Domain Adaptation

Cloth Interactive Transformer for Virtual Try-On

1 code implementation12 Apr 2021 Bin Ren, Hao Tang, Fanyang Meng, Runwei Ding, Philip H. S. Torr, Nicu Sebe

In the second stage, we put forth a CIT reasoning block for establishing global mutual interactive dependencies among person representation, the warped clothing item, and the corresponding warped cloth mask.

Virtual Try-on

Curriculum Graph Co-Teaching for Multi-Target Domain Adaptation

1 code implementation CVPR 2021 Subhankar Roy, Evgeny Krivosheev, Zhun Zhong, Nicu Sebe, Elisa Ricci

In this paper we address multi-target domain adaptation (MTDA), where given one labeled source dataset and multiple unlabeled target datasets that differ in data distributions, the task is to learn a robust predictor for all the target domains.

Blended-target Domain Adaptation Multi-target Domain Adaptation

Transformer-Based Attention Networks for Continuous Pixel-Wise Prediction

1 code implementation ICCV 2021 Guanglei Yang, Hao Tang, Mingli Ding, Nicu Sebe, Elisa Ricci

While convolutional neural networks have shown a tremendous impact on various computer vision tasks, they generally demonstrate limitations in explicitly modeling long-range dependencies due to the intrinsic locality of the convolution operation.

Depth Estimation Depth Prediction +1

Viewpoint and Scale Consistency Reinforcement for UAV Vehicle Re-Identification

1 code implementation IJCV 2021 Shangzhi Teng, Shiliang Zhang, Qingming Huang, Nicu Sebe

Moreover, our method also achieves competitive performance compared with recent works on existing vehicle ReID datasets including VehicleID, VeRi-776 and VERI-Wild.

Vehicle Re-Identification

Curriculum Learning: A Survey

no code implementations25 Jan 2021 Petru Soviany, Radu Tudor Ionescu, Paolo Rota, Nicu Sebe

Training machine learning models in a meaningful order, from the easy samples to the hard ones, using curriculum learning can provide performance improvements over the standard training approach based on random data shuffling, without any additional computational costs.

BIG-bench Machine Learning Clustering

Probabilistic Graph Attention Network with Conditional Kernels for Pixel-Wise Prediction

no code implementations8 Jan 2021 Dan Xu, Xavier Alameda-Pineda, Wanli Ouyang, Elisa Ricci, Xiaogang Wang, Nicu Sebe

In contrast to previous works directly considering multi-scale feature maps obtained from the inner layers of a primary CNN architecture, and simply fusing the features with weighted averaging or concatenation, we propose a probabilistic graph attention network structure based on a novel Attention-Gated Conditional Random Fields (AG-CRFs) model for learning and fusing multi-scale representations in a principled manner.

Graph Attention Monocular Depth Estimation +1

Exploiting Sample Correlation for Crowd Counting With Multi-Expert Network

no code implementations ICCV 2021 Xinyan Liu, Guorong Li, Zhenjun Han, Weigang Zhang, Yifan Yang, Qingming Huang, Nicu Sebe

Specifically, we propose a task-driven similarity metric based on sample's mutual enhancement, referred as co-fine-tune similarity, which can find a more efficient subset of data for training the expert network.

Crowd Counting

Semantically-Adaptive Upsampling for Layout-to-Image Translation

no code implementations1 Jan 2021 Hao Tang, Nicu Sebe

We propose the Semantically-Adaptive UpSampling (SA-UpSample), a general and highly effective upsampling method for the layout-to-image translation task.

Translation

CNNs for JPEGs: A Study in Computational Cost

no code implementations26 Dec 2020 Samuel Felipe dos Santos, Nicu Sebe, Jurandy Almeida

In this paper, we propose a further study of the computational cost of deep models designed for the frequency domain, evaluating the cost of decoding and passing the images through the network.

Edge-computing Image Classification +1

Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification

1 code implementation CVPR 2021 Yuyang Zhao, Zhun Zhong, Fengxiang Yang, Zhiming Luo, Yaojin Lin, Shaozi Li, Nicu Sebe

In this paper, we study the problem of multi-source domain generalization in ReID, which aims to learn a model that can perform well on unseen domains with only several labeled source domains.

Domain Generalization Meta-Learning +1

Deep traffic light detection by overlaying synthetic context on arbitrary natural images

1 code implementation7 Nov 2020 Jean Pablo Vieira de Mello, Lucas Tabelini, Rodrigo F. Berriel, Thiago M. Paixão, Alberto F. de Souza, Claudine Badue, Nicu Sebe, Thiago Oliveira-Santos

By providing real image samples with traffic context to the network, the model learns to detect and classify elements of interest, such as pedestrians, traffic signs, and traffic lights.

Autonomous Driving

SF-UDA$^{3D}$: Source-Free Unsupervised Domain Adaptation for LiDAR-Based 3D Object Detection

1 code implementation16 Oct 2020 Cristiano Saltori, Stéphane Lathuiliére, Nicu Sebe, Elisa Ricci, Fabio Galasso

In the case of LiDAR, in fact, domain shift is not only due to changes in the environment and in the object appearances, as for visual data from RGB cameras, but is also related to the geometry of the point clouds (e. g., point density variations).

3D Object Detection Object +2

Dual Attention GANs for Semantic Image Synthesis

1 code implementation29 Aug 2020 Hao Tang, Song Bai, Nicu Sebe

We also propose two novel modules, i. e., position-wise Spatial Attention Module (SAM) and scale-wise Channel Attention Module (CAM), to capture semantic structure attention in spatial and channel dimensions, respectively.

Image Generation Position

Bipartite Graph Reasoning GANs for Person Image Generation

1 code implementation10 Aug 2020 Hao Tang, Song Bai, Philip H. S. Torr, Nicu Sebe

We present a novel Bipartite Graph Reasoning GAN (BiGraphGAN) for the challenging person image generation task.

 Ranked #1 on Pose Transfer on Market-1501 (PCKh metric)

Pose Transfer

Describe What to Change: A Text-guided Unsupervised Image-to-Image Translation Approach

1 code implementation10 Aug 2020 Yahui Liu, Marco De Nadai, Deng Cai, Huayang Li, Xavier Alameda-Pineda, Nicu Sebe, Bruno Lepri

Our proposed model disentangles the image content from the visual attributes, and it learns to modify the latter using the textual description, before generating a new image from the content and the modified attribute representation.

Attribute Image Captioning +3

Dual In-painting Model for Unsupervised Gaze Correction and Animation in the Wild

1 code implementation9 Aug 2020 Jichao Zhang, Jingjing Chen, Hao Tang, Wei Wang, Yan Yan, Enver Sangineto, Nicu Sebe

In this paper we address the problem of unsupervised gaze correction in the wild, presenting a solution that works without the need for precise annotations of the gaze angle and the head pose.

XingGAN for Person Image Generation

2 code implementations ECCV 2020 Hao Tang, Song Bai, Li Zhang, Philip H. S. Torr, Nicu Sebe

We propose a novel Generative Adversarial Network (XingGAN or CrossingGAN) for person image generation tasks, i. e., translating the pose of a given person to a desired one.

 Ranked #1 on Pose Transfer on Market-1501 (IS metric)

Generative Adversarial Network Pose Transfer

Whitening for Self-Supervised Representation Learning

8 code implementations13 Jul 2020 Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, Nicu Sebe

Most of the current self-supervised representation learning (SSL) methods are based on the contrastive loss and the instance-discrimination task, where augmented versions of the same image instance ("positives") are contrasted with instances extracted from other images ("negatives").

Representation Learning Self-Supervised Learning

Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events

no code implementations9 May 2020 Weiyao Lin, Huabin Liu, Shizhan Liu, Yuxi Li, Rui Qian, Tao Wang, Ning Xu, Hongkai Xiong, Guo-Jun Qi, Nicu Sebe

To this end, we present a new large-scale dataset with comprehensive annotations, named Human-in-Events or HiEve (Human-centric video analysis in complex Events), for the understanding of human motions, poses, and actions in a variety of realistic events, especially in crowd & complex events.

Action Recognition Pose Estimation

Speak2Label: Using Domain Knowledge for Creating a Large Scale Driver Gaze Zone Estimation Dataset

no code implementations13 Apr 2020 Shreya Ghosh, Abhinav Dhall, Garima Sharma, Sarthak Gupta, Nicu Sebe

In this paper, a fully automatic technique for labelling an image based gaze behavior dataset for driver gaze zone estimation is proposed.

Gaze Prediction

Coarse-to-Fine Gaze Redirection with Numerical and Pictorial Guidance

1 code implementation7 Apr 2020 Jingjing Chen, Jichao Zhang, Enver Sangineto, Jiayuan Fan, Tao Chen, Nicu Sebe

In this paper, we propose to alleviate these problems by means of a novel gaze redirection framework which exploits both a numerical and a pictorial direction guidance, jointly with a coarse-to-fine learning strategy.

gaze redirection Image Generation

Binary Neural Networks: A Survey

2 code implementations31 Mar 2020 Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, Nicu Sebe

The binary neural network, largely saving the storage and computation, serves as a promising technique for deploying deep models on resource-limited devices.

Binarization Image Classification +4

GMM-UNIT: Unsupervised Multi-Domain and Multi-Modal Image-to-Image Translation via Attribute Gaussian Mixture Modeling

1 code implementation15 Mar 2020 Yahui Liu, Marco De Nadai, Jian Yao, Nicu Sebe, Bruno Lepri, Xavier Alameda-Pineda

Unsupervised image-to-image translation (UNIT) aims at learning a mapping between several visual domains by using unpaired training images.

Attribute Translation +1

Multi-Channel Attention Selection GANs for Guided Image-to-Image Translation

1 code implementation3 Feb 2020 Hao Tang, Philip H. S. Torr, Nicu Sebe

In the first stage, the input image and the conditional semantic guidance are fed into a cycled semantic-guided generation network to produce initial coarse results.

Generative Adversarial Network Image-to-Image Translation +1

Non-linear Neurons with Human-like Apical Dendrite Activations

1 code implementation2 Feb 2020 Mariana-Iuliana Georgescu, Radu Tudor Ionescu, Nicolae-Catalin Ristea, Nicu Sebe

In order to classify linearly non-separable data, neurons are typically organized into multi-layer neural networks that are equipped with at least one hidden layer.

Speech Emotion Recognition

Low-Budget Label Query through Domain Alignment Enforcement

no code implementations1 Jan 2020 Jurandy Almeida, Cristiano Saltori, Paolo Rota, Nicu Sebe

Deep learning revolution happened thanks to the availability of a massive amount of labelled data which have contributed to the development of models with extraordinary inference capabilities.

Unsupervised Domain Adaptation

Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation

2 code implementations CVPR 2020 Hao Tang, Dan Xu, Yan Yan, Philip H. S. Torr, Nicu Sebe

To tackle this issue, in this work we consider learning the scene generation in a local context, and correspondingly design a local class-specific generative network with semantic maps as a guidance, which separately constructs and learns sub-generators concentrating on the generation of different classes, and is able to provide more scene details.

Image Generation Scene Generation

Asymmetric Generative Adversarial Networks for Image-to-Image Translation

1 code implementation14 Dec 2019 Hao Tang, Dan Xu, Hong Liu, Nicu Sebe

In this paper, we analyze the limitation of the existing symmetric GAN models in asymmetric translation tasks, and propose an AsymmetricGAN model with both translation and reconstruction generators of unequal sizes and different parameter-sharing strategy to adapt to the asymmetric need in both unsupervised and supervised image-to-image translation tasks.

Image-to-Image Translation Translation

Unified Generative Adversarial Networks for Controllable Image-to-Image Translation

1 code implementation12 Dec 2019 Hao Tang, Hong Liu, Nicu Sebe

The proposed model consists of a single generator and a discriminator taking a conditional image and the target controllable structure as input.

Facial Expression Translation Generative Adversarial Network +3

AttentionGAN: Unpaired Image-to-Image Translation using Attention-Guided Generative Adversarial Networks

2 code implementations27 Nov 2019 Hao Tang, Hong Liu, Dan Xu, Philip H. S. Torr, Nicu Sebe

State-of-the-art methods in image-to-image translation are capable of learning a mapping from a source domain to a target domain with unpaired image data.

Image-to-Image Translation Translation

Curriculum Self-Paced Learning for Cross-Domain Object Detection

no code implementations15 Nov 2019 Petru Soviany, Radu Tudor Ionescu, Paolo Rota, Nicu Sebe

To alleviate this problem, researchers proposed various domain adaptation methods to improve object detection results in the cross-domain setting, e. g. by translating images with ground-truth labels from the source domain to the target domain using Cycle-GAN.

Domain Adaptation Object +2

Event Discovery for History Representation in Reinforcement Learning

no code implementations25 Sep 2019 Aleksandr Ermolov, Enver Sangineto, Nicu Sebe

To address this problem, a possible solution is to provide the agent with information about past observations.

reinforcement-learning Reinforcement Learning (RL)

Progressive Fusion for Unsupervised Binocular Depth Estimation using Cycled Networks

1 code implementation17 Sep 2019 Andrea Pilzer, Stéphane Lathuilière, Dan Xu, Mihai Marian Puscas, Elisa Ricci, Nicu Sebe

Extensive experiments on the publicly available datasets KITTI, Cityscapes and ApolloScape demonstrate the effectiveness of the proposed model which is competitive with other unsupervised deep learning methods for depth prediction.

Data Augmentation Depth Prediction +2

Structured Coupled Generative Adversarial Networks for Unsupervised Monocular Depth Estimation

no code implementations15 Aug 2019 Mihai Marian Puscas, Dan Xu, Andrea Pilzer, Nicu Sebe

Inspired by the success of adversarial learning, we propose a new end-to-end unsupervised deep learning framework for monocular depth estimation consisting of two Generative Adversarial Networks (GAN), deeply coupled with a structured Conditional Random Field (CRF) model.

Monocular Depth Estimation Unsupervised Monocular Depth Estimation

Cycle In Cycle Generative Adversarial Networks for Keypoint-Guided Image Generation

1 code implementation2 Aug 2019 Hao Tang, Dan Xu, Gaowen Liu, Wei Wang, Nicu Sebe, Yan Yan

In this work, we propose a novel Cycle In Cycle Generative Adversarial Network (C$^2$GAN) for the task of keypoint-guided image generation.

Generative Adversarial Network Image Generation

Cross-Domain Car Detection Using Unsupervised Image-to-Image Translation: From Day to Night

1 code implementation19 Jul 2019 Vinicius F. Arruda, Thiago M. Paixão, Rodrigo F. Berriel, Alberto F. De Souza, Claudine Badue, Nicu Sebe, Thiago Oliveira-Santos

In this work, a method for training a car detection system with annotated data from a source domain (day images) without requiring the image annotations of the target domain (night images) is presented.

Autonomous Vehicles object-detection +3

Gesture-to-Gesture Translation in the Wild via Category-Independent Conditional Maps

1 code implementation12 Jul 2019 Yahui Liu, Marco De Nadai, Gloria Zen, Nicu Sebe, Bruno Lepri

In this work, we propose a novel GAN architecture that decouples the required annotations into a category label - that specifies the gesture type - and a simple-to-draw category-independent conditional map - that expresses the location, rotation and size of the hand gesture.

Gesture-to-Gesture Translation Translation

GazeCorrection:Self-Guided Eye Manipulation in the wild using Self-Supervised Generative Adversarial Networks

no code implementations arXiv 2019 Jichao Zhang, Meng Sun, Jingjing Chen, Hao Tang, Yan Yan, Xueying Qin, Nicu Sebe

Gaze correction aims to redirect the person's gaze into the camera by manipulating the eye region, and it can be considered as a specific image resynthesis problem.

Resynthesis

Impact of facial landmark localization on facial expression recognition

no code implementations26 May 2019 Romain Belmonte, Benjamin Allaert, Pierre Tirilly, Ioan Marius Bilasco, Chaabane Djeraba, Nicu Sebe

Although facial landmark localization (FLL) approaches are becoming increasingly accurate for characterizing facial regions, one question remains unanswered: what is the impact of these approaches on subsequent related tasks?

Face Alignment Facial Expression Recognition +1

Regularized Evolutionary Algorithm for Dynamic Neural Topology Search

no code implementations15 May 2019 Cristiano Saltori, Subhankar Roy, Nicu Sebe, Giovanni Iacca

Although very effective, evolutionary algorithms rely heavily on having a large population of individuals (i. e., network architectures) and is therefore memory expensive.

Evolutionary Algorithms Neural Architecture Search +1

Budget-Aware Adapters for Multi-Domain Learning

no code implementations ICCV 2019 Rodrigo Berriel, Stéphane Lathuilière, Moin Nabi, Tassilo Klein, Thiago Oliveira-Santos, Nicu Sebe, Elisa Ricci

To implement this idea we derive specialized deep models for each domain by adapting a pre-trained architecture but, differently from other methods, we propose a novel strategy to automatically adjust the computational complexity of the network.

Expression Conditional GAN for Facial Expression-to-Expression Translation

no code implementations14 May 2019 Hao Tang, Wei Wang, Songsong Wu, Xinya Chen, Dan Xu, Nicu Sebe, Yan Yan

In this paper, we focus on the facial expression translation task and propose a novel Expression Conditional GAN (ECGAN) which can learn the mapping from one image domain to another one based on an additional expression attribute.

Attribute Facial expression generation +2

Attention-based Fusion for Multi-source Human Image Generation

no code implementations7 May 2019 Stéphane Lathuilière, Enver Sangineto, Aliaksandr Siarohin, Nicu Sebe

We present a generalization of the person-image generation task, in which a human image is generated conditioned on a target pose and a set X of source appearance images.

Image Generation

Whitening and Coloring transform for GANs

no code implementations ICLR 2019 Aliaksandr Siarohin, Enver Sangineto, Nicu Sebe

In this paper we propose to generalize both BN and cBN using a Whitening and Coloring based batch normalization.

Appearance and Pose-Conditioned Human Image Generation using Deformable GANs

1 code implementation30 Apr 2019 Aliaksandr Siarohin, Stéphane Lathuilière, Enver Sangineto, Nicu Sebe

Specifically, given an image xa of a person and a target pose P(xb), extracted from a different image xb, we synthesize a new image of that person in pose P(xb), while preserving the visual details in xa.

Data Augmentation Generative Adversarial Network +2

Online Adaptation through Meta-Learning for Stereo Depth Estimation

no code implementations17 Apr 2019 Zhen-Yu Zhang, Stéphane Lathuilière, Andrea Pilzer, Nicu Sebe, Elisa Ricci, Jian Yang

Our proposal is evaluated on the wellestablished KITTI dataset, where we show that our online method is competitive withstate of the art algorithms trained in a batch setting.

Meta-Learning Stereo Depth Estimation

Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation

3 code implementations CVPR 2019 Hao Tang, Dan Xu, Nicu Sebe, Yanzhi Wang, Jason J. Corso, Yan Yan

In this paper, we propose a novel approach named Multi-Channel Attention SelectionGAN (SelectionGAN) that makes it possible to generate images of natural scenes in arbitrary viewpoints, based on an image of the scene and a novel semantic map.

Bird View Synthesis Cross-View Image-to-Image Translation +1

Metric-Learning based Deep Hashing Network for Content Based Retrieval of Remote Sensing Images

1 code implementation2 Apr 2019 Subhankar Roy, Enver Sangineto, Begüm Demir, Nicu Sebe

Hashing methods have been recently found very effective in retrieval of remote sensing (RS) images due to their computational efficiency and fast search speed.

Computational Efficiency Deep Hashing +1

Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation

8 code implementations28 Mar 2019 Hao Tang, Dan Xu, Nicu Sebe, Yan Yan

To handle the limitation, in this paper we propose a novel Attention-Guided Generative Adversarial Network (AGGAN), which can detect the most discriminative semantic object and minimize changes of unwanted part for semantic manipulation problems without using extra data and models.

Generative Adversarial Network Translation +1

Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimation

no code implementations CVPR 2019 Andrea Pilzer, Stéphane Lathuilière, Nicu Sebe, Elisa Ricci

Therefore, recent works have proposed deep architectures for addressing the monocular depth prediction task as a reconstruction problem, thus avoiding the need of collecting ground-truth depth.

Depth Prediction Knowledge Distillation +2

Attribute-Guided Sketch Generation

1 code implementation28 Jan 2019 Hao Tang, Xinya Chen, Wei Wang, Dan Xu, Jason J. Corso, Nicu Sebe, Yan Yan

To this end, we propose a novel Attribute-Guided Sketch Generative Adversarial Network (ASGAN) which is an end-to-end framework and contains two pairs of generators and discriminators, one of which is used to generate faces with attributes while the other one is employed for image-to-sketch translation.

Attribute Generative Adversarial Network +1

Fast and Robust Dynamic Hand Gesture Recognition via Key Frames Extraction and Feature Fusion

1 code implementation15 Jan 2019 Hao Tang, Hong Liu, Wei Xiao, Nicu Sebe

Gesture recognition is a hot topic in computer vision and pattern recognition, which plays a vitally important role in natural human-computer interface.

Clustering Hand Gesture Recognition +1

Dual Generator Generative Adversarial Networks for Multi-Domain Image-to-Image Translation

1 code implementation14 Jan 2019 Hao Tang, Dan Xu, Wei Wang, Yan Yan, Nicu Sebe

State-of-the-art methods for image-to-image translation with Generative Adversarial Networks (GANs) can learn a mapping from one domain to another domain using unpaired image data.

Generative Adversarial Network Image-to-Image Translation +1

Low-Shot Learning from Imaginary 3D Model

no code implementations4 Jan 2019 Frederik Pahde, Mihai Puscas, Jannik Wolff, Tassilo Klein, Nicu Sebe, Moin Nabi

Since the advent of deep learning, neural networks have demonstrated remarkable results in many visual recognition tasks, constantly pushing the limits.

Few-Shot Learning

Predicting Group Cohesiveness in Images

no code implementations31 Dec 2018 Shreya Ghosh, Abhinav Dhall, Nicu Sebe, Tom Gedeon

We study the factors that influence the perception of group-level cohesion and propose methods for estimating the human-perceived cohesion on the group cohesiveness scale.

Attribute

Enhancing Perceptual Attributes with Bayesian Style Generation

1 code implementation3 Dec 2018 Aliaksandr Siarohin, Gloria Zen, Nicu Sebe, Elisa Ricci

Our approach takes as input a natural image and exploits recent models for deep style transfer and generative adversarial networks to change its style in order to modify a specific high-level attribute.

Attribute Style Transfer

Deep Micro-Dictionary Learning and Coding Network

1 code implementation11 Sep 2018 Hao Tang, Heng Wei, Wei Xiao, Wei Wang, Dan Xu, Yan Yan, Nicu Sebe

In this paper, we propose a novel Deep Micro-Dictionary Learning and Coding Network (DDLCN).

Dictionary Learning

GestureGAN for Hand Gesture-to-Gesture Translation in the Wild

1 code implementation14 Aug 2018 Hao Tang, Wei Wang, Dan Xu, Yan Yan, Nicu Sebe

Therefore, this task requires a high-level understanding of the mapping between the input source gesture and the output target gesture.

Data Augmentation Generative Adversarial Network +2

Unsupervised Adversarial Depth Estimation using Cycled Generative Networks

2 code implementations28 Jul 2018 Andrea Pilzer, Dan Xu, Mihai Marian Puscas, Elisa Ricci, Nicu Sebe

The proposed architecture consists of two generative sub-networks jointly trained with adversarial learning for reconstructing the disparity map and organized in a cycle such as to provide mutual constraints and supervision to each other.

Monocular Depth Estimation

Group Consistent Similarity Learning via Deep CRF for Person Re-Identification

no code implementations CVPR 2018 Dapeng Chen, Dan Xu, Hongsheng Li, Nicu Sebe, Xiaogang Wang

Extensive experiments demonstrate the effectiveness of our model that combines DNN and CRF for learning robust multi-scale local similarities.

Person Re-Identification

Whitening and Coloring batch transform for GANs

1 code implementation ICLR 2019 Aliaksandr Siarohin, Enver Sangineto, Nicu Sebe

In this paper we propose to generalize both BN and cBN using a Whitening and Coloring based batch normalization.

Image Generation

Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation

1 code implementation CVPR 2018 Dan Xu, Wei Wang, Hao Tang, Hong Liu, Nicu Sebe, Elisa Ricci

Recent works have shown the benefit of integrating Conditional Random Fields (CRFs) models into deep architectures for improving pixel-level prediction tasks.

Monocular Depth Estimation

Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction

no code implementations NeurIPS 2017 Dan Xu, Wanli Ouyang, Xavier Alameda-Pineda, Elisa Ricci, Xiaogang Wang, Nicu Sebe

Recent works have shown that exploiting multi-scale representations deeply learned via convolutional neural networks (CNN) is of tremendous importance for accurate contour detection.

Contour Detection

Depression Severity Estimation from Multiple Modalities

no code implementations10 Nov 2017 Evgeny Stepanov, Stephane Lathuiliere, Shammur Absar Chowdhury, Arindam Ghosh, Radu-Laurentiu Vieriu, Nicu Sebe, Giuseppe Riccardi

In this AVEC challenge we explore different modalities (speech, language and visual features extracted from face) to design and develop automatic methods for the detection of depression.

What your Facebook Profile Picture Reveals about your Personality

no code implementations3 Aug 2017 Cristina Segalin, Fabio Celli, Luca Polonio, Michal Kosinski, David Stillwell, Nicu Sebe, Marco Cristani, Bruno Lepri

We analyze the effectiveness of four families of visual features and we discuss some human interpretable patterns that explain the personality traits of the individuals.

General Classification

Spatio-Temporal Vector of Locally Max Pooled Features for Action Recognition in Videos

no code implementations CVPR 2017 Ionut Cosmin Duta, Bogdan Ionescu, Kiyoharu Aizawa, Nicu Sebe

The proposed method addresses an important problem of video understanding: how to build a video representation that incorporates the CNN features over the entire video.

Action Recognition In Videos Temporal Action Localization +1

Learning Cross-Modal Deep Representations for Robust Pedestrian Detection

2 code implementations CVPR 2017 Dan Xu, Wanli Ouyang, Elisa Ricci, Xiaogang Wang, Nicu Sebe

Then, the learned feature representations are transferred to a second deep network, which receives as input an RGB image and outputs the detection results.

Pedestrian Detection

How to Make an Image More Memorable? A Deep Style Transfer Approach

1 code implementation6 Apr 2017 Aliaksandr Siarohin, Gloria Zen, Cveta Majtanovic, Xavier Alameda-Pineda, Elisa Ricci, Nicu Sebe

In this work, we show that it is possible to automatically retrieve the best style seeds for a given image, thus remarkably reducing the number of human attempts needed to find a good match.

Image Generation Style Transfer

Viraliency: Pooling Local Virality

1 code implementation CVPR 2017 Xavier Alameda-Pineda, Andrea Pilzer, Dan Xu, Nicu Sebe, Elisa Ricci

In our overly-connected world, the automatic recognition of virality - the quality of an image or video to be rapidly and widely spread in social networks - is of crucial importance, and has recently awaken the interest of the computer vision community.

Plug-and-Play CNN for Crowd Motion Analysis: An Application in Abnormal Event Detection

no code implementations2 Oct 2016 Mahdyar Ravanbakhsh, Moin Nabi, Hossein Mousavi, Enver Sangineto, Nicu Sebe

In this paper, we show that keeping track of the changes in the CNN feature across time can facilitate capturing the local abnormality.

Anomaly Detection Event Detection +1

Are Safer Looking Neighborhoods More Lively? A Multimodal Investigation into Urban Life

1 code implementation1 Aug 2016 Marco De Nadai, Radu L. Vieriu, Gloria Zen, Stefan Dragicevic, Nikhil Naik, Michele Caraviello, Cesar A. Hidalgo, Nicu Sebe, Bruno Lepri

But in a world where the preference for safe looking neighborhoods is small, the connection between the perception of safety and liveliness will be either weak or nonexistent.

Computers and Society Social and Information Networks Physics and Society

A Survey on Learning to Hash

no code implementations1 Jun 2016 Jingdong Wang, Ting Zhang, Jingkuan Song, Nicu Sebe, Heng Tao Shen

In this paper, we present a comprehensive survey of the learning to hash algorithms, categorize them according to the manners of preserving the similarities into: pairwise similarity preserving, multiwise similarity preserving, implicit similarity preserving, as well as quantization, and discuss their relations.

Quantization

Self-Adaptive Matrix Completion for Heart Rate Estimation From Face Videos Under Realistic Conditions

no code implementations CVPR 2016 Sergey Tulyakov, Xavier Alameda-Pineda, Elisa Ricci, Lijun Yin, Jeffrey F. Cohn, Nicu Sebe

Recent studies in computer vision have shown that, while practically invisible to a human observer, skin color changes due to blood flow can be captured on face videos and, surprisingly, be used to estimate the heart rate (HR).

Heart rate estimation Matrix Completion

Cannot find the paper you are looking for? You can Submit a new open access paper.