no code implementations • ECCV 2020 • Yifan Yang, Guorong Li, Zhe Wu, Li Su, Qingming Huang, Nicu Sebe
We propose a soft-label sorting network along with the counting network, which sorts the given images by their crowd numbers.
no code implementations • 5 Dec 2023 • Weijie Wang, Guofeng Mei, Bin Ren, Xiaoshui Huang, Fabio Poiesi, Luc van Gool, Nicu Sebe, Bruno Lepri
The cornerstone of ZeroReg is the novel transfer of image features from keypoints to the point cloud, enriched by aggregating information from 3D geometric neighborhoods.
1 code implementation • 5 Dec 2023 • Victor G. Turrisi da Costa, Nicola Dall'Asen, Yiming Wang, Nicu Sebe, Elisa Ricci
Few-shot image classification aims to learn an image classifier using only a small set of labeled examples per class.
1 code implementation • 23 Nov 2023 • Yue Song, Nicu Sebe, Wei Wang
This observation motivates us to propose \texttt{RankFeat}, a simple yet effective \emph{post hoc} approach for OOD detection by removing the rank-1 matrix composed of the largest singular value and the associated singular vectors from the high-level feature.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
no code implementations • 20 Nov 2023 • Wenhao Li, Mengyuan Liu, Hong Liu, Pichao Wang, Jialun Cai, Nicu Sebe
Our HoT begins with pruning pose tokens of redundant frames and ends with recovering full-length tokens, resulting in a few pose tokens in the intermediate transformer blocks and thus improving the model efficiency.
1 code implementation • 2 Nov 2023 • Moreno D'Incà, Christos Tzelepis, Ioannis Patras, Nicu Sebe
These paths are then applied to augment images to improve the fairness of a given dataset.
no code implementations • 20 Sep 2023 • Samuel Felipe dos Santos, Nicu Sebe, Jurandy Almeida
In this paper, we propose a further study of the computational cost of deep models designed for the frequency domain, evaluating the cost of decoding and passing the images through the network.
no code implementations • 20 Sep 2023 • Samuel Felipe dos Santos, Rodrigo Berriel, Thiago Oliveira-Santos, Nicu Sebe, Jurandy Almeida
Nevertheless, the models are usually larger than the baseline for a single domain.
no code implementations • 16 Sep 2023 • Lucas Fernando Alvarenga e Silva, Nicu Sebe, Jurandy Almeida
Convolutional Neural Networks (CNNs) have brought revolutionary advances to many research areas due to their capacity of learning from raw data.
no code implementations • 3 Sep 2023 • Weijie Wang, Zhengyu Zhao, Nicu Sebe, Bruno Lepri
Although effective deepfake detectors have been proposed, they are substantially vulnerable to adversarial attacks.
1 code implementation • 28 Aug 2023 • Cristiano Saltori, Fabio Galasso, Giuseppe Fiameni, Nicu Sebe, Fabio Poiesi, Elisa Ricci
In this study, we introduce compositional semantic mixing for point cloud domain adaptation, representing the first unsupervised domain adaptation technique for point cloud segmentation based on semantic and geometric sample mixing.
no code implementations • 31 Jul 2023 • Elia Peruzzo, Willi Menapace, Vidit Goel, Federica Arrigoni, Hao Tang, Xingqian Xu, Arman Chopikyan, Nikita Orlov, Yuxiao Hu, Humphrey Shi, Nicu Sebe, Elisa Ricci
This paper advances the state of the art in this emerging research domain by proposing the first approach for Interactive NP.
1 code implementation • 22 Jul 2023 • Hao Tang, Guolei Sun, Nicu Sebe, Luc van Gool
To tackle 2), we design an effective module to selectively highlight class-dependent feature maps according to the original semantic layout to preserve the semantic information.
no code implementations • 18 Jul 2023 • Federico Betti, Jacopo Staiano, Lorenzo Baraldi, Rita Cucchiara, Nicu Sebe
Research in Image Generation has recently made significant progress, particularly boosted by the introduction of Vision-Language models which are able to produce high-quality visual content based on textual inputs.
1 code implementation • ICCV 2023 • Yue Song, Jichao Zhang, Nicu Sebe, Wei Wang
Generative Adversarial Networks (GANs), especially the recent style-based generators (StyleGANs), have versatile semantics in the structured latent space.
no code implementations • 28 May 2023 • Weizhi Nie, Chuanqi Jiao, Ruidong Chen, Weijie Wang, Bruno Lepri, Nicu Sebe, AnAn Liu
To retrieve similar shapes from the partial input, we also apply a contrastive learning-based pre-training scheme to transfer features of incomplete shapes into the domain of complete shape features.
no code implementations • 25 May 2023 • Weizhi Nie, Ruidong Chen, Weijie Wang, Bruno Lepri, Nicu Sebe
Meanwhile, to effectively integrate multi-modal prior knowledge into textual information, we adopt a novel multi-layer transformer structure to progressively fuse related shape and textual information, which can effectively compensate for the lack of structural information in the text and enhance the final performance of the 3D generation model.
no code implementations • 23 May 2023 • Nan Pu, Zhun Zhong, Xinyuan Ji, Nicu Sebe
On each client, GCL builds class-level contrastive learning with both local and global GMMs.
no code implementations • 18 May 2023 • Ziheng Chen, Yue Song, Gaowen Liu, Ramana Rao Kompella, XiaoJun Wu, Nicu Sebe
Moreover, we encompass the most popular classifier in existing SPD networks as a special case of our framework.
1 code implementation • 25 Apr 2023 • Yue Song, T. Anderson Keller, Nicu Sebe, Max Welling
In this work, we instead propose to model latent structures with a learned dynamic potential landscape, thereby performing latent traversals as the flow of samples down the landscape's gradient.
1 code implementation • 30 Mar 2023 • Vidit Goel, Elia Peruzzo, Yifan Jiang, Dejia Xu, Xingqian Xu, Nicu Sebe, Trevor Darrell, Zhangyang Wang, Humphrey Shi
We propose \textbf{PAIR} Diffusion, a generic framework that can enable a diffusion model to control the structure and appearance properties of each object in the image.
1 code implementation • CVPR 2023 • Nan Pu, Zhun Zhong, Nicu Sebe
This leads traditional novel category discovery (NCD) methods to be incapacitated for GCD, due to their assumption of unlabeled data are only from novel categories.
1 code implementation • 28 Mar 2023 • Mingxuan Liu, Subhankar Roy, Zhun Zhong, Nicu Sebe, Elisa Ricci
Discovering novel concepts from unlabelled data and in a continuous manner is an important desideratum of lifelong learners.
no code implementations • 26 Mar 2023 • Ziheng Chen, Yue Song, Tianyang Xu, Zhiwu Huang, Xiao-Jun Wu, Nicu Sebe
Symmetric Positive Definite (SPD) matrices have received wide attention in machine learning due to their intrinsic capacity of encoding underlying structural correlation in data.
1 code implementation • CVPR 2023 • Simone Barattin, Christos Tzelepis, Ioannis Patras, Nicu Sebe
By optimizing the latent codes directly, we ensure both that the identity is of a desired distance away from the original (with an identity obfuscation loss), whilst preserving the facial attributes (using a novel feature-matching loss in FaRL's deep feature space).
1 code implementation • 16 Mar 2023 • Zipeng Xu, Songlong Xing, Enver Sangineto, Nicu Sebe
However, directly using CLIP to guide style transfer leads to undesirable artifacts (mainly written words and unrelated visual entities) spread over the image.
1 code implementation • ICCV 2023 • Zipeng Xu, Enver Sangineto, Nicu Sebe
Despite the progress made in the style transfer task, most previous work focus on transferring only relatively simple features like color or texture, while missing more abstract concepts such as overall art expression or painter-specific traits.
no code implementations • CVPR 2023 • Hao Tang, Zhenyu Zhang, Humphrey Shi, Bo Li, Ling Shao, Nicu Sebe, Radu Timofte, Luc van Gool
We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for the challenging graph-constrained house generation task.
1 code implementation • 7 Mar 2023 • Juanjuan Weng, Zhiming Luo, Zhun Zhong, Shaozi Li, Nicu Sebe
In this work, we provide a comprehensive investigation of the CE loss function and find that the logit margin between the targeted and untargeted classes will quickly obtain saturation in CE, which largely limits the transferability.
no code implementations • 10 Jan 2023 • Mengyi Zhao, Mengyuan Liu, Bin Ren, Shuling Dai, Nicu Sebe
Diffusion-based generative models have recently emerged as powerful solutions for high-quality synthesis in multiple domains.
no code implementations • CVPR 2023 • Wei Wang, Zhun Zhong, Weijie Wang, Xi Chen, Charles Ling, Boyu Wang, Nicu Sebe
In this paper, we study the application of Test-time domain adaptation in semantic segmentation (TTDA-Seg) where both efficiency and effectiveness are crucial.
1 code implementation • 18 Dec 2022 • Yuyang Zhao, Zhun Zhong, Na Zhao, Nicu Sebe, Gim Hee Lee
Furthermore, we present a novel style hallucination module (SHM) to generate style-diversified samples that are essential to consistency learning.
1 code implementation • 11 Dec 2022 • Yue Song, Nicu Sebe, Wei Wang
Extensive experiments on visual recognition demonstrate that our methods can simultaneously improve covariance conditioning and generalization.
no code implementations • 8 Dec 2022 • Xinyan Liu, Guorong Li, Yuankai Qi, Zhenjun Han, Qingming Huang, Ming-Hsuan Yang, Nicu Sebe
Crowd localization aims to predict the spatial position of humans in a crowd scenario.
1 code implementation • 18 Nov 2022 • Daichi Horita, Jiaolong Yang, Dong Chen, Yuki Koyama, Kiyoharu Aizawa, Nicu Sebe
The structure generator generates an edge image representing plausible structures within the holes, which is then used for guiding the texture generation process.
1 code implementation • 12 Nov 2022 • Hao Tang, Ling Shao, Philip H. S. Torr, Nicu Sebe
To further capture the change in pose of each part more precisely, we propose a novel part-aware bipartite graph reasoning (PBGR) block to decompose the task of reasoning the global structure transformation with a bipartite graph into learning different local transformations for different semantic body/face parts.
no code implementations • 12 Nov 2022 • Hao Tang, Lei Ding, Songsong Wu, Bin Ren, Nicu Sebe, Paolo Rota
The proposed TSDPC is a generic and powerful framework and it has two advantages compared with previous works, one is that it can calculate the number of key frames automatically.
1 code implementation • 17 Oct 2022 • Guofeng Mei, Fabio Poiesi, Cristiano Saltori, Jian Zhang, Elisa Ricci, Nicu Sebe
Probabilistic 3D point cloud registration methods have shown competitive performance in overcoming noise, outliers, and density variations.
no code implementations • 14 Oct 2022 • Samuel Felipe dos Santos, Rodrigo Berriel, Thiago Oliveira-Santos, Nicu Sebe, Jurandy Almeida
Nevertheless, the models are usually larger than the baseline for a single domain.
1 code implementation • 6 Oct 2022 • Guofeng Mei, Cristiano Saltori, Fabio Poiesi, Jian Zhang, Elisa Ricci, Nicu Sebe, Qiang Wu
Unsupervised learning on 3D point clouds has undergone a rapid evolution, especially thanks to data augmentation-based contrastive methods.
no code implementations • 5 Oct 2022 • Ye Zhu, Yu Wu, Nicu Sebe, Yan Yan
We are perceiving and communicating with the world in a multisensory manner, where different information sources are sophisticatedly processed and interpreted by separate parts of the human brain to constitute a complex, yet harmonious and unified sensing system.
1 code implementation • 3 Oct 2022 • Yahui Liu, Enver Sangineto, Yajing Chen, Linchao Bao, Haoxian Zhang, Nicu Sebe, Bruno Lepri, Marco De Nadai
Multi-domain image-to-image (I2I) translations can transform a source image according to the style of a target domain.
no code implementations • 30 Sep 2022 • Weijie Wang, Nicu Sebe, Bruno Lepri
Due to the subjective crowdsourcing annotations and the inherent inter-class similarity of facial expressions, the real-world Facial Expression Recognition (FER) datasets usually exhibit ambiguous annotation.
Facial Expression Recognition
Facial Expression Recognition (FER)
1 code implementation • 18 Sep 2022 • Yue Song, Nicu Sebe, Wei Wang
The task of out-of-distribution (OOD) detection is crucial for deploying machine learning models in real-world settings.
1 code implementation • 5 Sep 2022 • Hao Tang, Nicu Sebe
We propose a simple yet powerful Landmark guided Generative Adversarial Network (LandmarkGAN) for the facial expression-to-expression translation using a single image, which is an important and challenging task in computer vision since the expression-to-expression translation is a non-linear and non-aligned problem.
1 code implementation • 26 Aug 2022 • Jichao Zhang, Aliaksandr Siarohin, Yahui Liu, Hao Tang, Nicu Sebe, Wei Wang
Generative Neural Radiance Fields (GNeRF) based 3D-aware GANs have demonstrated remarkable capabilities in generating high-quality images while maintaining strong 3D consistency.
1 code implementation • 16 Aug 2022 • Subhankar Roy, Martin Trapp, Andrea Pilzer, Juho Kannala, Nicu Sebe, Elisa Ricci, Arno Solin
Source-free domain adaptation (SFDA) aims to adapt a classifier to an unlabelled target data set by only using a pre-trained source model.
1 code implementation • 26 Jul 2022 • Victor G. Turrisi da Costa, Giacomo Zara, Paolo Rota, Thiago Oliveira-Santos, Nicu Sebe, Vittorio Murino, Elisa Ricci
On the other hand, the performance of a model in action recognition is heavily affected by domain shift.
2 code implementations • 20 Jul 2022 • Cristiano Saltori, Evgeny Krivosheev, Stéphane Lathuilière, Nicu Sebe, Fabio Galasso, Giuseppe Fiameni, Elisa Ricci, Fabio Poiesi
Our experiments show the effectiveness of our segmentation approach on thousands of real-world point clouds.
2 code implementations • 20 Jul 2022 • Cristiano Saltori, Fabio Galasso, Giuseppe Fiameni, Nicu Sebe, Elisa Ricci, Fabio Poiesi
We propose a new approach of sample mixing for point cloud UDA, namely Compositional Semantic Mix (CoSMix), the first UDA approach for point cloud segmentation based on sample mixing.
1 code implementation • 18 Jul 2022 • Subhankar Roy, Mingxuan Liu, Zhun Zhong, Nicu Sebe, Elisa Ricci
We study the new task of class-incremental Novel Class Discovery (class-iNCD), which refers to the problem of discovering novel categories in an unlabelled data set by leveraging a pre-trained model that has been trained on a labelled data set containing disjoint yet related categories.
no code implementations • 11 Jul 2022 • Zhun Zhong, Yuyang Zhao, Gim Hee Lee, Nicu Sebe
Experiments on two synthetic-to-real semantic segmentation benchmarks demonstrate that AdvStyle can significantly improve the model performance on unseen real domains and show that we can achieve the state of the art.
1 code implementation • 9 Jul 2022 • Yue Song, Nicu Sebe, Wei Wang
EigenDecomposition (ED) is at the heart of many computer vision algorithms and applications.
1 code implementation • 9 Jul 2022 • Bin Ren, Hao Tang, Yiming Wang, Xia Li, Wei Wang, Nicu Sebe
For semantic-guided cross-view image translation, it is crucial to learn where to sample pixels from the source view image and where to reallocate them guided by the target view semantic map, especially when there is little overlap or drastic view difference between the source and target images.
1 code implementation • 5 Jul 2022 • Yue Song, Nicu Sebe, Wei Wang
Inserting an SVD meta-layer into neural networks is prone to make the covariance ill-conditioned, which could harm the model in the training stability and generalization abilities.
1 code implementation • 4 Jul 2022 • Baptiste Chopin, Hao Tang, Naima Otberdout, Mohamed Daoudi, Nicu Sebe
To address this limitation, we propose a novel interaction Transformer (InterFormer) consisting of a Transformer network with both temporal and spatial attention.
1 code implementation • 1 Jul 2022 • Jichao Zhang, Jingjing Chen, Hao Tang, Enver Sangineto, Peng Wu, Yan Yan, Nicu Sebe, Wei Wang
Solving this problem using an unsupervised method remains an open problem, especially for high-resolution face images in the wild, which are not easy to annotate with gaze and head pose labels.
1 code implementation • 9 Jun 2022 • Elia Peruzzo, Enver Sangineto, Yahui Liu, Marco De Nadai, Wei Bi, Bruno Lepri, Nicu Sebe
In this work, we propose a different and complementary direction, in which a local bias is introduced using an auxiliary self-supervised task, performed jointly with standard supervised training.
1 code implementation • 26 May 2022 • Yue Song, Nicu Sebe, Wei Wang
Inspired by this observation, we propose a network branch dedicated to magnifying the importance of small eigenvalues.
Ranked #4 on
Fine-Grained Image Classification
on Stanford Dogs
Fine-Grained Image Classification
Fine-Grained Visual Categorization
+1
1 code implementation • CVPR 2023 • Bin Ren, Yahui Liu, Yue Song, Wei Bi, Rita Cucchiara, Nicu Sebe, Wei Wang
In particular, MJP first shuffles the selected patches via our block-wise random jigsaw puzzle shuffle algorithm, and their corresponding PEs are occluded.
no code implementations • 18 May 2022 • Florinel-Alin Croitoru, Nicolae-Catalin Ristea, Radu Tudor Ionescu, Nicu Sebe
In this work, we propose a novel curriculum learning approach termed Learning Rate Curriculum (LeRaC), which leverages the use of a different learning rate for each layer of a neural network to create a data-free curriculum during the initial training epochs.
Ranked #2 on
Speech Emotion Recognition
on CREMA-D
1 code implementation • IEEE Transactions on Image Processing (TIP) 2022 • Jinliang Lin, Zhedong Zheng, Zhun Zhong, Zhiming Luo, Shaozi Li, Yi Yang, Nicu Sebe
Inspired by the human visual system for mining local patterns, we propose a new framework called RK-Net to jointly learn the discriminative Representation and detect salient Keypoints with a single Network.
Ranked #2 on
Drone navigation
on University-1652
1 code implementation • 7 Apr 2022 • Aleksandr Ermolov, Enver Sangineto, Nicu Sebe
Inspired by human memory, we propose to represent history with only important changes in the environment and, in our approach, to obtain automatically this representation using self-supervision.
2 code implementations • 6 Apr 2022 • Yuyang Zhao, Zhun Zhong, Na Zhao, Nicu Sebe, Gim Hee Lee
Furthermore, we present a novel style hallucination module (SHM) to generate style-diversified samples that are essential to consistency learning.
1 code implementation • 22 Mar 2022 • Songsong Wu, Hao Tang, Xiao-Yuan Jing, Haifeng Zhao, Jianjun Qian, Nicu Sebe, Yan Yan
In this paper, we tackle the problem of synthesizing a ground-view panorama image conditioned on a top-view aerial image, which is a challenging problem due to the large gap between the two image domains with different view-points.
2 code implementations • CVPR 2022 • Aleksandr Ermolov, Leyla Mirvakhabova, Valentin Khrulkov, Nicu Sebe, Ivan Oseledets
Following this line of work, we propose a new hyperbolic-based model for metric learning.
Ranked #1 on
Metric Learning
on CUB-200-2011
no code implementations • 5 Mar 2022 • Fengxiang Yang, Zhun Zhong, Zhiming Luo, Shaozi Li, Nicu Sebe
During local training, the DFS are used to synthesize novel domain statistics with the proposed domain hallucinating, which is achieved by re-weighting DFS with random weights.
no code implementations • 3 Mar 2022 • Yongguo Ling, Zhun Zhong, Donglin Cao, Zhiming Luo, Yaojin Lin, Shaozi Li, Nicu Sebe
In this manner, the model will focus on reducing the inter-modality discrepancy while paying less attention to intra-identity variations, leading to a more effective modality alignment.
1 code implementation • 28 Feb 2022 • Hao Tang, Ling Shao, Philip H. S. Torr, Nicu Sebe
To learn more discriminative class-specific feature representations for the local generation, we also propose a novel classification module.
no code implementations • 22 Feb 2022 • Yuyu Guo, Lianli Gao, Jingkuan Song, Peng Wang, Nicu Sebe, Heng Tao Shen, Xuelong Li
Inspired by this observation, in this article, we propose a relation regularized network (R2-Net), which can predict whether there is a relationship between two objects and encode this relation into object feature refinement and better SGG.
no code implementations • 8 Feb 2022 • Yue Song, Hao Tang, Nicu Sebe, Wei Wang
Specifically, the detail modeling focuses on capturing the object edges by supervision of explicitly decomposed detail label that consists of the pixels that are nested on the edge and near the edge.
1 code implementation • 29 Jan 2022 • Yue Song, Nicu Sebe, Wei Wang
Computing the matrix square root and its inverse in a differentiable manner is important in a variety of computer vision tasks.
1 code implementation • ICLR 2022 • Yue Song, Nicu Sebe, Wei Wang
Previous methods either adopt the Singular Value Decomposition (SVD) to explicitly factorize the matrix or use the Newton-Schulz iteration (NS iteration) to derive the approximate solution.
1 code implementation • 14 Dec 2021 • Haoyu Chen, Hao Tang, Zitong Yu, Nicu Sebe, Guoying Zhao
Specifically, we propose a novel geometry-contrastive Transformer that has an efficient 3D structured perceiving ability to the global geometric inconsistencies across the given meshes.
1 code implementation • CVPR 2022 • Yuyang Zhao, Zhun Zhong, Nicu Sebe, Gim Hee Lee
We introduce a new setting of Novel Class Discovery in Semantic Segmentation (NCDSS), which aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes.
1 code implementation • 2 Dec 2021 • Jichao Zhang, Enver Sangineto, Hao Tang, Aliaksandr Siarohin, Zhun Zhong, Nicu Sebe, Wei Wang
However, they usually struggle to generate high-quality images representing non-rigid objects, such as the human body, which is of a great interest for many computer graphics applications.
1 code implementation • CVPR 2022 • Zipeng Xu, Tianwei Lin, Hao Tang, Fu Li, Dongliang He, Nicu Sebe, Radu Timofte, Luc van Gool, Errui Ding
We propose a novel framework, i. e., Predict, Prevent, and Evaluate (PPE), for disentangled text-driven image manipulation that requires little manual annotation while being applicable to a wide variety of manipulations.
1 code implementation • 19 Nov 2021 • Guanglei Yang, Hao Tang, Humphrey Shi, Mingli Ding, Nicu Sebe, Radu Timofte, Luc van Gool, Elisa Ricci
The global alignment network aims to transfer the input image from the source domain to the target domain.
1 code implementation • 19 Nov 2021 • Guanglei Yang, Zhun Zhong, Hao Tang, Mingli Ding, Nicu Sebe, Elisa Ricci
Specifically, in the image translation stage, Bi-Mix leverages the knowledge of day-night image pairs to improve the quality of nighttime image relighting.
1 code implementation • 20 Oct 2021 • Haoyu Chen, Hao Tang, Nicu Sebe, Guoying Zhao
Instead, we introduce AniFormer, a novel Transformer-based architecture, that generates animated 3D sequences by directly taking the raw driving sequences and arbitrary same-type target meshes as inputs.
1 code implementation • 19 Oct 2021 • Bin Ren, Hao Tang, Nicu Sebe
To ease this problem, we propose a novel two-stage framework with a new Cascaded Cross MLP-Mixer (CrossMLP) sub-network in the first stage and one refined pixel-level loss in the second stage.
1 code implementation • 26 Sep 2021 • Yahui Liu, Yajing Chen, Linchao Bao, Nicu Sebe, Bruno Lepri, Marco De Nadai
The ISF manipulates the semantics of an input latent code to make the image generated from it lying in the desired visual domain.
1 code implementation • 29 Aug 2021 • Hao Tang, Nicu Sebe
In this paper, we address the task of layout-to-image translation, which aims to translate an input semantic layout to a realistic image.
1 code implementation • ICCV 2021 • Haoyu Chen, Hao Tang, Henglin Shi, Wei Peng, Nicu Sebe, Guoying Zhao
With the strength of deep generative models, 3D pose transfer regains intensive research interests in recent years.
5 code implementations • 3 Aug 2021 • Victor G. Turrisi da Costa, Enrico Fini, Moin Nabi, Nicu Sebe, Elisa Ricci
This paper presents solo-learn, a library of self-supervised methods for visual representation learning.
1 code implementation • 21 Jun 2021 • Hao Tang, Nicu Sebe
Both generators are mutually connected and trained in an end-to-end fashion and explicitly form three cycled subnets, i. e., one image generation cycle and two guidance generation cycles.
1 code implementation • CVPR 2021 • Zhun Zhong, Enrico Fini, Subhankar Roy, Zhiming Luo, Elisa Ricci, Nicu Sebe
In this paper, we address Novel Class Discovery (NCD), the task of unveiling new classes in a set of unlabeled samples given a labeled dataset with known classes.
no code implementations • CVPR 2021 • Yahui Liu, Enver Sangineto, Yajing Chen, Linchao Bao, Haoxian Zhang, Nicu Sebe, Bruno Lepri, Wei Wang, Marco De Nadai
In this paper, we propose a new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space in which: 1) Both intra- and inter-domain interpolations correspond to gradual changes in the generated images and 2) The content of the source image is better preserved during the translation.
no code implementations • 7 Jun 2021 • Yuyang Zhao, Zhun Zhong, Zhiming Luo, Gim Hee Lee, Nicu Sebe
Second, CPSS can reduce the influence of noisy pseudo-labels and also avoid the model overfitting to the target domain during self-supervised learning, consistently boosting the performance on the target and open domains.
1 code implementation • NeurIPS 2021 • Yahui Liu, Enver Sangineto, Wei Bi, Nicu Sebe, Bruno Lepri, Marco De Nadai
This task encourages the VTs to learn spatial relations within an image and makes the VT training much more robust when training data are scarce.
1 code implementation • 31 May 2021 • Jichao Zhang, Aliaksandr Siarohin, Hao Tang, Enver Sangineto, Wei Wang, Humphrey Sh, Nicu Sebe
Moreover, we propose a novel Self-Training Part Replacement (STPR) strategy to refine the model for the texture-transfer task, which improves the quality of the generated clothes and the preservation ability of non-target regions.
1 code implementation • 28 May 2021 • Guanglei Yang, Hao Tang, Zhun Zhong, Mingli Ding, Ling Shao, Nicu Sebe, Elisa Ricci
In this paper, we study the task of source-free domain adaptation (SFDA), where the source data are not available during target adaptation.
1 code implementation • ICCV 2021 • Yue Song, Nicu Sebe, Wei Wang
Singular Value Decomposition (SVD) is used in GCP to compute the matrix square root.
1 code implementation • 12 Apr 2021 • Bin Ren, Hao Tang, Fanyang Meng, Runwei Ding, Philip H. S. Torr, Nicu Sebe
In the second stage, we put forth a CIT reasoning block for establishing global mutual interactive dependencies among person representation, the warped clothing item, and the corresponding warped cloth mask.
1 code implementation • CVPR 2021 • Subhankar Roy, Evgeny Krivosheev, Zhun Zhong, Nicu Sebe, Elisa Ricci
In this paper we address multi-target domain adaptation (MTDA), where given one labeled source dataset and multiple unlabeled target datasets that differ in data distributions, the task is to learn a robust predictor for all the target domains.
Ranked #2 on
Blended-target Domain Adaptation
on DomainNet
Blended-target Domain Adaptation
Multi-target Domain Adaptation
1 code implementation • ICCV 2021 • Guanglei Yang, Hao Tang, Mingli Ding, Nicu Sebe, Elisa Ricci
While convolutional neural networks have shown a tremendous impact on various computer vision tasks, they generally demonstrate limitations in explicitly modeling long-range dependencies due to the intrinsic locality of the convolution operation.
Ranked #7 on
Depth Estimation
on NYU-Depth V2
1 code implementation • CVPR 2021 • Fengxiang Yang, Zhun Zhong, Zhiming Luo, Yuanzheng Cai, Yaojin Lin, Shaozi Li, Nicu Sebe
This paper considers the problem of unsupervised person re-identification (re-ID), which aims to learn discriminative models with unlabeled data.
1 code implementation • IJCV 2021 • Shangzhi Teng, Shiliang Zhang, Qingming Huang, Nicu Sebe
Moreover, our method also achieves competitive performance compared with recent works on existing vehicle ReID datasets including VehicleID, VeRi-776 and VERI-Wild.
no code implementations • 25 Jan 2021 • Petru Soviany, Radu Tudor Ionescu, Paolo Rota, Nicu Sebe
Training machine learning models in a meaningful order, from the easy samples to the hard ones, using curriculum learning can provide performance improvements over the standard training approach based on random data shuffling, without any additional computational costs.
no code implementations • 8 Jan 2021 • Dan Xu, Xavier Alameda-Pineda, Wanli Ouyang, Elisa Ricci, Xiaogang Wang, Nicu Sebe
In contrast to previous works directly considering multi-scale feature maps obtained from the inner layers of a primary CNN architecture, and simply fusing the features with weighted averaging or concatenation, we propose a probabilistic graph attention network structure based on a novel Attention-Gated Conditional Random Fields (AG-CRFs) model for learning and fusing multi-scale representations in a principled manner.
no code implementations • ICCV 2021 • Xinyan Liu, Guorong Li, Zhenjun Han, Weigang Zhang, Yifan Yang, Qingming Huang, Nicu Sebe
Specifically, we propose a task-driven similarity metric based on sample's mutual enhancement, referred as co-fine-tune similarity, which can find a more efficient subset of data for training the expert network.
no code implementations • 1 Jan 2021 • Hao Tang, Nicu Sebe
We propose the Semantically-Adaptive UpSampling (SA-UpSample), a general and highly effective upsampling method for the layout-to-image translation task.
no code implementations • 26 Dec 2020 • Samuel Felipe dos Santos, Nicu Sebe, Jurandy Almeida
In this paper, we propose a further study of the computational cost of deep models designed for the frequency domain, evaluating the cost of decoding and passing the images through the network.
1 code implementation • CVPR 2021 • Yuyang Zhao, Zhun Zhong, Fengxiang Yang, Zhiming Luo, Yaojin Lin, Shaozi Li, Nicu Sebe
In this paper, we study the problem of multi-source domain generalization in ReID, which aims to learn a model that can perform well on unseen domains with only several labeled source domains.
1 code implementation • 7 Nov 2020 • Jean Pablo Vieira de Mello, Lucas Tabelini, Rodrigo F. Berriel, Thiago M. Paixão, Alberto F. de Souza, Claudine Badue, Nicu Sebe, Thiago Oliveira-Santos
By providing real image samples with traffic context to the network, the model learns to detect and classify elements of interest, such as pedestrians, traffic signs, and traffic lights.
1 code implementation • 16 Oct 2020 • Cristiano Saltori, Stéphane Lathuiliére, Nicu Sebe, Elisa Ricci, Fabio Galasso
In the case of LiDAR, in fact, domain shift is not only due to changes in the environment and in the object appearances, as for visual data from RGB cameras, but is also related to the geometry of the point clouds (e. g., point density variations).
1 code implementation • NeurIPS 2020 • Aleksandr Ermolov, Nicu Sebe
In this work we consider partially observable environments with sparse rewards.
1 code implementation • 29 Aug 2020 • Hao Tang, Song Bai, Nicu Sebe
We also propose two novel modules, i. e., position-wise Spatial Attention Module (SAM) and scale-wise Channel Attention Module (CAM), to capture semantic structure attention in spatial and channel dimensions, respectively.
1 code implementation • 11 Aug 2020 • Raul Gomez, Yahui Liu, Marco De Nadai, Dimosthenis Karatzas, Bruno Lepri, Nicu Sebe
In this paper we propose the use of an image retrieval system to assist the image-to-image translation task.
1 code implementation • 10 Aug 2020 • Hao Tang, Song Bai, Philip H. S. Torr, Nicu Sebe
We present a novel Bipartite Graph Reasoning GAN (BiGraphGAN) for the challenging person image generation task.
Ranked #1 on
Pose Transfer
on Market-1501
(PCKh metric)
1 code implementation • 10 Aug 2020 • Yahui Liu, Marco De Nadai, Deng Cai, Huayang Li, Xavier Alameda-Pineda, Nicu Sebe, Bruno Lepri
Our proposed model disentangles the image content from the visual attributes, and it learns to modify the latter using the textual description, before generating a new image from the content and the modified attribute representation.
1 code implementation • 9 Aug 2020 • Jichao Zhang, Jingjing Chen, Hao Tang, Wei Wang, Yan Yan, Enver Sangineto, Nicu Sebe
In this paper we address the problem of unsupervised gaze correction in the wild, presenting a solution that works without the need for precise annotations of the gaze angle and the head pose.
no code implementations • 30 Jul 2020 • Lucas Tabelini, Rodrigo Berriel, Thiago M. Paixão, Alberto F. de Souza, Claudine Badue, Nicu Sebe, Thiago Oliveira-Santos
The method does not aim at overcoming the training with real data, but to be a compatible alternative when the real data is not available.
2 code implementations • ECCV 2020 • Hao Tang, Song Bai, Li Zhang, Philip H. S. Torr, Nicu Sebe
We propose a novel Generative Adversarial Network (XingGAN or CrossingGAN) for person image generation tasks, i. e., translating the pose of a given person to a desired one.
Ranked #1 on
Pose Transfer
on Market-1501
(IS metric)
8 code implementations • 13 Jul 2020 • Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, Nicu Sebe
Most of the current self-supervised representation learning (SSL) methods are based on the contrastive loss and the instance-discrimination task, where augmented versions of the same image instance ("positives") are contrasted with instances extracted from other images ("negatives").
1 code implementation • 21 May 2020 • Hao Tang, Hong Liu, Wei Xiao, Nicu Sebe
Then the activated dictionary atoms are assembled and passed to the compound dictionary learning and coding layers.
no code implementations • 9 May 2020 • Weiyao Lin, Huabin Liu, Shizhan Liu, Yuxi Li, Rui Qian, Tao Wang, Ning Xu, Hongkai Xiong, Guo-Jun Qi, Nicu Sebe
To this end, we present a new large-scale dataset with comprehensive annotations, named Human-in-Events or HiEve (Human-centric video analysis in complex Events), for the understanding of human motions, poses, and actions in a variety of realistic events, especially in crowd & complex events.
no code implementations • 19 Apr 2020 • Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Nicu Sebe, Elisa Ricci
In this paper we propose the first approach for Multi-Source Domain Adaptation (MSDA) based on Generative Adversarial Networks.
no code implementations • 13 Apr 2020 • Shreya Ghosh, Abhinav Dhall, Garima Sharma, Sarthak Gupta, Nicu Sebe
In this paper, a fully automatic technique for labelling an image based gaze behavior dataset for driver gaze zone estimation is proposed.
no code implementations • CVPR 2021 • Zhun Zhong, Linchao Zhu, Zhiming Luo, Shaozi Li, Yi Yang, Nicu Sebe
In this paper, we tackle the problem of discovering new classes in unlabeled visual data given labeled data from disjoint classes.
1 code implementation • 7 Apr 2020 • Jingjing Chen, Jichao Zhang, Enver Sangineto, Jiayuan Fan, Tao Chen, Nicu Sebe
In this paper, we propose to alleviate these problems by means of a novel gaze redirection framework which exploits both a numerical and a pictorial direction guidance, jointly with a coarse-to-fine learning strategy.
2 code implementations • 7 Apr 2020 • Aliaksandr Siarohin, Subhankar Roy, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe
To overcome this limitation, we propose a self-supervised deep learning method for co-part segmentation.
Ranked #2 on
Unsupervised Human Pose Estimation
on Tai-Chi-HD
2 code implementations • 31 Mar 2020 • Haotong Qin, Ruihao Gong, Xianglong Liu, Xiao Bai, Jingkuan Song, Nicu Sebe
The binary neural network, largely saving the storage and computation, serves as a promising technique for deploying deep models on resource-limited devices.
2 code implementations • 31 Mar 2020 • Hao Tang, Xiaojuan Qi, Guolei Sun, Dan Xu, Nicu Sebe, Radu Timofte, Luc van Gool
We propose a novel ECGAN for the challenging semantic image synthesis task.
1 code implementation • 15 Mar 2020 • Yahui Liu, Marco De Nadai, Jian Yao, Nicu Sebe, Bruno Lepri, Xavier Alameda-Pineda
Unsupervised image-to-image translation (UNIT) aims at learning a mapping between several visual domains by using unpaired training images.
2 code implementations • NeurIPS 2019 • Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe
To achieve this, we decouple appearance and motion information using a self-supervised formulation.
Ranked #1 on
Video Reconstruction
on Tai-Chi-HD
1 code implementation • 3 Feb 2020 • Hao Tang, Philip H. S. Torr, Nicu Sebe
In the first stage, the input image and the conditional semantic guidance are fed into a cycled semantic-guided generation network to produce initial coarse results.
1 code implementation • 2 Feb 2020 • Mariana-Iuliana Georgescu, Radu Tudor Ionescu, Nicolae-Catalin Ristea, Nicu Sebe
In order to classify linearly non-separable data, neurons are typically organized into multi-layer neural networks that are equipped with at least one hidden layer.
Ranked #6 on
Speech Emotion Recognition
on CREMA-D
no code implementations • 1 Jan 2020 • Jurandy Almeida, Cristiano Saltori, Paolo Rota, Nicu Sebe
Deep learning revolution happened thanks to the availability of a massive amount of labelled data which have contributed to the development of models with extraordinary inference capabilities.
2 code implementations • CVPR 2020 • Hao Tang, Dan Xu, Yan Yan, Philip H. S. Torr, Nicu Sebe
To tackle this issue, in this work we consider learning the scene generation in a local context, and correspondingly design a local class-specific generative network with semantic maps as a guidance, which separately constructs and learns sub-generators concentrating on the generation of different classes, and is able to provide more scene details.
1 code implementation • 14 Dec 2019 • Hao Tang, Dan Xu, Hong Liu, Nicu Sebe
In this paper, we analyze the limitation of the existing symmetric GAN models in asymmetric translation tasks, and propose an AsymmetricGAN model with both translation and reconstruction generators of unequal sizes and different parameter-sharing strategy to adapt to the asymmetric need in both unsupervised and supervised image-to-image translation tasks.
1 code implementation • 12 Dec 2019 • Hao Tang, Hong Liu, Nicu Sebe
The proposed model consists of a single generator and a discriminator taking a conditional image and the target controllable structure as input.
Ranked #1 on
Cross-View Image-to-Image Translation
on cvusa
Facial Expression Translation
Gesture-to-Gesture Translation
+2
2 code implementations • 27 Nov 2019 • Hao Tang, Hong Liu, Dan Xu, Philip H. S. Torr, Nicu Sebe
State-of-the-art methods in image-to-image translation are capable of learning a mapping from a source domain to a target domain with unpaired image data.
Ranked #1 on
Facial Expression Translation
on CelebA
no code implementations • 15 Nov 2019 • Petru Soviany, Radu Tudor Ionescu, Paolo Rota, Nicu Sebe
To alleviate this problem, researchers proposed various domain adaptation methods to improve object detection results in the cross-domain setting, e. g. by translating images with ground-truth labels from the source domain to the target domain using Cycle-GAN.
no code implementations • 25 Sep 2019 • Aleksandr Ermolov, Enver Sangineto, Nicu Sebe
To address this problem, a possible solution is to provide the agent with information about past observations.
1 code implementation • 17 Sep 2019 • Andrea Pilzer, Stéphane Lathuilière, Dan Xu, Mihai Marian Puscas, Elisa Ricci, Nicu Sebe
Extensive experiments on the publicly available datasets KITTI, Cityscapes and ApolloScape demonstrate the effectiveness of the proposed model which is competitive with other unsupervised deep learning methods for depth prediction.
no code implementations • 15 Aug 2019 • Mihai Marian Puscas, Dan Xu, Andrea Pilzer, Nicu Sebe
Inspired by the success of adversarial learning, we propose a new end-to-end unsupervised deep learning framework for monocular depth estimation consisting of two Generative Adversarial Networks (GAN), deeply coupled with a structured Conditional Random Field (CRF) model.
Monocular Depth Estimation
Unsupervised Monocular Depth Estimation
1 code implementation • 2 Aug 2019 • Hao Tang, Dan Xu, Gaowen Liu, Wei Wang, Nicu Sebe, Yan Yan
In this work, we propose a novel Cycle In Cycle Generative Adversarial Network (C$^2$GAN) for the task of keypoint-guided image generation.
1 code implementation • 23 Jul 2019 • Lucas Tabelini Torres, Thiago M. Paixão, Rodrigo F. Berriel, Alberto F. de Souza, Claudine Badue, Nicu Sebe, Thiago Oliveira-Santos
Deep learning has been successfully applied to several problems related to autonomous driving.
1 code implementation • 19 Jul 2019 • Vinicius F. Arruda, Thiago M. Paixão, Rodrigo F. Berriel, Alberto F. De Souza, Claudine Badue, Nicu Sebe, Thiago Oliveira-Santos
In this work, a method for training a car detection system with annotated data from a source domain (day images) without requiring the image annotations of the target domain (night images) is presented.
1 code implementation • 12 Jul 2019 • Yahui Liu, Marco De Nadai, Gloria Zen, Nicu Sebe, Bruno Lepri
In this work, we propose a novel GAN architecture that decouples the required annotations into a category label - that specifies the gesture type - and a simple-to-draw category-independent conditional map - that expresses the location, rotation and size of the hand gesture.
no code implementations • CVPR 2019 • Zhen-Yu Zhang, Zhen Cui, Chunyan Xu, Yan Yan, Nicu Sebe, Jian Yang
In this paper, we propose a novel Pattern-Affinitive Propagation (PAP) framework to jointly predict depth, surface normal and semantic segmentation.
Ranked #44 on
Monocular Depth Estimation
on NYU-Depth V2
no code implementations • arXiv 2019 • Jichao Zhang, Meng Sun, Jingjing Chen, Hao Tang, Yan Yan, Xueying Qin, Nicu Sebe
Gaze correction aims to redirect the person's gaze into the camera by manipulating the eye region, and it can be considered as a specific image resynthesis problem.
no code implementations • 26 May 2019 • Romain Belmonte, Benjamin Allaert, Pierre Tirilly, Ioan Marius Bilasco, Chaabane Djeraba, Nicu Sebe
Although facial landmark localization (FLL) approaches are becoming increasingly accurate for characterizing facial regions, one question remains unanswered: what is the impact of these approaches on subsequent related tasks?
no code implementations • 15 May 2019 • Cristiano Saltori, Subhankar Roy, Nicu Sebe, Giovanni Iacca
Although very effective, evolutionary algorithms rely heavily on having a large population of individuals (i. e., network architectures) and is therefore memory expensive.
no code implementations • ICCV 2019 • Rodrigo Berriel, Stéphane Lathuilière, Moin Nabi, Tassilo Klein, Thiago Oliveira-Santos, Nicu Sebe, Elisa Ricci
To implement this idea we derive specialized deep models for each domain by adapting a pre-trained architecture but, differently from other methods, we propose a novel strategy to automatically adjust the computational complexity of the network.
no code implementations • 14 May 2019 • Hao Tang, Wei Wang, Songsong Wu, Xinya Chen, Dan Xu, Nicu Sebe, Yan Yan
In this paper, we focus on the facial expression translation task and propose a novel Expression Conditional GAN (ECGAN) which can learn the mapping from one image domain to another one based on an additional expression attribute.
Facial expression generation
Facial Expression Translation
+1
no code implementations • 7 May 2019 • Stéphane Lathuilière, Enver Sangineto, Aliaksandr Siarohin, Nicu Sebe
We present a generalization of the person-image generation task, in which a human image is generated conditioned on a target pose and a set X of source appearance images.
no code implementations • ICLR 2019 • Aliaksandr Siarohin, Enver Sangineto, Nicu Sebe
In this paper we propose to generalize both BN and cBN using a Whitening and Coloring based batch normalization.
1 code implementation • 30 Apr 2019 • Aliaksandr Siarohin, Stéphane Lathuilière, Enver Sangineto, Nicu Sebe
Specifically, given an image xa of a person and a target pose P(xb), extracted from a different image xb, we synthesize a new image of that person in pose P(xb), while preserving the visual details in xa.
no code implementations • 17 Apr 2019 • Zhen-Yu Zhang, Stéphane Lathuilière, Andrea Pilzer, Nicu Sebe, Elisa Ricci, Jian Yang
Our proposal is evaluated on the wellestablished KITTI dataset, where we show that our online method is competitive withstate of the art algorithms trained in a batch setting.
3 code implementations • CVPR 2019 • Hao Tang, Dan Xu, Nicu Sebe, Yanzhi Wang, Jason J. Corso, Yan Yan
In this paper, we propose a novel approach named Multi-Channel Attention SelectionGAN (SelectionGAN) that makes it possible to generate images of natural scenes in arbitrary viewpoints, based on an image of the scene and a novel semantic map.
Bird View Synthesis
Cross-View Image-to-Image Translation
+1
1 code implementation • 2 Apr 2019 • Subhankar Roy, Enver Sangineto, Begüm Demir, Nicu Sebe
Hashing methods have been recently found very effective in retrieval of remote sensing (RS) images due to their computational efficiency and fast search speed.
8 code implementations • 28 Mar 2019 • Hao Tang, Dan Xu, Nicu Sebe, Yan Yan
To handle the limitation, in this paper we propose a novel Attention-Guided Generative Adversarial Network (AGGAN), which can detect the most discriminative semantic object and minimize changes of unwanted part for semantic manipulation problems without using extra data and models.
Ranked #1 on
Facial Expression Translation
on Bu3dfe
no code implementations • CVPR 2019 • Andrea Pilzer, Stéphane Lathuilière, Nicu Sebe, Elisa Ricci
Therefore, recent works have proposed deep architectures for addressing the monocular depth prediction task as a reconstruction problem, thus avoiding the need of collecting ground-truth depth.
1 code implementation • CVPR 2019 • Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Samuel Rota Bulo, Nicu Sebe, Elisa Ricci
A classifier trained on a dataset seldom works on other datasets obtained under different conditions due to domain shift.
1 code implementation • 28 Jan 2019 • Hao Tang, Xinya Chen, Wei Wang, Dan Xu, Jason J. Corso, Nicu Sebe, Yan Yan
To this end, we propose a novel Attribute-Guided Sketch Generative Adversarial Network (ASGAN) which is an end-to-end framework and contains two pairs of generators and discriminators, one of which is used to generate faces with attributes while the other one is employed for image-to-sketch translation.
1 code implementation • 15 Jan 2019 • Hao Tang, Hong Liu, Wei Xiao, Nicu Sebe
Gesture recognition is a hot topic in computer vision and pattern recognition, which plays a vitally important role in natural human-computer interface.
Ranked #1 on
Hand Gesture Recognition
on Cambridge
1 code implementation • 14 Jan 2019 • Hao Tang, Dan Xu, Wei Wang, Yan Yan, Nicu Sebe
State-of-the-art methods for image-to-image translation with Generative Adversarial Networks (GANs) can learn a mapping from one domain to another domain using unpaired image data.