1 code implementation • 28 Aug 2019 • Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, Liqing Zhang
Image composition is an important operation in image processing, but the inconsistency between foreground and background significantly degrades the quality of composite image.
1 code implementation • CVPR 2020 • Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, Liqing Zhang
Image composition is an important operation in image processing, but the inconsistency between foreground and background significantly degrades the quality of composite image.
Ranked #6 on Image Harmonization on HAdobe5k(1024$\times$1024)
4 code implementations • 28 Jun 2021 • Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, Liqing Zhang
We have also contributed the first image composition toolbox: libcom https://github. com/bcmi/libcom, which assembles 10+ image composition related functions (e. g., image blending, image harmonization, object placement, shadow generation, generative composition).
2 code implementations • 16 Aug 2020 • Zhangxuan Gu, Siyuan Zhou, Li Niu, Zihan Zhao, Liqing Zhang
In this paper, we propose a novel context-aware feature generation method for zero-shot segmentation named CaGNet.
1 code implementation • 23 May 2021 • Lu He, Qianyu Zhou, Xiangtai Li, Li Niu, Guangliang Cheng, Xiao Li, Wenxuan Liu, Yunhai Tong, Lizhuang Ma, Liqing Zhang
Recently, DETR and Deformable DETR have been proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance as previous complex hand-crafted detectors.
1 code implementation • 8 Aug 2021 • Jing Liang, Li Niu, Fengjun Guo, Teng Long, Liqing Zhang
In the refinement stage, we integrate multi-level features to improve the texture quality of watermarked area.
1 code implementation • 6 Oct 2021 • Li Niu
Deep learning is a data-hungry approach, which requires massive training data.
1 code implementation • 21 Apr 2021 • Yan Hong, Li Niu, Jianfu Zhang, Liqing Zhang
In this work, we focus on generating plausible shadow for the foreground object in the composite image.
1 code implementation • 27 Jun 2019 • Zhangxuan Gu, Li Niu, Haohua Zhao, Liqing Zhang
Specifically, we propose a novel Loss Weight Module, which outputs a loss weight map by employing two depth-related measurements of hard pixels: Depth Prediction Error and Depthaware Segmentation Error.
1 code implementation • CVPR 2022 • Wenyan Cong, Xinhao Tao, Li Niu, Jing Liang, Xuesong Gao, Qihao Sun, Liqing Zhang
Conventional image harmonization methods learn global RGB-to-RGB transformation which could effortlessly scale to high resolution, but ignore diverse local context.
Ranked #3 on Image Harmonization on HAdobe5k(1024$\times$1024)
1 code implementation • 7 Apr 2021 • Bo Zhang, Li Niu, Liqing Zhang
Image composition assessment is crucial in aesthetic assessment, which aims to assess the overall composition quality of a given image.
Ranked #1 on Aesthetics Quality Assessment on CADB
1 code implementation • 19 Aug 2023 • Bo Zhang, Yuxuan Duan, Jun Lan, Yan Hong, Huijia Zhu, Weiqiang Wang, Li Niu
To address these challenges, we propose a controllable image composition method that unifies four tasks in one diffusion model: image blending, image harmonization, view synthesis, and generative composition.
1 code implementation • 5 Aug 2020 • Yan Hong, Li Niu, Jianfu Zhang, Weijie Zhao, Chen Fu, Liqing Zhang
In this paper, we propose a Fusing-and-Filling Generative Adversarial Network (F2GAN) to generate realistic and diverse images for a new category with only a few images.
3 code implementations • 5 Jul 2021 • Liu Liu, Zhenchen Liu, Bo Zhang, Jiangtong Li, Li Niu, Qingyang Liu, Liqing Zhang
Image composition aims to generate realistic composite image by inserting an object from one image into another background image, where the placement (e. g., location, size, occlusion) of inserted object may be unreasonable, which would significantly degrade the quality of the composite image.
1 code implementation • 23 Jul 2022 • Siyuan Zhou, Liu Liu, Li Niu, Liqing Zhang
Object placement aims to place a foreground object over a background image with a suitable location and size.
1 code implementation • 4 Mar 2023 • Yuxuan Duan, Yan Hong, Li Niu, Liqing Zhang
First, we train a data-efficient StyleGAN2 on defect-free images as the backbone.
1 code implementation • NeurIPS 2021 • Junjie Chen, Li Niu, Liu Liu, Liqing Zhang
In this setting, we propose a method called SimTrans to transfer pairwise semantic similarity from base categories to novel categories.
1 code implementation • 19 Sep 2020 • Wenyan Cong, Li Niu, Jianfu Zhang, Jing Liang, Liqing Zhang
Therefore, we propose an image harmonization network with a novel domain code extractor and well-tailored triplet losses, which could capture the background domain information to guide the foreground harmonization.
Ranked #13 on Image Harmonization on iHarmony4
1 code implementation • 18 Sep 2021 • Xinyuan Lu, Shengyuan Huang, Li Niu, Wenyan Cong, Liqing Zhang
In this work, we construct a new video harmonization dataset HYouTube by adjusting the foreground of real videos to create synthetic composite videos.
1 code implementation • 2 May 2022 • Xinyuan Lu, Shengyuan Huang, Li Niu, Wenyan Cong, Liqing Zhang
Video harmonization aims to adjust the foreground of a composite video to make it compatible with the background.
1 code implementation • 19 Aug 2023 • Qingyang Liu, Jianting Wang, Li Niu
In this work, we focus on generating plausible shadow for the inserted foreground object to make the composite image more realistic.
1 code implementation • 22 Mar 2024 • Qingyang Liu, Junqi You, Jianting Wang, Xinhao Tao, Bo Zhang, Li Niu
In the realm of image composition, generating realistic shadow for the inserted foreground remains a formidable challenge.
1 code implementation • NeurIPS 2021 • Yan Liu, Zhijie Zhang, Li Niu, Junjie Chen, Liqing Zhang
Specifically, the ability of using mask prior to help detect objects is learned from base categories and transferred to novel categories.
1 code implementation • 18 Sep 2020 • Yan Hong, Li Niu, Jianfu Zhang, Jing Liang, Liqing Zhang
In this work, we propose a novel Delta Generative Adversarial Network (DeltaGAN), which consists of a reconstruction subnetwork and a generation subnetwork.
1 code implementation • 21 Jul 2022 • Yan Hong, Li Niu, Jianfu Zhang, Liqing Zhang
In this work, we propose a novel Delta Generative Adversarial Network (DeltaGAN), which consists of a reconstruction subnetwork and a generation subnetwork.
1 code implementation • CVPR 2022 • Jiangtong Li, Li Niu, Liqing Zhang
We hope that Causal-VidQA can guide the research of video understanding from representation learning to deeper reasoning.
1 code implementation • 4 Aug 2023 • Lingxiao Lu, Jiangtong Li, Junyan Cao, Li Niu, Liqing Zhang
Painterly image harmonization aims to insert photographic objects into paintings and obtain artistically coherent composite images.
1 code implementation • 17 Dec 2022 • Junyan Cao, Yan Hong, Li Niu
In this work, we propose a novel painterly harmonization network consisting of a dual-domain generator and a dual-domain discriminator, which harmonizes the composite image in both spatial domain and frequency domain.
1 code implementation • 5 Oct 2022 • Junjie Chen, Li Niu, Siyuan Zhou, Jianlou Si, Chen Qian, Liqing Zhang
Proposal segmentation allows proposal-pixel similarity transfer from base classes to novel classes, which enables the mask learning of novel classes.
1 code implementation • 25 Sep 2020 • Zhangxuan Gu, Siyuan Zhou, Li Niu, Zihan Zhao, Liqing Zhang
Thus, we focus on zero-shot semantic segmentation, which aims to segment unseen objects with only category-level semantic representations provided for unseen categories.
1 code implementation • 5 Aug 2023 • Linfeng Tan, Jiangtong Li, Li Niu, Liqing Zhang
The network comprises a $RGB$ harmonization backbone, an $Lab$ encoding module, and an $Lab$ control module.
Ranked #1 on Image Harmonization on HAdobe5k(1024$\times$1024)
1 code implementation • 7 Mar 2020 • Yan Hong, Li Niu, Jianfu Zhang, Liqing Zhang
Matching generator can match random vectors with a few conditional images from the same category and generate new images for this category based on the fused features.
1 code implementation • ICCV 2023 • Li Niu, Junyan Cao, Wenyan Cong, Liqing Zhang
In particular, our designed SYthetic COmposite Network (SycoNet) takes in a real image with foreground mask and a random vector to learn suitable color transformation, which is applied to the foreground of this real image to produce a synthetic composite image.
1 code implementation • 28 May 2022 • Li Niu, Qingyang Liu, Zhenchen Liu, Jiangtong Li
However, given a pair of scaled foreground and background, to enumerate all the reasonable locations, existing OPA model needs to place the foreground at each location on the background and pass the obtained composite image through the model one at a time, which is very time-consuming.
1 code implementation • 19 Apr 2021 • Jing Liang, Li Niu, Liqing Zhang
The advance of image editing techniques allows users to create artistic works, but the manipulated regions may be incompatible with the background.
1 code implementation • 14 Dec 2020 • Ziqi Pan, Li Niu, Jianfu Zhang, Liqing Zhang
The information bottleneck (IB) method is a technique for extracting information that is relevant for predicting the target random variable from the source random variable, which is typically implemented by optimizing the IB Lagrangian that balances the compression and prediction terms.
1 code implementation • 21 Jul 2022 • Bo Zhang, Li Niu, Xing Zhao, Liqing Zhang
Image cropping aims to find visually appealing crops in an image, which is an important yet challenging task.
1 code implementation • ICCV 2023 • Bo Zhang, Jiacheng Sui, Li Niu
Additionally, previous works did not release their datasets, so we contribute two datasets for FOS task: S-FOSD dataset with synthetic composite images and R-FOSD dataset with real composite images.
1 code implementation • 31 Mar 2021 • Junyan Cao, Wenyan Cong, Li Niu, Jianfu Zhang, Liqing Zhang
Image harmonization has been significantly advanced with large-scale harmonization dataset.
1 code implementation • 30 Sep 2022 • Jing Liang, Li Niu, Penghao Wu, Fengjun Guo, Teng Long
Inharmonious region localization aims to localize the region in a synthetic image which is incompatible with surrounding background.
1 code implementation • 1 Jun 2022 • Haoxu Huang, Li Niu
Image harmonization targets at adjusting the foreground in a composite image to make it compatible with the background, producing a more realistic and harmonious image.
1 code implementation • ICCV 2023 • Li Niu, Linfeng Tan, Xinhao Tao, Junyan Cao, Fengjun Guo, Teng Long, Liqing Zhang
Given a composite image, image harmonization aims to adjust the foreground illumination to be consistent with background.
1 code implementation • 30 Jun 2023 • Xinhao Tao, Junyan Cao, Yan Hong, Li Niu
Specifically, in the first stage, we decompose shadow mask prediction into box prediction and shape prediction.
1 code implementation • 11 May 2023 • Yuxuan Duan, Li Niu, Yan Hong, Liqing Zhang
In this work, we introduce WeditGAN, which realizes model transfer by editing the intermediate latent codes $w$ in StyleGANs with learned constant offsets ($\Delta w$), discovering and constructing target latent spaces via simply relocating the distribution of source latent spaces.
1 code implementation • 20 Mar 2024 • Junjie Chen, Jiebin Yan, Yuming Fang, Li Niu
Existing methods only rely on the features extracted at support keypoints to predict or refine the keypoints on query image, but a few support feature vectors are local and inadequate for CAPE.
1 code implementation • 15 Dec 2023 • Li Niu, Yan Hong, Junyan Cao, Liqing Zhang
Painterly image harmonization aims to harmonize a photographic foreground object on the painterly background.
no code implementations • CVPR 2018 • Jiuxiang Gu, Jianfei Cai, Shafiq Joty, Li Niu, Gang Wang
Textual-visual cross-modal retrieval has been a hot research topic in both computer vision and natural language processing communities.
no code implementations • CVPR 2018 • Li Niu, Qingtao Tang, Ashok Veeraraghavan, Ashu Sabharwal
As tons of photos are being uploaded to public websites (e. g., Flickr, Bing, and Google) every day, learning from web data has become an increasingly popular research direction because of freely available web resources, which is also referred to as webly supervised learning.
no code implementations • 16 Nov 2017 • Li Niu, Jianfei Cai, Ashok Veeraraghavan
Zero-Shot Learning (ZSL) aims to classify a test instance from an unseen category based on the training instances from seen categories, in which the gap between seen categories and unseen categories is generally bridged via visual-semantic mapping between the low-level visual feature space and the intermediate semantic space.
no code implementations • 19 Nov 2018 • Li Niu, Ashok Veeraraghavan, Ashu Sabharwal
In the extreme case, given a set of test categories without any well-labeled training data, the majority of existing works can be grouped into the following two research directions: 1) crawl noisy labeled web data for the test categories as training data, which is dubbed as webly supervised learning; 2) transfer the knowledge from auxiliary categories with well-labeled training data to the test categories, which corresponds to zero-shot learning setting.
no code implementations • CVPR 2018 • Li Niu, Ashok Veeraraghavan, Ashutosh Sabharwal
The drawbacks of the above two directions motivate us to design a new framework which can jointly leverage both web data and auxiliary labeled categories to predict the test categories that are not associated with any well-labeled training images.
no code implementations • CVPR 2015 • Li Niu, Wen Li, Dong Xu
In this work, we formulate a new weakly supervised domain generalization problem for the visual recognition task by using loosely labeled web images/videos as training data.
no code implementations • ICCV 2015 • Li Niu, Wen Li, Dong Xu
Considering the recent works show the domain generalization capability can be enhanced by fusing multiple SVM classifiers, we build upon exemplar SVMs to learn a set of SVM classifiers by using one positive sample and all negative samples in the source domain each time.
no code implementations • CVPR 2020 • Yi Tu, Li Niu, Junjie Chen, Dawei Cheng, Liqing Zhang
However, crawled web images usually have two types of noises, label noise and background noise, which induce extra difficulties in utilizing them effectively.
Ranked #14 on Image Classification on WebVision-1000
no code implementations • 24 Nov 2019 • Ruicong Xu, Li Niu, Jianfu Zhang, Liqing Zhang
Activity image-to-video retrieval task aims to retrieve videos containing the similar activity as the query image, which is a challenging task because videos generally have many background segments irrelevant to the activity.
no code implementations • 24 Nov 2019 • Yi Tu, Li Niu, Weijie Zhao, Dawei Cheng, Liqing Zhang
Aesthetic image cropping is a practical but challenging task which aims at finding the best crops with the highest aesthetic quality in an image.
no code implementations • 29 Nov 2019 • Jiangtong Li, Zhixin Ling, Li Niu, Liqing Zhang
The goal of Sketch-Based Image Retrieval (SBIR) is using free-hand sketches to retrieve images of the same category from a natural image gallery.
no code implementations • 1 Dec 2019 • Yiyi Zhang, Li Niu, Ziqi Pan, Meichao Luo, Jianfu Zhang, Dawei Cheng, Liqing Zhang
Specifically, the VRE module includes a proxy task which imposes pseudo motion label constraint and temporal coherence constraint on unlabeled videos, while the MRA module could predict the motion information of a static action image by exploiting unlabeled videos.
no code implementations • 15 Mar 2020 • Yan Hong, Li Niu, Jianfu Zhang, Liqing Zhang
To address these issues, we propose our MTL with Selective Augmentation (MTL-SA) method to select the training samples in unlabeled datasets with confident pseudo labels and close data distribution to the labeled dataset.
no code implementations • 4 Oct 2021 • Siyuan Zhou, Li Niu, Jianlou Si, Chen Qian, Liqing Zhang
As a result, we find that pixel-level annotation of base categories can facilitate affinity learning and propagation, leading to higher-quality CAMs of novel categories.
no code implementations • ICCV 2021 • Wentao Wang, Jianfu Zhang, Li Niu, Haoyu Ling, Xue Yang, Liqing Zhang
Conventional deep image inpainting methods are based on auto-encoder architecture, in which the spatial details of images will be lost in the down-sampling process, leading to the degradation of generated results.
no code implementations • CVPR 2022 • Wentao Wang, Li Niu, Jianfu Zhang, Xue Yang, Liqing Zhang
Different from feed-forward methods, they seek for a closest latent code to the corrupted image and feed it to a pretrained generator.
no code implementations • 6 Jul 2022 • Bo Zhang, Yue Liu, Kaixin Lu, Li Niu, Liqing Zhang
Instead, we propose a novel correspondence learning network (CorrelNet) to model the correspondence between foreground and background using cross-attention maps, based on which we can predict the target coordinate that each source coordinate of foreground should be mapped to on the background.
no code implementations • 22 Jul 2022 • Yan Hong, Li Niu, Jianfu Zhang, Liqing Zhang
Few-shot image translation disentangles an image into style vector and content map.
no code implementations • 5 Oct 2022 • Penghao Wu, Li Niu, Jing Liang, Liqing Zhang
Synthetic images created by image editing operations are prevalent, but the color or illumination inconsistency between the manipulated region and background may make it unrealistic.
no code implementations • 5 Oct 2022 • Penghao Wu, Li Niu, Liqing Zhang
Based on the extracted style features, we also propose a novel style voting module to guide the localization of inharmonious region.
no code implementations • 6 Dec 2022 • Siyuan Zhou, Chunru Zhan, Biao Wang, Tiezheng Ge, Yuning Jiang, Li Niu
Given a video and a target image of interest, our objective is to simultaneously segment and track all objects in the video that are relevant to the target image.
no code implementations • CVPR 2023 • Chao Wang, Li Niu, Bo Zhang, Liqing Zhang
To address the first issue, we propose spatial-aware feature to encode the spatial relationship between candidate crops and aesthetic elements, by feeding the concatenation of crop mask and selectively aggregated feature maps to a light-weighted encoder.
no code implementations • 4 Aug 2023 • Jieteng Yao, Junjie Chen, Li Niu, Bin Sheng
Affordance learning considers the interaction opportunities for an actor in the scene and thus has wide application in scene understanding and intelligent robotics.
no code implementations • ICCV 2023 • Li Niu, Xing Zhao, Bo Zhang, Liqing Zhang
Visible watermark removal aims to erase the watermark from watermarked image and recover the background image, which is a challenging task due to the diverse watermarks.
no code implementations • ICCV 2023 • Jiangtong Li, Li Niu, Liqing Zhang
To tackle the challenge that the confounder in VideoQA is unobserved and non-enumerable in general, we propose a model-agnostic framework called Knowledge Proxy Intervention (KPI), which introduces an extra knowledge proxy variable in the causal graph to cut the backdoor path and remove the confounder.
no code implementations • 27 Sep 2023 • Lingxiao Lu, Jiangtong Li, Bo Zhang, Li Niu
The goal of image composition is merging a foreground object into a background image to obtain a realistic composite image.
no code implementations • 26 Oct 2023 • Junhong Gou, Bo Zhang, Li Niu, Jianfu Zhang, Jianlou Si, Chen Qian, Liqing Zhang
Specifically, our approach learns the human body priors and hallucinates the target locations of specified foreground keypoints in the background.
no code implementations • 15 Nov 2023 • Xudong Wang, Li Niu, Junyan Cao, Yan Hong, Liqing Zhang
In this work, we employ adversarial learning to bridge the domain gap between foreground feature map and background feature map.
1 code implementation • 15 Dec 2023 • Li Niu, Junyan Cao, Yan Hong, Liqing Zhang
In particular, we learn a mapping from background style and object information to object style based on painterly objects in artistic paintings.
no code implementations • 2 Feb 2024 • Jiaxuan Chen, Jieteng Yao, Li Niu
The recent development of generative models unleashes the potential of generating hyper-realistic fake images.