Search Results for author: Zhifei Zhang

Found 38 papers, 14 papers with code

SwapAnything: Enabling Arbitrary Object Swapping in Personalized Visual Editing

no code implementations8 Apr 2024 Jing Gu, Yilin Wang, Nanxuan Zhao, Wei Xiong, Qing Liu, Zhifei Zhang, He Zhang, Jianming Zhang, HyunJoon Jung, Xin Eric Wang

Compared with existing methods for personalized subject swapping, SwapAnything has three unique advantages: (1) precise control of arbitrary objects and parts rather than the main subject, (2) more faithful preservation of context pixels, (3) better adaptation of the personalized concept to the image.

Image Generation Object

PatSTEG: Modeling Formation Dynamics of Patent Citation Networks via The Semantic-Topological Evolutionary Graph

no code implementations3 Feb 2024 Ran Miao, Xueyu Chen, Liang Hu, Zhifei Zhang, Minghua Wan, Qi Zhang, Cairong Zhao

Patent documents in the patent database (PatDB) are crucial for research, development, and innovation as they contain valuable technical information.

Graph Learning

Improving Diffusion Models for Scene Text Editing with Dual Encoders

1 code implementation12 Apr 2023 Jiabao Ji, Guanhua Zhang, Zhaowen Wang, Bairu Hou, Zhifei Zhang, Brian Price, Shiyu Chang

Scene text editing is a challenging task that involves modifying or inserting specified texts in an image while maintaining its natural and realistic appearance.

Scene Text Editing Style Transfer +1

TopNet: Transformer-based Object Placement Network for Image Compositing

no code implementations CVPR 2023 Sijie Zhu, Zhe Lin, Scott Cohen, Jason Kuen, Zhifei Zhang, Chen Chen

Given a background image and a segmented object, the goal is to train a model to predict plausible placements (location and scale) of the object for compositing.


Modulating Pretrained Diffusion Models for Multimodal Image Synthesis

no code implementations24 Feb 2023 Cusuh Ham, James Hays, Jingwan Lu, Krishna Kumar Singh, Zhifei Zhang, Tobias Hinz

We show that MCM enables user control over the spatial layout of the image and leads to increased control over the image generation process.

Image Generation Semantic Segmentation

Counterfactual-based Saliency Map: Towards Visual Contrastive Explanations for Neural Networks

no code implementations ICCV 2023 Xue Wang, Zhibo Wang, Haiqin Weng, Hengchang Guo, Zhifei Zhang, Lu Jin, Tao Wei, Kui Ren

Considering the insufficient study on such complex causal questions, we make the first attempt to explain different causal questions by contrastive explanations in a unified framework, ie., Counterfactual Contrastive Explanation (CCE), which visually and intuitively explains the aforementioned questions via a novel positive-negative saliency-based explanation scheme.


Joint Implicit Neural Representation for High-fidelity and Compact Vector Fonts

no code implementations ICCV 2023 Chia-Hao Chen, Ying-Tian Liu, Zhifei Zhang, Yuan-Chen Guo, Song-Hai Zhang

Existing vector font generation approaches either struggle to preserve high-frequency corner details of the glyph or produce vector shapes that have redundant segments, which hinders their applications in practical scenarios.

Font Generation

ObjectStitch: Object Compositing With Diffusion Model

no code implementations CVPR 2023 Yizhi Song, Zhifei Zhang, Zhe Lin, Scott Cohen, Brian Price, Jianming Zhang, Soo Ye Kim, Daniel Aliaga

Object compositing based on 2D images is a challenging problem since it typically involves multiple processing stages such as color harmonization, geometry correction and shadow generation to generate realistic results.

Data Augmentation Object

Towards Transferable Targeted Adversarial Examples

1 code implementation CVPR 2023 Zhibo Wang, Hongshan Yang, Yunhe Feng, Peng Sun, Hengchang Guo, Zhifei Zhang, Kui Ren

In this paper, we propose the Transferable Targeted Adversarial Attack (TTAA), which can capture the distribution information of the target class from both label-wise and feature-wise perspectives, to generate highly transferable targeted adversarial examples.

Adversarial Attack

SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model

no code implementations CVPR 2023 Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, Kun Zhang

By contrast, multi-modal inpainting provides more flexible and useful controls on the inpainted content, \eg, a text prompt can be used to describe an object with richer attributes, and a mask can be used to constrain the shape of the inpainted object rather than being only considered as a missing area.

Image Inpainting Object +1

ObjectStitch: Generative Object Compositing

1 code implementation2 Dec 2022 Yizhi Song, Zhifei Zhang, Zhe Lin, Scott Cohen, Brian Price, Jianming Zhang, Soo Ye Kim, Daniel Aliaga

Object compositing based on 2D images is a challenging problem since it typically involves multiple processing stages such as color harmonization, geometry correction and shadow generation to generate realistic results.

Data Augmentation Object

Toward Understanding WordArt: Corner-Guided Transformer for Scene Text Recognition

1 code implementation31 Jul 2022 Xudong Xie, Ling Fu, Zhifei Zhang, Zhaowen Wang, Xiang Bai

Thirdly, we utilize Transformer to learn the global feature on image-level and model the global relationship of the corner points, with the assistance of a corner-query cross-attention mechanism.

Scene Text Recognition

SImProv: Scalable Image Provenance Framework for Robust Content Attribution

no code implementations28 Jun 2022 Alexander Black, Tu Bui, Simon Jenni, Zhifei Zhang, Viswanathan Swaminanthan, John Collomosse

We present SImProv - a scalable image provenance framework to match a query image back to a trusted database of originals and identify possible manipulations on the query.

Re-Ranking Retrieval

Vanilla Feature Distillation for Improving the Accuracy-Robustness Trade-Off in Adversarial Training

no code implementations5 Jun 2022 Guodong Cao, Zhibo Wang, Xiaowei Dong, Zhifei Zhang, Hengchang Guo, Zhan Qin, Kui Ren

However, most existing works are still trapped in the dilemma between higher accuracy and stronger robustness since they tend to fit a model towards robust features (not easily tampered with by adversaries) while ignoring those non-robust but highly predictive features.

Knowledge Distillation

GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing

no code implementations31 Mar 2022 Sijie Zhu, Zhe Lin, Scott Cohen, Jason Kuen, Zhifei Zhang, Chen Chen

To move a step further, this paper proposes GALA (Geometry-and-Lighting-Aware), a generic foreground object search method with discriminative modeling on geometry and lighting compatibility for open-world image compositing.


Fairness-aware Adversarial Perturbation Towards Bias Mitigation for Deployed Deep Models

no code implementations CVPR 2022 Zhibo Wang, Xiaowei Dong, Henry Xue, Zhifei Zhang, Weifeng Chiu, Tao Wei, Kui Ren

Prioritizing fairness is of central importance in artificial intelligence (AI) systems, especially for those societal applications, e. g., hiring systems should recommend applicants equally from different demographic groups, and risk assessment systems must eliminate racism in criminal justice.


Font Completion and Manipulation by Cycling Between Multi-Modality Representations

1 code implementation30 Aug 2021 Ye Yuan, Wuyang Chen, Zhaowen Wang, Matthew Fisher, Zhifei Zhang, Zhangyang Wang, Hailin Jin

The novel graph constructor maps a glyph's latent code to its graph representation that matches expert knowledge, which is trained to help the translation task.

Image-to-Image Translation Representation Learning +2

Feature Importance-aware Transferable Adversarial Attacks

3 code implementations ICCV 2021 Zhibo Wang, Hengchang Guo, Zhifei Zhang, Wenxin Liu, Zhan Qin, Kui Ren

More specifically, we obtain feature importance by introducing the aggregate gradient, which averages the gradients with respect to feature maps of the source model, computed on a batch of random transforms of the original clean image.

Feature Importance

Rethinking Text Segmentation: A Novel Dataset and A Text-Specific Refinement Approach

1 code implementation CVPR 2021 Xingqian Xu, Zhifei Zhang, Zhaowen Wang, Brian Price, Zhonghao Wang, Humphrey Shi

We also introduce Text Refinement Network (TexRNet), a novel text segmentation approach that adapts to the unique properties of text, e. g. non-convex boundary, diverse texture, etc., which often impose burdens on traditional segmentation models.

Segmentation Style Transfer +2

Texture Hallucination for Large-Factor Painting Super-Resolution

no code implementations ECCV 2020 Yulun Zhang, Zhifei Zhang, Stephen DiVerdi, Zhaowen Wang, Jose Echevarria, Yun Fu

We aim to super-resolve digital paintings, synthesizing realistic details from high-resolution reference painting materials for very large scaling factors (e. g., 8X, 16X).

4k Hallucination +2

A Semantic-based Medical Image Fusion Approach

no code implementations1 Jun 2019 Fanda Fan, Yunyou Huang, Lei Wang, Xingwang Xiong, Zihan Jiang, Zhifei Zhang, Jianfeng Zhan

Medical image fusion is a promising approach to providing overall information from medical images of different modalities.

Image Super-Resolution by Neural Texture Transfer

2 code implementations CVPR 2019 Zhifei Zhang, Zhaowen Wang, Zhe Lin, Hairong Qi

Reference-based super-resolution (RefSR), on the other hand, has proven to be promising in recovering high-resolution (HR) details when a reference (Ref) image with similar content as that of the LR input is given.

Image Stylization Image Super-Resolution +1

Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning

1 code implementation3 Dec 2018 Zhibo Wang, Mengkai Song, Zhifei Zhang, Yang song, Qian Wang, Hairong Qi

Although the state-of-the-art attacking techniques that incorporated the advance of Generative adversarial networks (GANs) could construct class representatives of the global data distribution among all clients, it is still challenging to distinguishably attack a specific client (i. e., user-level privacy leakage), which is a stronger privacy threat to precisely recover the private data from a specific client.

Edge-computing Federated Learning +1

Reference-Conditioned Super-Resolution by Neural Texture Transfer

no code implementations10 Apr 2018 Zhifei Zhang, Zhaowen Wang, Zhe Lin, Hairong Qi

We focus on transferring the high-resolution texture from reference images to the super-resolution process without the constraint of content similarity between reference and target images, which is a key difference from previous example-based methods.

Image Stylization Image Super-Resolution +1

Decoupled Learning for Conditional Adversarial Networks

1 code implementation21 Jan 2018 Zhifei Zhang, Yang song, Hairong Qi

Incorporating encoding-decoding nets with adversarial nets has been widely adopted in image generation tasks.

Image Generation

r-BTN: Cross-domain Face Composite and Synthesis from Limited Facial Patches

no code implementations2 Jun 2017 Yang Song, Zhifei Zhang, Hairong Qi

A more generalized question is that if a large proportion (e. g., more than 50%) of the face/sketch is missing, can a realistic whole face sketch/image still be estimated.

Facial Inpainting Transfer Learning

Age Progression/Regression by Conditional Adversarial Autoencoder

5 code implementations CVPR 2017 Zhifei Zhang, Yang song, Hairong Qi

In CAAE, the face is first mapped to a latent vector through a convolutional encoder, and then the vector is projected to the face manifold conditional on age through a deconvolutional generator.

Attribute regression

Derivative Delay Embedding: Online Modeling of Streaming Time Series

1 code implementation24 Sep 2016 Zhifei Zhang, Yang song, Wei Wang, Hairong Qi

The staggering amount of streaming time series coming from the real world calls for more efficient and effective online modeling solution.

General Classification Time Series +1

Cannot find the paper you are looking for? You can Submit a new open access paper.