Search Results for author: Jiacheng Sun

Found 20 papers, 5 papers with code

SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models

1 code implementation NeurIPS 2023 Shuchen Xue, Mingyang Yi, Weijian Luo, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, Zhi-Ming Ma

Based on our analysis, we propose SA-Solver, which is an improved efficient stochastic Adams method for solving diffusion SDE to generate data with high quality.

Image Generation

Training Energy-Based Models with Diffusion Contrastive Divergences

no code implementations4 Jul 2023 Weijian Luo, Hao Jiang, Tianyang Hu, Jiacheng Sun, Zhenguo Li, Zhihua Zhang

In image generation experiments, the proposed DCD is capable of training an energy-based model for generating the Celab-A $32\times 32$ dataset, which is comparable to existing EBMs.

Image Denoising Image Generation

Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models

no code implementations NeurIPS 2023 Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, Zhihua Zhang

To demonstrate the effectiveness and universality of Diff-Instruct, we consider two scenarios: distilling pre-trained diffusion models and refining existing GAN models.

On the Generalization of Diffusion Model

no code implementations24 May 2023 Mingyang Yi, Jiacheng Sun, Zhenguo Li

To understand this contradiction, we empirically verify the difference between the sufficiently trained diffusion model and the empirical optima.

Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels

2 code implementations NeurIPS 2023 Zebin You, Yong Zhong, Fan Bao, Jiacheng Sun, Chongxuan Li, Jun Zhu

In an effort to further advance semi-supervised generative and classification tasks, we propose a simple yet effective training strategy called dual pseudo training (DPT), built upon strong semi-supervised learners and diffusion models.

Classification

Why Are Conditional Generative Models Better Than Unconditional Ones?

no code implementations1 Dec 2022 Fan Bao, Chongxuan Li, Jiacheng Sun, Jun Zhu

Extensive empirical evidence demonstrates that conditional generative models are easier to train and perform better than unconditional ones by exploiting the labels of data.

Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models

1 code implementation15 Jun 2022 Fan Bao, Chongxuan Li, Jiacheng Sun, Jun Zhu, Bo Zhang

Thus, the generation performance on a subset of timesteps is crucial, which is greatly influenced by the covariance design in DPMs.

Computational Efficiency

Layer-Parallel Training of Residual Networks with Auxiliary-Variable Networks

no code implementations10 Dec 2021 Qi Sun, Hexin Dong, Zewei Chen, Jiacheng Sun, Zhenguo Li, Bin Dong

Gradient-based methods for the distributed training of residual networks (ResNets) typically require a forward pass of the input data, followed by back-propagating the error gradient to update model parameters, which becomes time-consuming as the network goes deeper.

Data Augmentation

Rethinking Adversarial Transferability from a Data Distribution Perspective

no code implementations ICLR 2022 Yao Zhu, Jiacheng Sun, Zhenguo Li

Adversarial transferability enables attackers to generate adversarial examples from the source model to attack the target model, which has raised security concerns about the deployment of DNNs in practice.

Adversarial Attack

Nonlinear ICA Using Volume-Preserving Transformations

no code implementations ICLR 2022 Xiaojiang Yang, Yi Wang, Jiacheng Sun, Xing Zhang, Shifeng Zhang, Zhenguo Li, Junchi Yan

Nonlinear ICA is a fundamental problem in machine learning, aiming to identify the underlying independent components (sources) from data which is assumed to be a nonlinear function (mixing function) of these sources.

Layer-Parallel Training of Residual Networks with Auxiliary Variables

no code implementations NeurIPS Workshop DLDE 2021 Qi Sun, Hexin Dong, Zewei Chen, Weizhen Dian, Jiacheng Sun, Yitong Sun, Zhenguo Li, Bin Dong

Backpropagation algorithm is indispensable for training modern residual networks (ResNets) and usually tends to be time-consuming due to its inherent algorithmic lockings.

Data Augmentation

Towards Understanding the Generative Capability of Adversarially Robust Classifiers

no code implementations ICCV 2021 Yao Zhu, Jiacheng Ma, Jiacheng Sun, Zewei Chen, Rongxin Jiang, Zhenguo Li

We find that adversarial training contributes to obtaining an energy function that is flat and has low energy around the real data, which is the key for generative capability.

Image Generation

Improved OOD Generalization via Adversarial Training and Pre-training

no code implementations24 May 2021 Mingyang Yi, Lu Hou, Jiacheng Sun, Lifeng Shang, Xin Jiang, Qun Liu, Zhi-Ming Ma

In this paper, after defining OOD generalization via Wasserstein distance, we theoretically show that a model robust to input perturbation generalizes well on OOD data.

Image Classification Natural Language Understanding

SAD: Saliency Adversarial Defense without Adversarial Training

no code implementations1 Jan 2021 Yao Zhu, Jiacheng Sun, Zewei Chen, Zhenguo Li

We justify the algorithm with a linear model that the added saliency maps pull data away from its closest decision boundary.

Adversarial Defense

Batch Group Normalization

no code implementations4 Dec 2020 Xiao-Yun Zhou, Jiacheng Sun, Nanyang Ye, Xu Lan, Qijun Luo, Bo-Lin Lai, Pedro Esperanca, Guang-Zhong Yang, Zhenguo Li

Among previous normalization methods, Batch Normalization (BN) performs well at medium and large batch sizes and is with good generalizability to multiple vision tasks, while its performance degrades significantly at small batch sizes.

Few-Shot Learning Image Classification +2

A Practical Layer-Parallel Training Algorithm for Residual Networks

no code implementations3 Sep 2020 Qi Sun, Hexin Dong, Zewei Chen, Weizhen Dian, Jiacheng Sun, Yitong Sun, Zhenguo Li, Bin Dong

Gradient-based algorithms for training ResNets typically require a forward pass of the input data, followed by back-propagating the objective gradient to update parameters, which are time-consuming for deep ResNets.

Data Augmentation

New Interpretations of Normalization Methods in Deep Learning

no code implementations16 Jun 2020 Jiacheng Sun, Xiangyong Cao, Hanwen Liang, Weiran Huang, Zewei Chen, Zhenguo Li

In recent years, a variety of normalization methods have been proposed to help train neural networks, such as batch normalization (BN), layer normalization (LN), weight normalization (WN), group normalization (GN), etc.

LEMMA

DARTS+: Improved Differentiable Architecture Search with Early Stopping

no code implementations13 Sep 2019 Hanwen Liang, Shifeng Zhang, Jiacheng Sun, Xingqiu He, Weiran Huang, Kechen Zhuang, Zhenguo Li

Therefore, we propose a simple and effective algorithm, named "DARTS+", to avoid the collapse and improve the original DARTS, by "early stopping" the search procedure when meeting a certain criterion.

Cannot find the paper you are looking for? You can Submit a new open access paper.