Search Results for author: Wanmo Kang

Found 13 papers, 9 papers with code

Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models

2 code implementations ICLR 2020 Cheolhyoung Lee, Kyunghyun Cho, Wanmo Kang

We empirically evaluate the proposed mixout and its variants on finetuning a pretrained language model on downstream tasks.

Language Modelling

Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation

1 code implementation10 Jun 2021 Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, Il-Chul Moon

This paper investigates with sufficient empirical evidence that such inverse correlation happens because density estimation is significantly contributed by small diffusion time, whereas sample generation mainly depends on large diffusion time.

Ranked #2 on Image Generation on CIFAR-10 (Inception score metric)

Density Estimation Image Generation

Maximum Likelihood Training of Implicit Nonlinear Diffusion Models

1 code implementation27 May 2022 Dongjun Kim, Byeonghu Na, Se Jung Kwon, Dongsoo Lee, Wanmo Kang, Il-Chul Moon

Whereas diverse variations of diffusion models exist, extending the linear diffusion into a nonlinear diffusion process is investigated by very few works.

Image Generation

Label-Noise Robust Diffusion Models

1 code implementation27 Feb 2024 Byeonghu Na, Yeongmin Kim, HeeSun Bae, Jung Hyun Lee, Se Jung Kwon, Wanmo Kang, Il-Chul Moon

This paper proposes Transition-aware weighted Denoising Score Matching (TDSM) for training conditional diffusion models with noisy labels, which is the first study in the line of diffusion models.

Denoising

SAAL: Sharpness-Aware Active Learning

1 code implementation Proceedings of the 40th International Conference on Machine Learning 2023 Yoon-Yeong Kim, Youngjae Cho, JoonHo Jang, Byeonghu Na, Yeongmin Kim, Kyungwoo Song, Wanmo Kang, Il-Chul Moon

Specifically, our proposed method, Sharpness-Aware Active Learning (SAAL), constructs its acquisition function by selecting unlabeled instances whose perturbed loss becomes maximum.

Active Learning Image Classification +3

Training Unbiased Diffusion Models From Biased Dataset

1 code implementation2 Mar 2024 Yeongmin Kim, Byeonghu Na, Minsang Park, JoonHo Jang, Dongjun Kim, Wanmo Kang, Il-Chul Moon

While directly applying it to score-matching is intractable, we discover that using the time-dependent density ratio both for reweighting and score correction can lead to a tractable form of the objective function to regenerate the unbiased data density.

Sequential Likelihood-Free Inference with Neural Proposal

1 code implementation15 Oct 2020 Dongjun Kim, Kyungwoo Song, YoonYeong Kim, Yongjin Shin, Wanmo Kang, Il-Chul Moon, Weonyoung Joo

This paper introduces a new sampling approach, called Neural Proposal (NP), of the simulation input that resolves the biased data collection as it guarantees the i. i. d.

Bayesian Inference

Neural Posterior Regularization for Likelihood-Free Inference

1 code implementation15 Feb 2021 Dongjun Kim, Kyungwoo Song, Seungjae Shin, Wanmo Kang, Il-Chul Moon, Weonyoung Joo

A simulation is useful when the phenomenon of interest is either expensive to regenerate or irreproducible with the same context.

Bayesian Inference

Directional Analysis of Stochastic Gradient Descent via von Mises-Fisher Distributions in Deep learning

no code implementations ICLR 2019 Cheolhyoung Lee, Kyunghyun Cho, Wanmo Kang

We empirically verify our result using deep convolutional networks and observe a higher correlation between the gradient stochasticity and the proposed directional uniformity than that against the gradient norm stochasticity, suggesting that the directional statistics of minibatch gradients is a major factor behind SGD.

Counterfactual Fairness with Disentangled Causal Effect Variational Autoencoder

no code implementations24 Nov 2020 Hyemi Kim, Seungjae Shin, JoonHo Jang, Kyungwoo Song, Weonyoung Joo, Wanmo Kang, Il-Chul Moon

Therefore, this paper proposes Disentangled Causal Effect Variational Autoencoder (DCEVAE) to resolve this limitation by disentangling the exogenous uncertainty into two latent variables: either 1) independent to interventions or 2) correlated to interventions without causality.

Attribute Causal Inference +3

High Precision Score-based Diffusion Models

no code implementations29 Sep 2021 Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, Il-Chul Moon

From the theory side, the difficulty arises in estimating the high precision diffusion because the data score goes to $\infty$ as $t \rightarrow 0$ of the diffusion time.

Image Generation Vocal Bursts Intensity Prediction

Maximum Likelihood Training of Parametrized Diffusion Model

no code implementations29 Sep 2021 Dongjun Kim, Byeonghu Na, Se Jung Kwon, Dongsoo Lee, Wanmo Kang, Il-Chul Moon

Specifically, PDM utilizes the flow to non-linearly transform a data variable into a latent variable, and PDM applies the diffusion process to the transformed latent distribution with the linear diffusing mechanism.

Image Generation

Cannot find the paper you are looking for? You can Submit a new open access paper.