3 code implementations • 12 Jul 2017 • Sungrae Park, Jun-Keon Park, Su-Jin Shin, Il-Chul Moon
Recently, the training with adversarial examples, which are generated by adding a small but worst-case perturbation on input examples, has been proved to improve generalization performance of neural networks.
1 code implementation • ICLR 2019 • Weonyoung Joo, Wonsung Lee, Sungrae Park, Il-Chul Moon
The experimental results show that 1) DirVAE models the latent representation result with the best log-likelihood compared to the baselines; and 2) DirVAE produces more interpretable latent values with no collapsing issues which the baseline models suffer from.
no code implementations • ICLR 2019 • Su-Jin Shin, Kyungwoo Song, Il-Chul Moon
The joint optimization of representation learning and clustering in the embedding space has experienced a breakthrough in recent years.
2 code implementations • 22 Apr 2019 • Sungrae Park, Kyungwoo Song, Mingi Ji, Wonsung Lee, Il-Chul Moon
Successful application processing sequential data, such as text and speech, requires an improved generalization performance of recurrent neural networks (RNNs).
1 code implementation • 26 Apr 2019 • Kyungwoo Song, Wonsung Lee, Il-Chul Moon
Understanding politics is challenging because the politics take the influence from everything.
1 code implementation • 26 Apr 2019 • Kyungwoo Song, Mingi Ji, Sungrae Park, Il-Chul Moon
The analyses on the user history require the robust sequential model to anticipate the transitions and the decays of user interests.
1 code implementation • 25 May 2019 • Kyungwoo Song, JoonHo Jang, Seung jae Shin, Il-Chul Moon
Long Short-Term Memory (LSTM) infers the long term dependency through a cell state maintained by the input and the forget gate structures, which models a gate output as a value in [0, 1] through a sigmoid function.
no code implementations • 9 Aug 2019 • Dongjun Kim, Tae-Sub Yun, Il-Chul Moon
While this parameter calibration has been fixed throughout a simulation execution, this paper expands the static parameter calibration in two dimensions: dynamic calibration and heterogeneous calibration.
no code implementations • 15 Nov 2019 • Mingi Ji, Weonyoung Joo, Kyungwoo Song, Yoon-Yeong Kim, Il-Chul Moon
This work merges the self-attention of the Transformer and the sequential recommendation by adding a probabilistic model of the recommendation task specifics.
no code implementations • 4 Mar 2020 • Weonyoung Joo, Dongjun Kim, Seungjae Shin, Il-Chul Moon
Stochastic gradient estimators of discrete random variables are widely explored, for example, Gumbel-Softmax reparameterization trick for Bernoulli and categorical distributions.
no code implementations • 7 Apr 2020 • Seungjae Shin, Kyungwoo Song, JoonHo Jang, Hyemi Kim, Weonyoung Joo, Il-Chul Moon
Recent research demonstrates that word embeddings, trained on the human-generated corpus, have strong gender biases in embedding spaces, and these biases can result in the discriminative results from the various downstream tasks.
no code implementations • 13 Apr 2020 • Dongjun Kim, Weonyoung Joo, Seungjae Shin, Kyungwoo Song, Il-Chul Moon
Generative Adversarial Network (GAN) can be viewed as an implicit estimator of a data distribution, and this perspective motivates using the adversarial concept in the true input parameter estimation of black-box generators.
no code implementations • 11 Jun 2020 • Kyungwoo Song, Yohan Jung, Dongjun Kim, Il-Chul Moon
For the attention in Transformer and GAT, we derive that the attention is a product of two parts: 1) the RBF kernel to measure the similarity of two instances and 2) the exponential of $L^{2}$ norm to compute the importance of individual instances.
1 code implementation • 15 Oct 2020 • Dongjun Kim, Kyungwoo Song, YoonYeong Kim, Yongjin Shin, Wanmo Kang, Il-Chul Moon, Weonyoung Joo
This paper introduces a new sampling approach, called Neural Proposal (NP), of the simulation input that resolves the biased data collection as it guarantees the i. i. d.
no code implementations • Findings of the Association for Computational Linguistics 2020 • Seungjae Shin, Kyungwoo Song, JoonHo Jang, Hyemi Kim, Weonyoung Joo, Il-Chul Moon
Recent research demonstrates that word embeddings, trained on the human-generated corpus, have strong gender biases in embedding spaces, and these biases can result in the discriminative results from the various downstream tasks.
no code implementations • NeurIPS 2021 • Yoon-Yeong Kim, Kyungwoo Song, JoonHo Jang, Il-Chul Moon
Active learning effectively collects data instances for training deep learning models when the labeled dataset is limited and the annotation cost is high.
no code implementations • 24 Nov 2020 • Hyemi Kim, Seungjae Shin, JoonHo Jang, Kyungwoo Song, Weonyoung Joo, Wanmo Kang, Il-Chul Moon
Therefore, this paper proposes Disentangled Causal Effect Variational Autoencoder (DCEVAE) to resolve this limitation by disentangling the exogenous uncertainty into two latent variables: either 1) independent to interventions or 2) correlated to interventions without causality.
1 code implementation • 15 Feb 2021 • Dongjun Kim, Kyungwoo Song, Seungjae Shin, Wanmo Kang, Il-Chul Moon, Weonyoung Joo
A simulation is useful when the phenomenon of interest is either expensive to regenerate or irreproducible with the same context.
1 code implementation • CVPR 2021 • Mingi Ji, Seungjae Shin, Seunghyun Hwang, Gibeom Park, Il-Chul Moon
Knowledge distillation is a method of transferring the knowledge from a pretrained complex teacher model to a student model, so a smaller network can replace a large teacher network at the deployment stage.
1 code implementation • NeurIPS 2021 • Yooon-Yeong Kim, Kyungwoo Song, JoonHo Jang, Il-Chul Moon
Active learning effectively collects data instances for training deep learning models when the labeled dataset is limited and the annotation cost is high.
1 code implementation • 10 Jun 2021 • Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, Il-Chul Moon
This paper investigates with sufficient empirical evidence that such inverse correlation happens because density estimation is significantly contributed by small diffusion time, whereas sample generation mainly depends on large diffusion time.
Ranked #2 on Image Generation on CIFAR-10 (Inception score metric)
no code implementations • 29 Sep 2021 • Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, Il-Chul Moon
From the theory side, the difficulty arises in estimating the high precision diffusion because the data score goes to $\infty$ as $t \rightarrow 0$ of the diffusion time.
no code implementations • 29 Sep 2021 • Dongjun Kim, Byeonghu Na, Se Jung Kwon, Dongsoo Lee, Wanmo Kang, Il-Chul Moon
Specifically, PDM utilizes the flow to non-linearly transform a data variable into a latent variable, and PDM applies the diffusion process to the transformed latent distribution with the linear diffusing mechanism.
no code implementations • 7 Mar 2022 • Dongjun Kim, Tae-Sub Yun, Il-Chul Moon, Jang Won Bae
Agent-based models (ABMs) highlight the importance of simulation validation, such as qualitative face validation and quantitative empirical validation.
2 code implementations • 2 May 2022 • HeeSun Bae, Seungjae Shin, Byeonghu Na, JoonHo Jang, Kyungwoo Song, Il-Chul Moon
We suggest a new branch of method, Noisy Prediction Calibration (NPC) in learning with noisy labels.
1 code implementation • 27 May 2022 • Dongjun Kim, Byeonghu Na, Se Jung Kwon, Dongsoo Lee, Wanmo Kang, Il-Chul Moon
Whereas diverse variations of diffusion models exist, extending the linear diffusion into a nonlinear diffusion process is investigated by very few works.
Ranked #4 on Image Generation on CelebA 64x64
1 code implementation • 15 Jun 2022 • JoonHo Jang, Byeonghu Na, DongHyeok Shin, Mingi Ji, Kyungwoo Song, Il-Chul Moon
Therefore, we propose Unknown-Aware Domain Adversarial Learning (UADAL), which $\textit{aligns}$ the source and the target-$\textit{known}$ distribution while simultaneously $\textit{segregating}$ the target-$\textit{unknown}$ distribution in the feature alignment procedure.
2 code implementations • 28 Nov 2022 • Dongjun Kim, Yeongmin Kim, Se Jung Kwon, Wanmo Kang, Il-Chul Moon
In sample generation, we add an auxiliary term to the pre-trained score to deceive the discriminator.
Ranked #1 on Conditional Image Generation on CIFAR-10
1 code implementation • Proceedings of the 40th International Conference on Machine Learning 2023 • Yoon-Yeong Kim, Youngjae Cho, JoonHo Jang, Byeonghu Na, Yeongmin Kim, Kyungwoo Song, Wanmo Kang, Il-Chul Moon
Specifically, our proposed method, Sharpness-Aware Active Learning (SAAL), constructs its acquisition function by selecting unlabeled instances whose perturbed loss becomes maximum.
1 code implementation • 8 Mar 2023 • Seungjae Shin, HeeSun Bae, DongHyeok Shin, Weonyoung Joo, Il-Chul Moon
Training neural networks on a large dataset requires substantial computational costs.
1 code implementation • NeurIPS 2023 • DongHyeok Shin, Seungjae Shin, Il-Chul Moon
This paper presents FreD, a novel parameterization method for dataset distillation, which utilizes the frequency domain to distill a small-sized synthetic dataset from a large-sized original dataset.
1 code implementation • 9 Jan 2024 • Youngjae Cho, HeeSun Bae, Seungjae Shin, Yeo Dong Youn, Weonyoung Joo, Il-Chul Moon
This paper presents a Bayesian-based framework of prompt learning, which could alleviate the overfitting issues on few-shot learning application and increase the adaptability of prompts on unseen instances.
1 code implementation • 27 Feb 2024 • Byeonghu Na, Yeongmin Kim, HeeSun Bae, Jung Hyun Lee, Se Jung Kwon, Wanmo Kang, Il-Chul Moon
This paper proposes Transition-aware weighted Denoising Score Matching (TDSM) for training conditional diffusion models with noisy labels, which is the first study in the line of diffusion models.
1 code implementation • 2 Mar 2024 • Yeongmin Kim, Byeonghu Na, Minsang Park, JoonHo Jang, Dongjun Kim, Wanmo Kang, Il-Chul Moon
While directly applying it to score-matching is intractable, we discover that using the time-dependent density ratio both for reweighting and score correction can lead to a tractable form of the objective function to regenerate the unbiased data density.
1 code implementation • 2 Mar 2024 • Hyungho Na, Yunkyeong Seo, Il-Chul Moon
To address this, we introduce Efficient episodic Memory Utilization (EMU) for MARL, with two primary objectives: (a) accelerating reinforcement learning by leveraging semantically coherent memory from an episodic buffer and (b) selectively promoting desirable transitions to prevent local convergence.
1 code implementation • 5 Mar 2024 • HeeSun Bae, Seungjae Shin, Byeonghu Na, Il-Chul Moon
We propose good utilization of the transition matrix is crucial and suggest a new utilization method based on resampling, coined RENT.
no code implementations • 12 Mar 2024 • Seungjae Shin, HeeSun Bae, Byeonghu Na, Yoon-Yeong Kim, Il-Chul Moon
In particular, by aligning the loss landscape acquired in the source domain to the loss landscape of perturbed domains, we expect to achieve generalization grounded on these flat minima for the unknown domains.