Search Results for author: Seanie Lee

Found 19 papers, 11 papers with code

Self-Supervised Dataset Distillation for Transfer Learning

2 code implementations10 Oct 2023 Dong Bok Lee, Seanie Lee, Joonho Ko, Kenji Kawaguchi, Juho Lee, Sung Ju Hwang

To achieve this, we also introduce the MSE between representations of the inner model and the self-supervised target model on the original full dataset for outer optimization.

Bilevel Optimization Meta-Learning +3

Drug Discovery with Dynamic Goal-aware Fragments

no code implementations2 Oct 2023 Seul Lee, Seanie Lee, Kenji Kawaguchi, Sung Ju Hwang

Additionally, the existing fragment-based generative models cannot update the fragment vocabulary with goal-aware fragments newly discovered during the generation.

Drug Discovery

Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks

1 code implementation NeurIPS 2023 Minki Kang, Seanie Lee, Jinheon Baek, Kenji Kawaguchi, Sung Ju Hwang

Large Language Models (LLMs) have shown promising performance in knowledge-intensive reasoning tasks that require a compound understanding of knowledge.

Memorization StrategyQA

DiffusionNAG: Predictor-guided Neural Architecture Generation with Diffusion Models

1 code implementation26 May 2023 Sohyun An, Hayeon Lee, Jaehyeong Jo, Seanie Lee, Sung Ju Hwang

To tackle such limitations of existing NAS methods, we propose a paradigm shift from NAS to a novel conditional Neural Architecture Generation (NAG) framework based on diffusion models, dubbed DiffusionNAG.

Bayesian Optimization Neural Architecture Search +1

On Divergence Measures for Bayesian Pseudocoresets

1 code implementation12 Oct 2022 Balhae Kim, JungWon Choi, Seanie Lee, Yoonho Lee, Jung-Woo Ha, Juho Lee

Finally, we propose a novel Bayesian pseudocoreset algorithm based on minimizing forward KL divergence.

Bayesian Inference Image Classification

Self-Distillation for Further Pre-training of Transformers

no code implementations30 Sep 2022 Seanie Lee, Minki Kang, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi

Pre-training a large transformer model on a massive amount of unlabeled data and fine-tuning it on labeled datasets for diverse downstream tasks has proven to be a successful strategy, for a variety of vision and natural language processing tasks.

text-classification Text Classification

Scalable Set Encoding with Universal Mini-Batch Consistency and Unbiased Full Set Gradient Approximation

1 code implementation26 Aug 2022 Jeffrey Willette, Seanie Lee, Bruno Andreis, Kenji Kawaguchi, Juho Lee, Sung Ju Hwang

Recent work on mini-batch consistency (MBC) for set functions has brought attention to the need for sequentially processing and aggregating chunks of a partitioned set while guaranteeing the same output for all partitions.

Point Cloud Classification text-classification +1

Set-based Meta-Interpolation for Few-Task Meta-Learning

no code implementations20 May 2022 Seanie Lee, Bruno Andreis, Kenji Kawaguchi, Juho Lee, Sung Ju Hwang

Recently, several task augmentation methods have been proposed to tackle this issue using domain-specific knowledge to design augmentation techniques to densify the meta-training task distribution.

Bilevel Optimization Image Classification +6

Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning

no code implementations ICLR 2022 Seanie Lee, Hae Beom Lee, Juho Lee, Sung Ju Hwang

Thanks to the gradients aligned between tasks by our method, the model becomes less vulnerable to negative transfer and catastrophic forgetting.

Continual Learning Multi-Task Learning +1

Task Conditioned Stochastic Subsampling

no code implementations29 Sep 2021 Andreis Bruno, Seanie Lee, A. Tuan Nguyen, Juho Lee, Eunho Yang, Sung Ju Hwang

Deep Learning algorithms are designed to operate on huge volumes of high dimensional data such as images.

Image Classification Image Reconstruction

Learning to Perturb Word Embeddings for Out-of-distribution QA

1 code implementation ACL 2021 Seanie Lee, Minki Kang, Juho Lee, Sung Ju Hwang

QA models based on pretrained language mod-els have achieved remarkable performance on various benchmark datasets. However, QA models do not generalize well to unseen data that falls outside the training distribution, due to distributional shifts. Data augmentation (DA) techniques which drop/replace words have shown to be effective in regularizing the model from overfitting to the training data. Yet, they may adversely affect the QA tasks since they incur semantic changes that may lead to wrong answers for the QA task.

Data Augmentation Domain Generalization +1

Self-supervised Text-to-SQL Learning with Header Alignment Training

no code implementations11 Mar 2021 Donggyu Kim, Seanie Lee

Since we can leverage a large amount of unlabeled data without any human supervision to train a model and transfer the knowledge to target tasks, self-supervised learning is a de-facto component for the recent success of deep learning in various fields.

Self-Supervised Learning Text-To-SQL

Meta-GMVAE: Mixture of Gaussian VAE for Unsupervised Meta-Learning

1 code implementation ICLR 2021 Dong Bok Lee, Dongchan Min, Seanie Lee, Sung Ju Hwang

Then, the learned model can be used for downstream few-shot classification tasks, where we obtain task-specific parameters by performing semi-supervised EM on the latent representations of the support and query set, and predict labels of the query set by computing aggregated posteriors.

Meta-Learning Unsupervised Few-Shot Image Classification +2

Contrastive Learning with Adversarial Perturbations for Conditional Text Generation

1 code implementation ICLR 2021 Seanie Lee, Dong Bok Lee, Sung Ju Hwang

In this work, we propose to mitigate the conditional text generation problem by contrasting positive pairs with negative pairs, such that the model is exposed to various valid or incorrect perturbations of the inputs, for improved generalization.

Conditional Text Generation Contrastive Learning +5

Set Based Stochastic Subsampling

no code implementations25 Jun 2020 Bruno Andreis, Seanie Lee, A. Tuan Nguyen, Juho Lee, Eunho Yang, Sung Ju Hwang

Deep models are designed to operate on huge volumes of high dimensional data such as images.

feature selection Image Classification +2

Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs

1 code implementation ACL 2020 Dong Bok Lee, Seanie Lee, Woo Tae Jeong, Donghwan Kim, Sung Ju Hwang

We validate our Information Maximizing Hierarchical Conditional Variational AutoEncoder (Info-HCVAE) on several benchmark datasets by evaluating the performance of the QA model (BERT-base) using only the generated QA pairs (QA-based evaluation) or by using both the generated and human-labeled pairs (semi-supervised learning) for training, against state-of-the-art baseline models.

Question-Answer-Generation Question Answering +1

g2pM: A Neural Grapheme-to-Phoneme Conversion Package for Mandarin Chinese Based on a New Open Benchmark Dataset

1 code implementation7 Apr 2020 Kyubyong Park, Seanie Lee

Conversion of Chinese graphemes to phonemes (G2P) is an essential component in Mandarin Chinese Text-To-Speech (TTS) systems.

Polyphone disambiguation

Cannot find the paper you are looking for? You can Submit a new open access paper.