2 code implementations • 6 Jun 2021 • Gihun Lee, Minchan Jeong, Yongjin Shin, Sangmin Bae, Se-Young Yun
In federated learning, a strong global model is collaboratively learned by aggregating clients' locally trained models.
3 code implementations • 4 Jun 2021 • Jaehoon Oh, Sangmook Kim, Se-Young Yun
Based on this observation, we propose a novel federated learning algorithm, coined FedBABU, which only updates the body of the model during federated training (i. e., the head is randomly initialized and never updated), and the head is fine-tuned for personalization during the evaluation process.
1 code implementation • ICLR 2022 • Jaehoon Oh, Sangmook Kim, Se-Young Yun
Based on this observation, we propose a novel federated learning algorithm, coined FedBABU, which only updates the body of the model during federated training (i. e., the head is randomly initialized and never updated), and the head is fine-tuned for personalization during the evaluation process.
1 code implementation • 20 Dec 2022 • Namgyu Ho, Laura Schmid, Se-Young Yun
We evaluate our method on a wide range of public models and complex tasks.
2 code implementations • 6 Feb 2024 • Jongwoo Ko, Sungnyun Kim, Tianyi Chen, Se-Young Yun
Knowledge distillation (KD) is widely used for compressing a teacher model to a smaller student model, reducing its inference cost and memory footprint while preserving model capabilities.
2 code implementations • 7 Dec 2022 • Gihun Lee, Sangmook Kim, Joonkee Kim, Se-Young Yun
Cell segmentation is a fundamental task for computational biology analysis.
1 code implementation • 13 Oct 2020 • Sungnyun Kim, Gihun Lee, Sangmin Bae, Se-Young Yun
Contrastive learning has shown remarkable results in recent self-supervised approaches for visual representation.
1 code implementation • 23 May 2023 • Sangmin Bae, June-Woo Kim, Won-Yang Cho, Hyerim Baek, Soyoun Son, Byungjo Lee, Changwan Ha, Kyongpil Tae, Sungnyun Kim, Se-Young Yun
Respiratory sound contains crucial information for the early diagnosis of fatal lung diseases.
Ranked #1 on Audio Classification on ICBHI Respiratory Sound Database (using extra training data)
1 code implementation • 9 Oct 2023 • Sangmin Bae, Jongwoo Ko, Hwanjun Song, Se-Young Yun
To tackle the high inference latency exhibited by autoregressive language models, previous studies have proposed an early-exiting framework that allocates adaptive computation paths for each token based on the complexity of generating the subsequent token.
1 code implementation • ICLR 2021 • Jaehoon Oh, Hyungjun Yoo, ChangHwan Kim, Se-Young Yun
It has recently been hypothesized that representation reuse, which makes little change in efficient representations, is the dominant factor in the performance of the meta-initialized model through MAML in contrast to representation change, which causes a significant change in representations.
1 code implementation • 19 May 2021 • Taehyeon Kim, Jaehoon Oh, Nakyil Kim, Sangwook Cho, Se-Young Yun
From this observation, we consider an intuitive KD loss function, the mean squared error (MSE) between the logit vectors, so that the student model can directly learn the logit of the teacher model.
1 code implementation • NeurIPS 2021 • Taehyeon Kim, Jongwoo Ko, Sangwook Cho, Jinhwan Choi, Se-Young Yun
Our framework, coined as filtering noisy instances via their eigenvectors (FINE), provides a robust detector with derivative-free simple methods having theoretical guarantees.
Ranked #2 on Image Classification on WebVision
1 code implementation • 19 May 2023 • Kangwook Jang, Sungnyun Kim, Se-Young Yun, Hoirin Kim
Transformer-based speech self-supervised learning (SSL) models, such as HuBERT, show surprising performance in various speech processing tasks.
1 code implementation • CVPR 2023 • Sangmook Kim, Sangmin Bae, Hwanjun Song, Se-Young Yun
In this work, we first demonstrate that the superiority of two selector models depends on the global and local inter-class diversity.
2 code implementations • 1 Feb 2022 • Jaehoon Oh, Sungnyun Kim, Namgyu Ho, Jin-Hwa Kim, Hwanjun Song, Se-Young Yun
This data enables self-supervised pre-training on the target domain, in addition to supervised pre-training on the source domain.
1 code implementation • 5 Jul 2022 • Mingyu Kim, Jihwan Oh, Yongsik Lee, Joonkee Kim, SeongHwan Kim, Song Chong, Se-Young Yun
This challenge, on the other hand, is interested in the exploration capability of MARL algorithms to efficiently learn implicit multi-stage tasks and environmental factors as well as micro-control.
Ranked #1 on SMAC+ on Off_Superhard_parallel
1 code implementation • 30 Jun 2022 • Taehyeon Kim, Namgyu Ho, Donggyu Kim, Se-Young Yun
Historically, this challenge has been tackled using numerical weather prediction (NWP) models, grounded on physics-based simulations.
1 code implementation • CVPR 2023 • Sungnyun Kim, Sangmin Bae, Se-Young Yun
Fortunately, the recent self-supervised learning (SSL) is a promising approach to pretrain a model without annotations, serving as an effective initialization for any downstream tasks.
1 code implementation • 10 Feb 2023 • Sumyeong Ahn, Jongwoo Ko, Se-Young Yun
To handle this restriction, several methods have been developed that increase the representations of minority samples by leveraging the features of the majority samples.
Ranked #13 on Long-tail Learning on CIFAR-100-LT (ρ=10)
1 code implementation • 24 Apr 2020 • Gihun Lee, Sangmin Bae, Jaehoon Oh, Se-Young Yun
With the success of deep learning in various fields and the advent of numerous Internet of Things (IoT) devices, it is essential to lighten models suitable for low-power devices.
1 code implementation • 29 Jun 2021 • Sangmin Bae, Sungnyun Kim, Jongwoo Ko, Gihun Lee, Seungjong Noh, Se-Young Yun
To this end, we propose Self-Contrastive (SelfCon) learning, which self-contrasts within multiple outputs from the different levels of a single network.
1 code implementation • 1 Nov 2023 • Yongjin Yang, Joonkee Kim, Yujin Kim, Namgyu Ho, James Thorne, Se-Young Yun
With the proliferation of social media, accurate detection of hate speech has become critical to ensure safety online.
1 code implementation • 1 Nov 2023 • Taehyeon Kim, Joonkee Kim, Gihun Lee, Se-Young Yun
Notably, utilizing 'opposite' as the noisy instruction in ID, which exhibits the maximum divergence from the original instruction, consistently produces the most significant performance gains across multiple models and tasks.
1 code implementation • 16 Oct 2023 • Jongwoo Ko, Seungjoon Park, Yujin Kim, Sumyeong Ahn, Du-Seong Chang, Euijai Ahn, Se-Young Yun
Structured pruning methods have proven effective in reducing the model size and accelerating inference speed in various network architectures such as Transformers.
1 code implementation • ICLR 2022 • Mingyu Kim, Kyeongryeol Go, Se-Young Yun
Neural processes (NPs) aim to stochastically complete unseen data points based on a given context dataset.
Ranked #1 on Multi-agent Reinforcement Learning on SMAC-Exp
1 code implementation • 15 Jun 2022 • Jongwoo Ko, Bongsoo Yi, Se-Young Yun
While existing methods address this problem in various directions, they still produce unpredictable sub-optimal results since they rely on the posterior information estimated by the feature extractor corrupted by noisy labels.
1 code implementation • 3 May 2022 • Sangmook Kim, Wonyoung Shin, Soohyuk Jang, Hwanjun Song, Se-Young Yun
Robustness is becoming another important challenge of federated learning in that the data collection process in each client is naturally accompanied by noisy labels.
2 code implementations • 5 Dec 2022 • Taehyeon Kim, Shinhwan Kang, Hyeonjeong Shin, Deukryeol Yoon, Seongha Eom, Kijung Shin, Se-Young Yun
The Weather4Cast competition (hosted by NeurIPS 2022) required competitors to predict super-resolution rain movies in various regions of Europe when low-resolution satellite contexts covering wider regions are given.
1 code implementation • 31 May 2022 • Sumyeong Ahn, Seongyoon Kim, Se-Young Yun
In this study, we propose a debiasing algorithm, called PGD (Per-sample Gradient-based Debiasing), that comprises three steps: (1) training a model on uniform batch sampling, (2) setting the importance of each sample in proportion to the norm of the sample gradient, and (3) training the model using importance-batch sampling, whose probability is obtained in step (2).
1 code implementation • IEEE Access 2022 • Taehyeon Kim, Se-Young Yun
Recent research in deep Convolutional Neural Networks(CNN) faces the challenges of vanishing/exploding gradient issues, training instability, and feature redundancy.
1 code implementation • 27 Nov 2023 • Yongjin Yang, Jongwoo Ko, Se-Young Yun
Vision-Language Models (VLMs) like CLIP have demonstrated remarkable applicability across a variety of downstream tasks, including zero-shot image classification.
1 code implementation • 18 Dec 2023 • Yongjin Yang, Taehyeon Kim, Se-Young Yun
Second, to address the pitfalls of noisy statistics, we deploy two strategies: a progressive training of the two adapters and an adaptive distillation technique derived from features determined by the model solely with the adapter devoid of a normalization layer.
1 code implementation • 13 Feb 2024 • Haeju Lee, Minchan Jeong, Se-Young Yun, Kee-Eung Kim
We argue that when we extract knowledge from source tasks via training source prompts, we need to consider this correlation among source tasks for better transfer to target tasks.
1 code implementation • 8 Feb 2019 • Daniel Bienstock, Minchan Jeong, Apurv Shukla, Se-Young Yun
We consider streaming principal component analysis when the stochastic data-generating model is subject to perturbations.
1 code implementation • 3 Mar 2017 • Jung-hun Kim, Se-Young Yun, Minchan Jeong, Jun Hyun Nam, Jinwoo Shin, Richard Combes
This implies that classical approaches cannot guarantee a non-trivial regret bound.
1 code implementation • 3 Jun 2022 • Taehyeon Kim, Se-Young Yun
The approach is inspired by observing that averaging parameters during model aggregation for FL is similar to weight-sharing in supernet training.
1 code implementation • 18 Oct 2022 • Jaehoon Oh, Jongwoo Ko, Se-Young Yun
Translation has played a crucial role in improving the performance on multilingual tasks: (1) to generate the target language data from the source language data for training and (2) to generate the source language data from the target language data for inference.
1 code implementation • 9 Mar 2023 • Junghyun Lee, Laura Schmid, Se-Young Yun
Then, to mitigate the issue of high communication costs incurred by flooding in complex networks, we propose a new protocol called Flooding with Absorption (FwA).
2 code implementations • 28 Oct 2023 • Junghyun Lee, Se-Young Yun, Kwang-Sung Jun
Logistic bandit is a ubiquitous framework of modeling users' choices, e. g., click vs. no click for advertisement recommender system.
1 code implementation • NeurIPS 2023 • Junghyun Lee, Hanseul Cho, Se-Young Yun, Chulhee Yun
Fair Principal Component Analysis (PCA) is a problem setting where we aim to perform PCA while making the resulting representation fair in that the projected distributions, conditional on the sensitive attributes, match one another.
no code implementations • NeurIPS 2016 • Se-Young Yun, Alexandre Proutiere
We find the set of parameters such that there exists a clustering algorithm with at most $s$ misclassified items in average under the general LSBM and for any $s=o(n)$, which solves one open problem raised in \cite{abbe2015community}.
no code implementations • 13 Apr 2015 • Se-Young Yun, Marc Lelarge, Alexandre Proutiere
We propose a streaming algorithm which produces an estimate of the original matrix with a vanishing mean square error, uses memory space scaling linearly with the ambient dimension of the matrix, i. e. the memory required to store the output alone, and spends computations as much as the number of non-zero entries of the input matrix.
no code implementations • 26 Oct 2018 • Jaehoon Oh, Duyeon Kim, Se-Young Yun
The proposed model can be used for not only singing voice separation but also multi-instrument separation by changing only the number of output channels.
no code implementations • NeurIPS 2015 • Se-Young Yun, Marc Lelarge, Alexandre Proutiere
This means that its average mean-square error converges to 0 as $m$ and $n$ grow large (i. e., $\|\hat{M}^{(k)}-M^{(k)} \|_F^2 = o(mn)$ with high probability, where $\hat{M}^{(k)}$ and $M^{(k)}$ denote the output of SLA and the optimal rank $k$ approximation of $M$, respectively).
no code implementations • NeurIPS 2014 • Se-Young Yun, Marc Lelarge, Alexandre Proutiere
The first algorithm is {\it offline}, as it needs to store and keep the assignments of nodes to clusters, and requires a memory that scales linearly with the network size.
1 code implementation • 1 Jan 2019 • Milan Vojnovic, Se-Young Yun, Kaifang Zhou
In this paper, we study a popular method for inference of the Bradley-Terry model parameters, namely the MM algorithm, for maximum likelihood estimation and maximum a posteriori probability estimation.
no code implementations • 14 May 2019 • Sejun Park, Eunho Yang, Se-Young Yun, Jinwoo Shin
Our contribution is two-fold: (a) we first propose a fully polynomial-time approximation scheme (FPTAS) for approximating the partition function of GM associating with a low-rank coupling matrix; (b) for general high-rank GMs, we design a spectral mean-field scheme utilizing (a) as a subroutine, where it approximates a high-rank GM into a product of rank-1 GMs for an efficient approximation of the partition function.
no code implementations • 23 Dec 2014 • Se-Young Yun, Alexandre Proutiere
We consider the problem of community detection in the Stochastic Block Model with a finite number $K$ of communities of sizes linearly growing with the network size $n$.
Social and Information Networks Data Structures and Algorithms
no code implementations • 29 Jul 2019 • Kuang Xu, Se-Young Yun
We focus on a family of decision rules where the agent makes a new choice by randomly selecting an action with a probability approximately proportional to the amount of past rewards associated with each action in her memory.
no code implementations • 14 Oct 2019 • Kaito Ariu, Jungseul Ok, Alexandre Proutiere, Se-Young Yun
The objective is to devise an algorithm with a minimal cluster recovery error rate.
1 code implementation • NeurIPS 2019 • Se-Young Yun, Alexandre Proutiere
We derive information-theoretical upper bounds on the cluster recovery rate.
no code implementations • 1 Jan 2021 • Taehyeon Kim, Jaehoon Oh, Nakyil Kim, Sangwook Cho, Se-Young Yun
To verify this conjecture, we test an extreme logit learning model, where the KD is implemented with Mean Squared Error (MSE) between the student's logit and the teacher's logit.
no code implementations • 1 Jan 2021 • Sungnyun Kim, Se-Young Yun
As numerous meta-learning algorithms improve performance when solving few-shot classification problems for practical applications, accurate prediction of uncertainty, though challenging, has been considered essential.
no code implementations • NeurIPS 2020 • Kaito Ariu, Narae Ryu, Se-Young Yun, Alexandre Proutière
Interestingly, our analysis reveals the relative weights of the different components of regret: the component due to the constraint of not presenting the same item twice to the same user, that due to learning the chances users like items, and finally that arising when learning the underlying structure.
no code implementations • 6 Dec 2020 • Jin-woo Lee, Jaehoon Oh, Sungsu Lim, Se-Young Yun, Jae-Gil Lee
Federated learning has emerged as a new paradigm of collaborative machine learning; however, many prior studies have used global aggregation along a star topology without much consideration of the communication scalability or the diurnal property relied on clients' local time variety.
1 code implementation • 30 Dec 2020 • Dabeen Lee, Milan Vojnovic, Se-Young Yun
Motivated by recent developments in designing algorithms based on individual item scores for solving utility maximization problems, we study the framework of using test scores, defined as a statistic of observed individual item performance data, for solving the budgeted stochastic utility maximization problem.
no code implementations • 29 Sep 2021 • Sumyeong Ahn, Se-Young Yun
The performance of deep neural networks (DNNs) primarily depends on the configuration of the training set.
no code implementations • 29 Sep 2021 • Sangmin Bae, Sungnyun Kim, Jongwoo Ko, Gihun Lee, Seungjong Noh, Se-Young Yun
This paper proposes a novel contrastive learning framework, called Self-Contrastive (SelfCon) Learning, that self-contrasts within multiple outputs from the different levels of a multi-exit network.
1 code implementation • 31 Jan 2022 • Jung-hun Kim, Milan Vojnovic, Se-Young Yun
We consider the infinitely many-armed bandit problem with rotting rewards, where the mean reward of an arm decreases at each pull of the arm according to an arbitrary trend with maximum rotting rate $\varrho=o(1)$.
no code implementations • 8 Apr 2022 • Stephen Cha, Taehyeon Kim, Hayeon Lee, Se-Young Yun
The survey analyses supernet optimization methods based on their approaches to spatial and temporal optimization.
no code implementations • 11 May 2022 • Jaehoon Oh, Sungnyun Kim, Namgyu Ho, Jin-Hwa Kim, Hwanjun Song, Se-Young Yun
Cross-domain few-shot learning (CD-FSL), where there are few target samples under extreme differences between source and target domains, has recently attracted huge attention.
no code implementations • 13 May 2022 • Yujin Kim, Jaehoon Oh, Sungnyun Kim, Se-Young Yun
Next, we show that data augmentation cannot guarantee few-shot performance improvement and investigate the effectiveness of data augmentation based on the intensity of augmentation.
no code implementations • 30 May 2022 • Jung-hun Kim, Se-Young Yun
We study the adversarial bandit problem against arbitrary strategies, in which $S$ is the parameter for the hardness of the problem and this parameter is not given to the agent.
no code implementations • 18 Jun 2022 • Jaehoon Oh, Se-Young Yun
Few-shot class-incremental learning (FSCIL) has addressed challenging real-world scenarios where unseen novel classes continually arrive with few samples.
no code implementations • 27 Jun 2022 • Taehyeon Kim, Heesoo Myeong, Se-Young Yun
Knowledge Distillation (KD) has recently emerged as a popular method for compressing neural networks.
no code implementations • 28 Jun 2022 • Jihwan Oh, Joonkee Kim, Se-Young Yun
Distributional reinforcement learning demonstrates state-of-the-art performance in continuous and discrete control settings with the features of variance and risk, which can be used to explore.
Distributional Reinforcement Learning reinforcement-learning +2
1 code implementation • 17 Aug 2022 • Yassir Jedra, Junghyun Lee, Alexandre Proutière, Se-Young Yun
We investigate the problems of model estimation and reward-free learning in episodic Block MDPs.
no code implementations • 1 Dec 2022 • Sumyeong Ahn, Se-Young Yun
Furthermore, we find that running denoising algorithms before debiasing is ineffective because denoising algorithms reduce the impact of difficult-to-learn samples, including valuable bias-conflicting samples.
1 code implementation • 3 Feb 2023 • Jongwoo Ko, Seungjoon Park, Minchan Jeong, Sukjin Hong, Euijai Ahn, Du-Seong Chang, Se-Young Yun
Knowledge distillation (KD) is a highly promising method for mitigating the computational problems of pre-trained language models (PLMs).
no code implementations • 3 Mar 2023 • Jihwan Oh, Joonkee Kim, Minchan Jeong, Se-Young Yun
In this paper, we present a risk-based exploration that leads to collaboratively optimistic behavior by shifting the sampling region of distribution.
Distributional Reinforcement Learning Multi-agent Reinforcement Learning +2
no code implementations • 18 Jun 2023 • Kaito Ariu, Alexandre Proutiere, Se-Young Yun
To this end, we revisit instance-specific lower bounds on the expected number of misclassified items satisfied by any clustering algorithm.
no code implementations • 24 Aug 2023 • Gihun Lee, Minchan Jeong, Sangmook Kim, Jaehoon Oh, Se-Young Yun
FedSOL is designed to identify gradients of local objectives that are inherently orthogonal to directions affecting the proximal objective.
no code implementations • 29 Aug 2023 • Seongha Eom, Namgyu Ho, Jaehoon Oh, Se-Young Yun
Given a query image, we harness the power of CLIP's cross-modal representations to retrieve relevant textual information from an external image-text pair dataset.
no code implementations • 11 Oct 2023 • Seonghyun Park, Narae Ryu, Gahee Kim, Dongyeop Woo, Se-Young Yun, Sungsoo Ahn
In this work, we propose to resolve such a redundancy via the non-backtracking graph neural network (NBA-GNN) that updates a message without incorporating the message from the previously visited node.
no code implementations • 24 Oct 2023 • Sumyeong Ahn, Sihyeon Kim, Jongwoo Ko, Se-Young Yun
To tackle this issue, researchers have explored methods for Learning with Noisy Labels to identify clean samples and reduce the influence of noisy labels.
no code implementations • 13 Nov 2023 • Felix den Breejen, Sangmin Bae, Stephen Cha, Tae-Young Kim, Seoung Hyun Koh, Se-Young Yun
While interests in tabular deep learning has significantly grown, conventional tree-based models still outperform deep learning methods.
no code implementations • 14 Nov 2023 • Yujin Kim, Jaehong Yoon, Seonghyeon Ye, Sangmin Bae, Namgyu Ho, Sung Ju Hwang, Se-Young Yun
The dynamic nature of knowledge in an ever-changing world presents challenges for language models trained on static data; the model in the real world often requires not only acquiring new knowledge but also overwriting outdated information into updated ones.
no code implementations • 22 Nov 2023 • Seongyoon Kim, Gihun Lee, Jaehoon Oh, Se-Young Yun
Additionally, we observe that as data heterogeneity increases, the gap between higher feature norms for observed classes, obtained from local models, and feature norms of unobserved classes widens, in contrast to the behavior of classifier weight norms.
no code implementations • 8 Feb 2024 • Taehyeon Kim, Donggyu Kim, Se-Young Yun
In the evolving landscape of federated learning (FL), addressing label noise presents unique challenges due to the decentralized and diverse nature of data collection across clients.
no code implementations • 10 Feb 2024 • Marc Bartholet, Taehyeon Kim, Ami Beuret, Se-Young Yun, Joachim M. Buhmann
We propose an innovative federated algorithm, termed hFedF for hypernetwork-based Federated Fusion, designed to bridge the performance gap between generalization and personalization, capable of addressing various degrees of domain shift.
no code implementations • 22 Apr 2024 • Jung-hun Kim, Milan Vojnovic, Se-Young Yun
In this study, we consider the infinitely many armed bandit problems in rotting environments, where the mean reward of an arm may decrease with each pull, while otherwise, it remains unchanged.
no code implementations • 29 Apr 2024 • Yongjin Yang, Sihyeon Kim, Sangmook Kim, Gyubok Lee, Se-Young Yun, Edward Choi
Incorporating unanswerable questions into EHR QA systems is crucial for testing the trustworthiness of a system, as providing non-existent responses can mislead doctors in their diagnoses.