Search Results for author: Eunho Yang

Found 72 papers, 26 papers with code

Adaptive Proximal Gradient Methods for Structured Neural Networks

no code implementations NeurIPS 2021 Jihun Yun, Aurelie C. Lozano, Eunho Yang

We consider the training of structured neural networks where the regularizer can be non-smooth and possibly non-convex.

Quantization

Graph Transplant: Node Saliency-Guided Graph Mixup with Local Structure Preservation

no code implementations10 Nov 2021 Joonhyung Park, Hajin Shim, Eunho Yang

Graph-structured datasets usually have irregular graph sizes and connectivities, rendering the use of recent data augmentation techniques, such as Mixup, difficult.

Data Augmentation Graph Classification

Online Hyperparameter Meta-Learning with Hypergradient Distillation

no code implementations6 Oct 2021 Hae Beom Lee, Hayeon Lee, Jaewoong Shin, Eunho Yang, Timothy Hospedales, Sung Ju Hwang

Many gradient-based meta-learning methods assume a set of parameters that do not participate in inner-optimization, which can be considered as hyperparameters.

Hyperparameter Optimization Knowledge Distillation +1

Bias Decay Matters : Improving Large Batch Optimization with Connectivity Sharpness

no code implementations29 Sep 2021 Sungyub Kim, Sihwan Park, Yong-Deok Kim, Eunho Yang

To mitigate this issue, we propose simple bias decay methods including a novel adaptive one and found that this simple remedy can fill a large portion of the performance gaps that occur in large batch optimization.

Stop just recalling memorized relations: Extracting Unseen Relational Triples from the context

no code implementations29 Sep 2021 Juhyuk Lee, Min-Joong Lee, June Yong Yang, Eunho Yang

In this paper, we show that although existing extraction models are able to memorize and recall already seen triples, they cannot generalize effectively for unseen triples.

Knowledge Graphs

Distilling Linguistic Context for Language Model Compression

no code implementations EMNLP 2021 Geondo Park, Gyeongman Kim, Eunho Yang

A computationally expensive and memory intensive neural network lies behind the recent success of language representation learning.

Knowledge Distillation Language Modelling +3

Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss

no code implementations ICCV 2021 Jung Hyun Lee, Jihun Yun, Sung Ju Hwang, Eunho Yang

Network quantization, which aims to reduce the bit-lengths of the network weights and activations, has emerged for their deployments to resource-limited devices.

Quantization

FedMix: Approximation of Mixup under Mean Augmented Federated Learning

no code implementations ICLR 2021 Tehrim Yoon, Sumin Shin, Sung Ju Hwang, Eunho Yang

Federated learning (FL) allows edge devices to collectively learn a model without directly sharing data within each device, thus preserving privacy and eliminating the need to store data globally.

Data Augmentation Federated Learning

Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation

2 code implementations6 Jun 2021 Dongchan Min, Dong Bok Lee, Eunho Yang, Sung Ju Hwang

In this work, we propose StyleSpeech, a new TTS model which not only synthesizes high-quality speech but also effectively adapts to new speakers.

Fine-tuning

Online Coreset Selection for Rehearsal-based Continual Learning

no code implementations2 Jun 2021 Jaehong Yoon, Divyam Madaan, Eunho Yang, Sung Ju Hwang

We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.

Continual Learning

RetCL: A Selection-based Approach for Retrosynthesis via Contrastive Learning

no code implementations3 May 2021 Hankook Lee, Sungsoo Ahn, Seung-Woo Seo, You Young Song, Eunho Yang, Sung-Ju Hwang, Jinwoo Shin

Retrosynthesis, of which the goal is to find a set of reactants for synthesizing a target product, is an emerging research area of deep learning.

Contrastive Learning

Mutually-Constrained Monotonic Multihead Attention for Online ASR

no code implementations26 Mar 2021 Jaeyun Song, Hajin Shim, Eunho Yang

Despite the feature of real-time decoding, Monotonic Multihead Attention (MMA) shows comparable performance to the state-of-the-art offline methods in machine translation and automatic speech recognition (ASR) tasks.

automatic-speech-recognition Machine Translation +2

Model-Augmented Q-learning

no code implementations7 Feb 2021 Youngmin Oh, Jinwoo Shin, Eunho Yang, Sung Ju Hwang

We show that the proposed scheme, called Model-augmented $Q$-learning (MQL), obtains a policy-invariant solution which is identical to the solution obtained by learning with true reward.

Q-Learning

Contextual Knowledge Distillation for Transformer Compression

no code implementations1 Jan 2021 Geondo Park, Gyeongman Kim, Eunho Yang

A computationally expensive and memory intensive neural network lies behind the recent success of language representation learning.

Knowledge Distillation Language Modelling +2

Rapid Neural Pruning for Novel Datasets with Set-based Task-Adaptive Meta-Pruning

no code implementations1 Jan 2021 Minyoung Song, Jaehong Yoon, Eunho Yang, Sung Ju Hwang

As deep neural networks are growing in size and being increasingly deployed to more resource-limited devices, there has been a recent surge of interest in network pruning methods, which aim to remove less important weights or activations of a given network.

Network Pruning

Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bitwise Regularization

no code implementations1 Jan 2021 Jung Hyun Lee, Jihun Yun, Sung Ju Hwang, Eunho Yang

As a natural extension of DropBits, we further introduce the way of learning heterogeneous quantization levels to find proper bit-length for each layer using DropBits.

Quantization

Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated Label Mixing

no code implementations1 Jan 2021 Joonhyung Park, June Yong Yang, Jinwoo Shin, Sung Ju Hwang, Eunho Yang

By stochastically sampling the features and ‘grafting’ them onto another sample, our method effectively generates diverse yet meaningful samples.

Data Augmentation

Attribution Preservation in Network Compression for Reliable Network Interpretation

1 code implementation NeurIPS 2020 Geondo Park, June Yong Yang, Sung Ju Hwang, Eunho Yang

Neural networks embedded in safety-sensitive applications such as self-driving cars and wearable health monitors rely on two important techniques: input attribution for hindsight analysis and network compression to reduce its size for edge-computing.

Edge-computing Self-Driving Cars

Neural Complexity Measures

1 code implementation NeurIPS 2020 Yoonho Lee, Juho Lee, Sung Ju Hwang, Eunho Yang, Seungjin Choi

While various complexity measures for deep neural networks exist, specifying an appropriate measure capable of predicting and explaining generalization in deep networks has proven challenging.

Meta-Learning

Bootstrapping Neural Processes

1 code implementation NeurIPS 2020 Juho Lee, Yoonho Lee, Jungtaek Kim, Eunho Yang, Sung Ju Hwang, Yee Whye Teh

While this "data-driven" way of learning stochastic processes has proven to handle various types of data, NPs still rely on an assumption that uncertainty in stochastic processes is modeled by a single latent variable, which potentially limits the flexibility.

Few-shot Visual Reasoning with Meta-analogical Contrastive Learning

no code implementations NeurIPS 2020 Youngsung Kim, Jinwoo Shin, Eunho Yang, Sung Ju Hwang

While humans can solve a visual puzzle that requires logical reasoning by observing only few samples, it would require training over large amount of data for state-of-the-art deep reasoning models to obtain similar performance on the same task.

Contrastive Learning Visual Reasoning

Time-Reversal Symmetric ODE Network

1 code implementation NeurIPS 2020 In Huh, Eunho Yang, Sung Ju Hwang, Jinwoo Shin

Time-reversal symmetry, which requires that the dynamics of a system should not change with the reversal of time axis, is a fundamental property that frequently holds in classical and quantum mechanics.

Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning

1 code implementation NeurIPS 2020 Jaehyung Kim, Youngbum Hur, Sejun Park, Eunho Yang, Sung Ju Hwang, Jinwoo Shin

While semi-supervised learning (SSL) has proven to be a promising way for leveraging unlabeled data when labeled data is scarce, the existing SSL algorithms typically assume that training class distributions are balanced.

A General Family of Stochastic Proximal Gradient Methods for Deep Learning

no code implementations15 Jul 2020 Jihun Yun, Aurelie C. Lozano, Eunho Yang

We propose a unified framework for stochastic proximal gradient descent, which we term ProxGen, that allows for arbitrary positive preconditioners and lower semi-continuous regularizers.

Quantization

Learning to Sample with Local and Global Contexts in Experience Replay Buffer

no code implementations ICLR 2021 Youngmin Oh, Kimin Lee, Jinwoo Shin, Eunho Yang, Sung Ju Hwang

Experience replay, which enables the agents to remember and reuse experience from the past, has played a significant role in the success of off-policy reinforcement learning (RL).

A Revision of Neural Tangent Kernel-based Approaches for Neural Networks

no code implementations2 Jul 2020 Kyung-Su Kim, Aurélie C. Lozano, Eunho Yang

(2) A generalization error bound invariant of network size was derived by using a data-dependent complexity measure (CMD).

Few-Shot Learning Learning Theory

Compressed Sensing via Measurement-Conditional Generative Models

no code implementations2 Jul 2020 Kyung-Su Kim, Jung Hyun Lee, Eunho Yang

A pre-trained generator has been frequently adopted in compressed sensing (CS) due to its ability to effectively estimate signals with the prior of NNs.

Stochastic Subset Selection for Efficient Training and Inference of Neural Networks

no code implementations25 Jun 2020 Bruno Andreis, A. Tuan Nguyen, Seanie Lee, Juho Lee, Eunho Yang, Sung Ju Hwang

We also show in our experiments that our method enhances scalability of nonparametric models such as Neural Processes.

Feature Selection Meta-Learning

Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learning

1 code implementation23 Jun 2020 A. Tuan Nguyen, Hyewon Jeong, Eunho Yang, Sung Ju Hwang

Existing asymmetric multi-task learning methods tackle this negative transfer problem by performing knowledge transfer from tasks with low loss to tasks with high loss.

Knowledge Graphs Multi-Task Learning +2

Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning

1 code implementation ICLR 2021 Wonyong Jeong, Jaehong Yoon, Eunho Yang, Sung Ju Hwang

Through extensive experimental validation of our method in the two different scenarios, we show that our method outperforms both local semi-supervised learning and baselines which naively combine federated learning with semi-supervised learning.

Federated Learning

Rapid Structural Pruning of Neural Networks with Set-based Task-Adaptive Meta-Pruning

no code implementations22 Jun 2020 Minyoung Song, Jaehong Yoon, Eunho Yang, Sung Ju Hwang

As deep neural networks are growing in size and being increasingly deployed to more resource-limited devices, there has been a recent surge of interest in network pruning methods, which aim to remove less important weights or activations of a given network.

Network Pruning

Cost-effective Interactive Attention Learning with Neural Attention Processes

2 code implementations9 Jun 2020 Jay Heo, Junhyeon Park, Hyewon Jeong, Kwang Joon Kim, Juho Lee, Eunho Yang, Sung Ju Hwang

Moreover, it is almost infeasible for the human annotators to examine attentions on tons of instances and features.

Time Series

Meta Dropout: Learning to Perturb Latent Features for Generalization

2 code implementations ICLR 2020 Hae Beom Lee, Taewook Nam, Eunho Yang, Sung Ju Hwang

Specifically, we meta-learn a noise generator which outputs a multiplicative noise distribution for latent features, to obtain low errors on the test instances in an input-dependent manner.

Meta-Learning

Federated Continual Learning with Weighted Inter-client Transfer

1 code implementation6 Mar 2020 Jaehong Yoon, Wonyong Jeong, Giwoong Lee, Eunho Yang, Sung Ju Hwang

There has been a surge of interest in continual learning and federated learning, both of which are important in deep neural networks in real-world scenarios.

Continual Learning Federated Learning +1

Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bit-wise Regularization

no code implementations29 Nov 2019 Jung Hyun Lee, Jihun Yun, Sung Ju Hwang, Eunho Yang

As a natural extension of DropBits, we further introduce the way of learning heterogeneous quantization levels to find proper bit-length for each layer using DropBits.

Quantization

Temporal Probabilistic Asymmetric Multi-task Learning

no code implementations25 Sep 2019 Nguyen Anh Tuan, Hyewon Jeong, Eunho Yang, Sungju Hwang

To capture such dynamically changing asymmetric relationships between tasks and long-range temporal dependencies in time-series data, we propose a novel temporal asymmetric multi-task learning model, which learns to combine features from other tasks at diverse timesteps for the prediction of each task.

Multi-Task Learning Time Series +1

Reliable Estimation of Individual Treatment Effect with Causal Information Bottleneck

no code implementations7 Jun 2019 Sungyub Kim, Yongsu Baek, Sung Ju Hwang, Eunho Yang

We also introduce an additional form of a regularizer from the perspective of understanding ITE in the semi-supervised learning framework to ensure more reliable representations.

Causal Inference

Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks

1 code implementation ICLR 2020 Joonyoung Yi, Juhyuk Lee, Kwang Joon Kim, Sung Ju Hwang, Eunho Yang

Among many approaches, the simplest and most intuitive way is zero imputation, which treats the value of a missing entry simply as zero.

Collaborative Filtering Imputation

Meta Dropout: Learning to Perturb Features for Generalization

1 code implementation30 May 2019 Hae Beom Lee, Taewook Nam, Eunho Yang, Sung Ju Hwang

Specifically, we meta-learn a noise generator which outputs a multiplicative noise distribution for latent features, to obtain low errors on the test instances in an input-dependent manner.

Meta-Learning

Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Tasks

1 code implementation ICLR 2020 Hae Beom Lee, Hayeon Lee, Donghyun Na, Saehoon Kim, Minseop Park, Eunho Yang, Sung Ju Hwang

While tasks could come with varying the number of instances and classes in realistic settings, the existing meta-learning approaches for few-shot classification assume that the number of instances per task and class is fixed.

Bayesian Inference Meta-Learning +1

Stochastic Gradient Methods with Block Diagonal Matrix Adaptation

no code implementations26 May 2019 Jihun Yun, Aurelie C. Lozano, Eunho Yang

Extensive experiments reveal that block-diagonal approaches achieve state-of-the-art results on several deep learning tasks, and can outperform adaptive diagonal methods, vanilla Sgd, as well as a modified version of full-matrix adaptation proposed very recently.

Spectral Approximate Inference

no code implementations14 May 2019 Sejun Park, Eunho Yang, Se-Young Yun, Jinwoo Shin

Our contribution is two-fold: (a) we first propose a fully polynomial-time approximation scheme (FPTAS) for approximating the partition function of GM associating with a low-rank coupling matrix; (b) for general high-rank GMs, we design a spectral mean-field scheme utilizing (a) as a subroutine, where it approximates a high-rank GM into a product of rank-1 GMs for an efficient approximation of the partition function.

Scalable and Order-robust Continual Learning with Additive Parameter Decomposition

1 code implementation ICLR 2020 Jaehong Yoon, Saehoon Kim, Eunho Yang, Sung Ju Hwang

First, a continual learning model should effectively handle catastrophic forgetting and be efficient to train even with a large number of tasks.

Continual Learning Fairness +1

Joint Active Feature Acquisition and Classification with Variable-Size Set Encoding

no code implementations NeurIPS 2018 Hajin Shim, Sung Ju Hwang, Eunho Yang

We consider the problem of active feature acquisition where the goal is to sequentially select the subset of features in order to achieve the maximum prediction performance in the most cost-effective way at test time.

Classification General Classification

ADAPTIVE NETWORK SPARSIFICATION VIA DEPENDENT VARIATIONAL BETA-BERNOULLI DROPOUT

no code implementations27 Sep 2018 Juho Lee, Saehoon Kim, Jaehong Yoon, Hae Beom Lee, Eunho Yang, Sung Ju Hwang

With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss.

Deep Mixed Effect Model using Gaussian Processes: A Personalized and Reliable Prediction for Healthcare

2 code implementations5 Jun 2018 Ingyo Chung, Saehoon Kim, Juho Lee, Kwang Joon Kim, Sung Ju Hwang, Eunho Yang

We present a personalized and reliable prediction model for healthcare, which can provide individually tailored medical services such as diagnosis, disease treatment, and prevention.

Gaussian Processes Time Series

Adaptive Network Sparsification with Dependent Variational Beta-Bernoulli Dropout

1 code implementation28 May 2018 Juho Lee, Saehoon Kim, Jaehong Yoon, Hae Beom Lee, Eunho Yang, Sung Ju Hwang

With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss.

DropMax: Adaptive Stochastic Softmax

no code implementations ICLR 2018 Hae Beom Lee, Juho Lee, Eunho Yang, Sung Ju Hwang

Moreover, the learning of dropout probabilities for non-target classes on each instance allows the classifier to focus more on classification against the most confusing classes.

Classification General Classification +1

DropMax: Adaptive Variational Softmax

4 code implementations NeurIPS 2018 Hae Beom Lee, Juho Lee, Saehoon Kim, Eunho Yang, Sung Ju Hwang

Moreover, the learning of dropout rates for non-target classes on each instance allows the classifier to focus more on classification against the most confusing classes.

Classification General Classification +1

Why Pay More When You Can Pay Less: A Joint Learning Framework for Active Feature Acquisition and Classification

no code implementations18 Sep 2017 Hajin Shim, Sung Ju Hwang, Eunho Yang

We consider the problem of active feature acquisition, where we sequentially select the subset of features in order to achieve the maximum prediction performance in the most cost-effective way.

General Classification

Lifelong Learning with Dynamically Expandable Networks

2 code implementations ICLR 2018 Jaehong Yoon, Eunho Yang, Jeongtae Lee, Sung Ju Hwang

We propose a novel deep network architecture for lifelong learning which we refer to as Dynamically Expandable Network (DEN), that can dynamically decide its network capacity as it trains on a sequence of tasks, to learn a compact overlapping knowledge sharing structure among tasks.

Deep Asymmetric Multi-task Feature Learning

1 code implementation ICML 2018 Hae Beom Lee, Eunho Yang, Sung Ju Hwang

We propose Deep Asymmetric Multitask Feature Learning (Deep-AMTFL) which can learn deep representations shared across multiple tasks while effectively preventing negative transfer that may happen in the feature sharing process.

Image Classification Transfer Learning

Sparse + Group-Sparse Dirty Models: Statistical Guarantees without Unreasonable Conditions and a Case for Non-Convexity

no code implementations ICML 2017 Eunho Yang, Aurélie C. Lozano

Imposing sparse + group-sparse superposition structures in high-dimensional parameter estimation is known to provide flexible regularization that is more realistic for many real-world problems.

Multi-Task Learning

Ordinal Graphical Models: A Tale of Two Approaches

no code implementations ICML 2017 Arun Sai Suggala, Eunho Yang, Pradeep Ravikumar

While there have been some work on tractable approximations, these do not come with strong statistical guarantees, and moreover are relatively computationally expensive.

Learning task structure via sparsity grouped multitask learning

1 code implementation13 May 2017 Meghana Kshirsagar, Eunho Yang, Aurélie C. Lozano

We further demonstrate that our proposed method recovers groups and the sparsity patterns in the task parameters accurately by extensive experiments.

Feature Selection Sparse Learning

Sequential Local Learning for Latent Graphical Models

no code implementations12 Mar 2017 Sejun Park, Eunho Yang, Jinwoo Shin

Learning parameters of latent graphical models (GM) is inherently much harder than that of no-latent ones since the latent variables make the corresponding log-likelihood non-concave.

A Review of Multivariate Distributions for Count Data Derived from the Poisson Distribution

1 code implementation31 Aug 2016 David I. Inouye, Eunho Yang, Genevera I. Allen, Pradeep Ravikumar

The Poisson distribution has been widely studied and used for modeling univariate count-valued data.

A General Family of Trimmed Estimators for Robust High-dimensional Data Analysis

no code implementations26 May 2016 Eunho Yang, Aurelie Lozano, Aleksandr Aravkin

We consider the problem of robustifying high-dimensional structured estimation.

Robust Gaussian Graphical Modeling with the Trimmed Graphical Lasso

no code implementations NeurIPS 2015 Eunho Yang, Aurélie C. Lozano

In this paper, we propose the Trimmed Graphical Lasso for robust estimation of sparse GGMs.

Elementary Estimators for Graphical Models

no code implementations NeurIPS 2014 Eunho Yang, Aurelie C. Lozano, Pradeep K. Ravikumar

We propose a class of closed-form estimators for sparsity-structured graphical models, expressed as exponential family distributions, under high-dimensional settings.

A General Framework for Mixed Graphical Models

no code implementations2 Nov 2014 Eunho Yang, Pradeep Ravikumar, Genevera I. Allen, Yulia Baker, Ying-Wooi Wan, Zhandong Liu

"Mixed Data" comprising a large number of heterogeneous variables (e. g. count, binary, continuous, skewed continuous, among other data types) are prevalent in varied areas such as genomics and proteomics, imaging genetics, national security, social networking, and Internet advertising.

On Poisson Graphical Models

no code implementations NeurIPS 2013 Eunho Yang, Pradeep K. Ravikumar, Genevera I. Allen, Zhandong Liu

Undirected graphical models, such as Gaussian graphical models, Ising, and multinomial/categorical graphical models, are widely used in a variety of applications for modeling distributions over a large number of variables.

Conditional Random Fields via Univariate Exponential Families

no code implementations NeurIPS 2013 Eunho Yang, Pradeep K. Ravikumar, Genevera I. Allen, Zhandong Liu

We thus introduce a “novel subclass of CRFs”, derived by imposing node-wise conditional distributions of response variables conditioned on the rest of the responses and the covariates as arising from univariate exponential families.

Dirty Statistical Models

no code implementations NeurIPS 2013 Eunho Yang, Pradeep K. Ravikumar

We provide a unified framework for the high-dimensional analysis of “superposition-structured” or “dirty” statistical models: where the model parameters are a “superposition” of structurally constrained parameters.

On Graphical Models via Univariate Exponential Family Distributions

no code implementations17 Jan 2013 Eunho Yang, Pradeep Ravikumar, Genevera I. Allen, Zhandong Liu

Undirected graphical models, or Markov networks, are a popular class of statistical models, used in a wide variety of applications.

Graphical Models via Generalized Linear Models

no code implementations NeurIPS 2012 Eunho Yang, Genevera Allen, Zhandong Liu, Pradeep K. Ravikumar

Our models allow one to estimate networks for a wide class of exponential distributions, such as the Poisson, negative binomial, and exponential, by fitting penalized GLMs to select the neighborhood for each node.

Cannot find the paper you are looking for? You can Submit a new open access paper.