Search Results for author: Jaegul Choo

Found 158 papers, 67 papers with code

Novel Natural Language Summarization of Program Code via Leveraging Multiple Input Representations

no code implementations Findings (EMNLP) 2021 Fuxiang Chen, Mijung Kim, Jaegul Choo

To tackle this problem, previous work on code summarization, the task of automatically generating code description given a piece of code reported that an auxiliary learning model trained to produce API (Application Programming Interface) embeddings showed promising results when applied to a downstream, code summarization model.

Auxiliary Learning Code Summarization +1

PromptDresser: Improving the Quality and Controllability of Virtual Try-On via Generative Textual Prompt and Prompt-aware Mask

1 code implementation22 Dec 2024 Jeongho Kim, Hoiyeong Jin, Sunghyun Park, Jaegul Choo

In the text-editable virtual try-on, three key aspects exist: (i) designing rich text descriptions for paired person-clothing data to train the model, (ii) addressing the conflicts where textual information of the existing person's clothing interferes the generation of the new clothing, and (iii) adaptively adjust the inpainting mask aligned with the text descriptions, ensuring proper editing areas while preserving the original person's appearance irrelevant to the new clothing.

In-Context Learning Virtual Try-on

Enabling Region-Specific Control via Lassos in Point-Based Colorization

no code implementations18 Dec 2024 Sanghyeon Lee, Jooyeol Yun, Jaegul Choo

Point-based interactive colorization techniques allow users to effortlessly colorize grayscale images using user-provided color hints.

Colorization

Sparse autoencoders reveal selective remapping of visual concepts during adaptation

1 code implementation6 Dec 2024 Hyesu Lim, Jinho Choi, Jaegul Choo, Steffen Schneider

Adapting foundation models for specific purposes has become a standard approach to build machine learning systems for downstream applications.

Image Classification

Spatiotemporal Skip Guidance for Enhanced Video Diffusion Sampling

no code implementations27 Nov 2024 Junha Hyung, Kinam Kim, Susung Hong, Min-Jung Kim, Jaegul Choo

Our contributions include: (1) introducing STG as an efficient, high-performing guidance technique for video diffusion models, (2) eliminating the need for auxiliary models by simulating a weak model through layer skipping, and (3) ensuring quality-enhanced guidance without compromising sample diversity or dynamics unlike CFG.

Diversity

Forecasting Future International Events: A Reliable Dataset for Text-Based Event Modeling

1 code implementation21 Nov 2024 Daehoon Gwak, Junwoo Park, Minho Park, ChaeHun Park, Hyunchan Lee, Edward Choi, Jaegul Choo

Predicting future international events from textual information, such as news articles, has tremendous potential for applications in global policy, strategic decision-making, and geopolitics.

Benchmarking Decision Making

Evaluating and Improving Automatic Speech Recognition Systems for Korean Meteorological Experts

no code implementations24 Oct 2024 ChaeHun Park, Hojun Cho, Jaegul Choo

This paper explores integrating Automatic Speech Recognition (ASR) into natural language query systems to improve weather forecasting efficiency for Korean meteorologists.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Breaking Chains: Unraveling the Links in Multi-Hop Knowledge Unlearning

no code implementations17 Oct 2024 Minseok Choi, ChaeHun Park, Dohyun Lee, Jaegul Choo

Large language models (LLMs) serve as giant information stores, often including personal or copyrighted data, and retraining them from scratch is not a viable option.

Decision Making Language Modeling +1

Generative Location Modeling for Spatially Aware Object Insertion

no code implementations17 Oct 2024 Jooyeol Yun, Davide Abati, Mohamed Omran, Jaegul Choo, Amirhossein Habibian, Auke Wiggers

In this paper, we focus on the former, creating a location model dedicated to identifying realistic object locations.

Object

SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatars

no code implementations15 Oct 2024 Jaeseong Lee, Taewoong Kang, Marcel C. Bühler, Min-Jung Kim, Sungwon Hwang, Junha Hyung, Hyojin Jang, Jaegul Choo

Recent advancements in head avatar rendering using Gaussian primitives have achieved significantly high-fidelity results.

SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning

3 code implementations13 Oct 2024 Hojoon Lee, Dongyoon Hwang, Donghu Kim, Hyunseung Kim, Jun Jet Tai, Kaushik Subramanian, Peter R. Wurman, Jaegul Choo, Peter Stone, Takuma Seno

Recent advances in CV and NLP have been largely driven by scaling up the number of network parameters, despite traditional theories suggesting that larger networks are prone to overfitting.

Computational Efficiency Deep Reinforcement Learning

Bones Can't Be Triangles: Accurate and Efficient Vertebrae Keypoint Estimation through Collaborative Error Revision

no code implementations5 Sep 2024 Jinhee Kim, Taesung Kim, Jaegul Choo

Recent advances in interactive keypoint estimation methods have enhanced accuracy while minimizing user intervention.

Keypoint Estimation

Skip-and-Play: Depth-Driven Pose-Preserved Image Generation for Any Objects

no code implementations4 Sep 2024 Kyungmin Jo, Jaegul Choo

However, depth-based pose control confronts issues of shape dependency, as depth maps influence not only the pose but also the shape of the generated images.

Depth Estimation Image Generation

What to Preserve and What to Transfer: Faithful, Identity-Preserving Diffusion-based Hairstyle Transfer

1 code implementation29 Aug 2024 Chaeyeon Chung, Sunghyun Park, Jeongho Kim, Jaegul Choo

Hairstyle transfer is a challenging task in the image editing field that modifies the hairstyle of a given face image while preserving its other appearance and background features.

Training Spatial-Frequency Visual Prompts and Probabilistic Clusters for Accurate Black-Box Transfer Learning

no code implementations15 Aug 2024 Wonwoo Cho, Kangyeol Kim, Saemee Choi, Jaegul Choo

Despite the growing prevalence of black-box pre-trained models (PTMs) such as prediction API services, there remains a significant challenge in directly applying general models to real-world scenarios due to the data distribution gap.

Transfer Learning

Is user feedback always informative? Retrieval Latent Defending for Semi-Supervised Domain Adaptation without Source Data

1 code implementation22 Jul 2024 Junha Song, Tae Soo Kim, Junha Kim, Gunhee Nam, Thijs Kooi, Jaegul Choo

This approach helps existing SemiSDA methods to adapt the model with a balanced supervised signal by utilizing latent defending samples throughout the adaptation process.

Domain Adaptation Image Classification +2

Pacer and Runner: Cooperative Learning Framework between Single- and Cross-Domain Sequential Recommendation

1 code implementation15 Jul 2024 Chung Park, Taesan Kim, Hyungjun Yoon, Junui Hong, Yelim Yu, Mincheol Cho, Minsung Choi, Jaegul Choo

To this end, our model compares the performance of a model trained on multiple domains (CDSR) with a model trained solely on the specific domain (SDSR) to evaluate the negative transfer of each domain using our asymmetric cooperative network.

Sequential Recommendation

Layout-and-Retouch: A Dual-stage Framework for Improving Diversity in Personalized Image Generation

no code implementations13 Jul 2024 Kangyeol Kim, Wooseok Seo, Sehyun Nam, Bodam Kim, Suhyeon Jeong, Wonwoo Cho, Jaegul Choo, Youngjae Yu

Personalized text-to-image (P-T2I) generation aims to create new, text-guided images featuring the personalized subject with a few reference images.

Diversity Image Generation +1

TCAN: Animating Human Images with Temporally Consistent Pose Guidance using Diffusion Models

no code implementations12 Jul 2024 Jeongho Kim, Min-Jung Kim, Junsoo Lee, Jaegul Choo

Extensive experiments demonstrate that the proposed method can achieve promising results in video synthesis tasks encompassing various poses, like chibi.

Image Animation

Scaling Up Personalized Image Aesthetic Assessment via Task Vector Customization

1 code implementation9 Jul 2024 Jooyeol Yun, Jaegul Choo

To overcome this long-standing scalability challenge, we present a unique approach that leverages readily available databases for general image aesthetic assessment and image quality assessment.

Image Quality Assessment

VEGS: View Extrapolation of Urban Scenes in 3D Gaussian Splatting using Learned Priors

1 code implementation3 Jul 2024 Sungwon Hwang, Min-Jung Kim, Taewoong Kang, Jayeon Kang, Jaegul Choo

To the best of our knowledge, we are the first to address the EVS problem in urban scene reconstruction.

Neural Rendering

Protecting Privacy Through Approximating Optimal Parameters for Sequence Unlearning in Language Models

no code implementations20 Jun 2024 Dohyun Lee, Daniel Rim, Minseok Choi, Jaegul Choo

In this work, we propose Privacy Protection via Optimal Parameters (POP), a novel unlearning method that effectively forgets the target token sequences from the pretrained LM by applying optimal gradient updates to the parameters.

Machine Unlearning Memorization

The Comparative Trap: Pairwise Comparisons Amplifies Biased Preferences of LLM Evaluators

no code implementations18 Jun 2024 Hawon Jeong, ChaeHun Park, Jimin Hong, Hojoon Lee, Jaegul Choo

As large language models (LLMs) are increasingly used as evaluators for natural language generation tasks, ensuring unbiased assessments is essential.

Instruction Following Text Generation

Can Tool-augmented Large Language Models be Aware of Incomplete Conditions?

no code implementations18 Jun 2024 Seungbin Yang, ChaeHun Park, Taehee Kim, Jaegul Choo

However, these tool-augmented LLMs often encounter incomplete scenarios when users provide partial information or the necessary tools are unavailable.

Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models

1 code implementation18 Jun 2024 Minseok Choi, Kyunghyun Min, Jaegul Choo

Pretrained language models memorize vast amounts of information, including private and copyrighted data, raising significant safety concerns.

Machine Unlearning

Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting

no code implementations17 Jun 2024 Junha Hyung, Susung Hong, Sungwon Hwang, Jaeseong Lee, Jaegul Choo, Jin-Hwa Kim

3D reconstruction from multi-view images is one of the fundamental challenges in computer vision and graphics.

3DGS 3D Reconstruction

Adapting Pretrained ViTs with Convolution Injector for Visuo-Motor Control

1 code implementation10 Jun 2024 Dongyoon Hwang, Byungkun Lee, Hojoon Lee, Hyunseung Kim, Jaegul Choo

To this end, we introduce Convolution Injector (CoIn), an add-on module that injects convolutions which are rich in locality and equivariance biases into a pretrained ViT for effective adaptation in visuo-motor control.

Inductive Bias

Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learning

1 code implementation10 Jun 2024 Donghu Kim, Hojoon Lee, Kyungmin Lee, Dongyoon Hwang, Jaegul Choo

In contrast, objectives focused on learning task-specific knowledge (e. g., identifying agents and fitting reward functions) improve performance in environments similar to the pre-training dataset but not in varied ones.

Atari Games Reinforcement Learning (RL)

Regularized Training with Generated Datasets for Name-Only Transfer of Vision-Language Models

1 code implementation8 Jun 2024 Minho Park, Sunghyun Park, Jooyeol Yun, Jaegul Choo

However, despite the high fidelity of generated images, we observed a significant performance degradation when fine-tuning the model using the generated datasets due to the domain gap between real and generated images.

Diversity Language Modelling +1

Translation Deserves Better: Analyzing Translation Artifacts in Cross-lingual Visual Question Answering

no code implementations4 Jun 2024 ChaeHun Park, Koanho Lee, Hyesu Lim, Jaeseok Kim, Junmo Park, Yu-Jung Heo, Du-Seong Chang, Jaegul Choo

Building a reliable visual question answering~(VQA) system across different languages is a challenging problem, primarily due to the lack of abundant samples for training.

Data Augmentation Machine Translation +3

Do's and Don'ts: Learning Desirable Skills with Instruction Videos

no code implementations1 Jun 2024 Hyunseung Kim, Byungkun Lee, Hojoon Lee, Dongyoon Hwang, Donghu Kim, Jaegul Choo

First, in an instruction learning stage, DoDont leverages action-free instruction videos to train an instruction network to distinguish desirable transitions from undesirable ones.

continuous-control Continuous Control +1

Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks

1 code implementation1 Jun 2024 Hojoon Lee, Hyeonseo Cho, Hyunseung Kim, Donghu Kim, Dugki Min, Jaegul Choo, Clare Lyle

By periodically reinitializing the Hare network to the Tortoise's weights, our method preserves plasticity while retaining general knowledge.

General Knowledge Hippocampus

Enhancing Intrinsic Features for Debiasing via Investigating Class-Discerning Common Attributes in Bias-Contrastive Pair

no code implementations CVPR 2024 Jeonghoon Park, Chaeyeon Chung, Juyoung Lee, Jaegul Choo

In the image classification task, deep neural networks frequently rely on bias attributes that are spuriously correlated with a target class in the presence of dataset bias, resulting in degraded performance when applied to data without bias attributes.

Image Classification

EPIC: Effective Prompting for Imbalanced-Class Data Synthesis in Tabular Data Classification via Large Language Models

1 code implementation15 Apr 2024 Jinhee Kim, Taesung Kim, Jaegul Choo

In this work, we explore the effectiveness of LLMs for generating realistic synthetic tabular data, identifying key prompt design elements to optimize performance.

In-Context Learning Synthetic Data Generation +1

PairEval: Open-domain Dialogue Evaluation with Pairwise Comparison

1 code implementation1 Apr 2024 ChaeHun Park, Minseok Choi, Dohyun Lee, Jaegul Choo

Recent studies proposed evaluation metrics that assess generated responses by considering their relevance to previous dialogue histories.

Dialogue Evaluation

SelfSwapper: Self-Supervised Face Swapping via Shape Agnostic Masked AutoEncoder

no code implementations12 Feb 2024 Jaeseong Lee, Junha Hyung, SOHYUN JEONG, Jaegul Choo

Most previous face swapping approaches have relied on the seesaw game training scheme, also known as the target-oriented approach.

Face Swapping

Self-Supervised Contrastive Learning for Long-term Forecasting

1 code implementation3 Feb 2024 Junwoo Park, Daehoon Gwak, Jaegul Choo, Edward Choi

To this end, our contrastive loss incorporates global autocorrelation held in the whole time series, which facilitates the construction of positive and negative pairs in a self-supervised manner.

Contrastive Learning Time Series +1

When Model Meets New Normals: Test-time Adaptation for Unsupervised Time-series Anomaly Detection

1 code implementation19 Dec 2023 Dongmin Kim, Sunghyun Park, Jaegul Choo

Time-series anomaly detection deals with the problem of detecting anomalous timesteps by learning normality from the sequence of observations.

Anomaly Detection Test-time Adaptation +2

StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On

1 code implementation CVPR 2024 Jeongho Kim, Gyojung Gu, Minho Park, Sunghyun Park, Jaegul Choo

Given a clothing image and a person image, an image-based virtual try-on aims to generate a customized image that appears natural and accurately reflects the characteristics of the clothing image.

Semantic correspondence Virtual Try-on

Cracking the Code of Negative Transfer: A Cooperative Game Theoretic Approach for Cross-Domain Sequential Recommendation

1 code implementation22 Nov 2023 Chung Park, Taesan Kim, Taekyoon Choi, Junui Hong, Yelim Yu, Mincheol Cho, Kyunam Lee, Sungil Ryu, Hyungjun Yoon, Minsung Choi, Jaegul Choo

This paper investigates Cross-Domain Sequential Recommendation (CDSR), a promising method that uses information from multiple domains (more than three) to generate accurate and diverse recommendations, and takes into account the sequential nature of user interactions.

Contrastive Learning Sequential Recommendation

Towards Calibrated Robust Fine-Tuning of Vision-Language Models

1 code implementation3 Nov 2023 Changdae Oh, Hyesu Lim, Mijoo Kim, Dongyoon Han, Sangdoo Yun, Jaegul Choo, Alexander Hauptmann, Zhi-Qi Cheng, Kyungwoo Song

Improving out-of-distribution (OOD) generalization during in-distribution (ID) adaptation is a primary goal of robust fine-tuning of zero-shot models beyond naive fine-tuning.

Autonomous Driving Medical Diagnosis

Expression Domain Translation Network for Cross-domain Head Reenactment

1 code implementation16 Oct 2023 Taewoong Kang, Jeongsik Oh, Jaeseong Lee, Sunghyun Park, Jaegul Choo

Specifically, to maintain the geometric consistency of expressions between the input and output of the expression domain translation network, we employ a 3D geometric-aware loss function that reduces the distances between the vertices in the 3D mesh of the human and anime.

Translation

SimCKP: Simple Contrastive Learning of Keyphrase Representations

1 code implementation12 Oct 2023 Minseok Choi, Chaeheon Gwak, SeHo Kim, Si Hyeong Kim, Jaegul Choo

Keyphrase generation (KG) aims to generate a set of summarizing words or phrases given a source document, while keyphrase extraction (KE) aims to identify them from the text.

Contrastive Learning Keyphrase Extraction +1

PASTA: PArallel Spatio-Temporal Attention with spatial auto-correlation gating for fine-grained crowd flow prediction

no code implementations2 Oct 2023 Chung Park, Junui Hong, Cheonbok Park, Taesan Kim, Minsung Choi, Jaegul Choo

Understanding the movement patterns of objects (e. g., humans and vehicles) in a city is essential for many applications, including city planning and management.

Management

Pre-training Contextual Location Embeddings in Personal Trajectories via Efficient Hierarchical Location Representations

no code implementations2 Oct 2023 Chung Park, Taesan Kim, Junui Hong, Minsung Choi, Jaegul Choo

To tackle this problem, we propose a Geo-Tokenizer, designed to efficiently reduce the number of locations to be trained by representing a location as a combination of several grids at different scales.

Learning to Diversify Neural Text Generation via Degenerative Model

no code implementations22 Sep 2023 Jimin Hong, ChaeHun Park, Jaegul Choo

We then enhance the diversity of the second model by focusing on patterns that the first model fails to learn.

Dialogue Generation Diversity +2

SideGAN: 3D-Aware Generative Model for Improved Side-View Image Synthesis

no code implementations19 Sep 2023 Kyungmin Jo, Wonjoon Jin, Jaegul Choo, Hyunjoon Lee, Sunghyun Cho

In this paper, we propose SideGAN, a novel 3D GAN training method to generate photo-realistic images irrespective of the camera pose, especially for faces of side-view angles.

Image Generation

MagiCapture: High-Resolution Multi-Concept Portrait Customization

no code implementations13 Sep 2023 Junha Hyung, Jaeyo Shin, Jaegul Choo

The main challenge with this task is the absence of ground truth for the composed concepts, leading to a reduction in the quality of the final output and an identity shift of the source subject.

Image Generation Weakly-supervised Learning

Text2Control3D: Controllable 3D Avatar Generation in Neural Radiance Fields using Geometry-Guided Text-to-Image Diffusion Model

no code implementations7 Sep 2023 Sungwon Hwang, Junha Hyung, Jaegul Choo

Our main strategy is to construct the 3D avatar in Neural Radiance Fields (NeRF) optimized with a set of controlled viewpoint-aware images that we generate from ControlNet, whose condition input is the depth map extracted from the input video.

3D Generation NeRF +2

Towards Validating Long-Term User Feedbacks in Interactive Recommendation Systems

no code implementations22 Aug 2023 Hojoon Lee, Dongyoon Hwang, Kyushik Min, Jaegul Choo

In this work, we revisited experiments on IRS with review datasets and compared RL-based models with a simple reward model that greedily recommends the item with the highest one-step reward.

Recommendation Systems Reinforcement Learning (RL)

ST-RAP: A Spatio-Temporal Framework for Real Estate Appraisal

1 code implementation21 Aug 2023 Hojoon Lee, Hawon Jeong, Byungkun Lee, Kyungyup Lee, Jaegul Choo

In this paper, we introduce ST-RAP, a novel Spatio-Temporal framework for Real estate APpraisal.

Graph Neural Network

Label Shift Adapter for Test-Time Adaptation under Covariate and Label Shifts

no code implementations ICCV 2023 Sunghyun Park, Seunghan Yang, Jaegul Choo, Sungrack Yun

Test-time adaptation (TTA) aims to adapt a pre-trained model to the target domain in a batch-by-batch manner during inference.

Test-time Adaptation

Learning to Generate Semantic Layouts for Higher Text-Image Correspondence in Text-to-Image Synthesis

1 code implementation ICCV 2023 Minho Park, Jooyeol Yun, Seunghwan Choi, Jaegul Choo

Our experiments reveal that we can guide text-to-image generation models to be aware of the semantics of different image regions, by training the model to generate semantic labels for each pixel.

multimodal generation Multi-Task Learning +1

Towards Open-Set Test-Time Adaptation Utilizing the Wisdom of Crowds in Entropy Minimization

1 code implementation ICCV 2023 Jungsoo Lee, Debasmit Das, Jaegul Choo, Sungha Choi

To be more specific, entropy minimization attempts to raise the confidence values of an individual sample's prediction, but individual confidence values may rise or fall due to the influence of signals from numerous other predictions (i. e., wisdom of crowds).

Image Classification Semantic Segmentation +1

FaceCLIPNeRF: Text-driven 3D Face Manipulation using Deformable Neural Radiance Fields

no code implementations ICCV 2023 Sungwon Hwang, Junha Hyung, Daejin Kim, Min-Jung Kim, Jaegul Choo

To do so, we first train a scene manipulator, a latent code-conditional deformable NeRF, over a dynamic scene to control a face deformation using the latent code.

3D Face Reconstruction Attribute +2

PixelHuman: Animatable Neural Radiance Fields from Few Images

no code implementations18 Jul 2023 Gyumin Shim, Jaeseong Lee, Junha Hyung, Jaegul Choo

In this paper, we propose PixelHuman, a novel human rendering model that generates animatable human scenes from a few images of a person with unseen identity, views, and poses.

HistRED: A Historical Document-Level Relation Extraction Dataset

1 code implementation10 Jul 2023 Soyoung Yang, Minseok Choi, Youngwoo Cho, Jaegul Choo

To demonstrate the usefulness of our dataset, we propose a bilingual RE model that leverages both Korean and Hanja contexts to predict relations between entities.

Document-level Relation Extraction Relation +1

Local 3D Editing via 3D Distillation of CLIP Knowledge

no code implementations CVPR 2023 Junha Hyung, Sungwon Hwang, Daejin Kim, Hyunji Lee, Jaegul Choo

Specifically, we present three add-on modules of LENeRF, the Latent Residual Mapper, the Attention Field Network, and the Deformation Network, which are jointly used for local manipulations of 3D features by estimating a 3D attention field.

NeRF

On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learning

1 code implementation9 Jun 2023 Hojoon Lee, Koanho Lee, Dongyoon Hwang, Hyunho Lee, Byungkun Lee, Jaegul Choo

To address this issue, we propose a novel URL framework that causally predicts future states while increasing the dimension of the latent manifold by decorrelating the features in the latent space.

Reinforcement Learning (RL) Representation Learning

DEnsity: Open-domain Dialogue Evaluation Metric using Density Estimation

1 code implementation8 May 2023 ChaeHun Park, Seungil Chad Lee, Daniel Rim, Jaegul Choo

Despite the recent advances in open-domain dialogue systems, building a reliable evaluation metric is still a challenging problem.

Contrastive Learning Density Estimation +1

Improving Scene Text Recognition for Character-Level Long-Tailed Distribution

no code implementations31 Mar 2023 Sunghyun Park, Sunghyo Chung, Jungsoo Lee, Jaegul Choo

However, STR models show a large performance degradation on languages with a numerous number of characters (e. g., Chinese and Korean), especially on characters that rarely appear due to the long-tailed distribution of characters in such languages.

Scene Text Recognition

Reference-based Image Composition with Sketch via Structure-aware Diffusion Model

1 code implementation31 Mar 2023 Kangyeol Kim, Sunghyun Park, Junsoo Lee, Jaegul Choo

Recent remarkable improvements in large-scale text-to-image generative models have shown promising results in generating high-fidelity images.

Image Manipulation

RobustSwap: A Simple yet Robust Face Swapping Model against Attribute Leakage

no code implementations28 Mar 2023 Jaeseong Lee, Taewoo Kim, Sunghyun Park, Younggun Lee, Jaegul Choo

However, we observed that previous approaches still suffer from source attribute leakage, where the source image's attributes interfere with the target image's.

Attribute Face Swapping

Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays

1 code implementation9 Mar 2023 Daeun Kyung, Kyungmin Jo, Jaegul Choo, Joonseok Lee, Edward Choi

X-ray computed tomography (CT) is one of the most common imaging techniques used to diagnose various diseases in the medical field.

Computed Tomography (CT) CT Reconstruction

Deep Imbalanced Time-series Forecasting via Local Discrepancy Density

1 code implementation27 Feb 2023 Junwoo Park, Jungsoo Lee, Youngin Cho, Woncheol Shin, Dongmin Kim, Jaegul Choo, Edward Choi

Based on our findings, we propose a reweighting framework that down-weights the losses incurred by abrupt changes and up-weights those by normal states.

Time Series Time Series Forecasting

TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation

no code implementations10 Feb 2023 Hyesu Lim, Byeonggeun Kim, Jaegul Choo, Sungha Choi

In this paper, we identify that CBN and TBN are in a trade-off relationship and present a new test-time normalization (TTN) method that interpolates the statistics by adjusting the importance between CBN and TBN according to the domain-shift sensitivity of each BN layer.

Test-time Adaptation

3D-Aware Generative Model for Improved Side-View Image Synthesis

no code implementations ICCV 2023 Kyungmin Jo, Wonjoon Jin, Jaegul Choo, Hyunjoon Lee, Sunghyun Cho

In this paper, we propose SideGAN, a novel 3D GAN training method to generate photo-realistic images irrespective of the camera pose, especially for faces of side-view angles.

Image Generation

ReFu: Refine and Fuse the Unobserved View for Detail-Preserving Single-Image 3D Human Reconstruction

no code implementations9 Nov 2022 Gyumin Shim, Minsoo Lee, Jaegul Choo

Single-image 3D human reconstruction aims to reconstruct the 3D textured surface of the human body given a single image.

3D Human Reconstruction

Pneg: Prompt-based Negative Response Generation for Dialogue Response Selection Task

no code implementations31 Oct 2022 Nyoungwoo Lee, ChaeHun Park, Ho-Jin Choi, Jaegul Choo

To overcome these limitations, this paper proposes a simple but efficient method for generating adversarial negative responses leveraging a large-scale language model.

Language Modeling Language Modelling +2

Guiding Users to Where to Give Color Hints for Efficient Interactive Sketch Colorization via Unsupervised Region Prioritization

no code implementations25 Oct 2022 Youngin Cho, Junsoo Lee, Soyoung Yang, Juntae Kim, Yeojeong Park, Haneol Lee, Mohammad Azam Khan, Daesik Kim, Jaegul Choo

Existing deep interactive colorization models have focused on ways to utilize various types of interactions, such as point-wise color hints, scribbles, or natural-language texts, as methods to reflect a user's intent at runtime.

Colorization Sketch Colorization

WaveBound: Dynamic Error Bounds for Stable Time Series Forecasting

no code implementations25 Oct 2022 Youngin Cho, Daejin Kim, Dongmin Kim, Mohammad Azam Khan, Jaegul Choo

Time series forecasting has become a critical task due to its high practicality in real-world applications such as traffic, energy consumption, economics and finance, and disease analysis.

Time Series Time Series Forecasting

Enemy Spotted: in-game gun sound dataset for gunshot classification and localization

1 code implementation12 Oct 2022 Junwoo Park, Youngwoo Cho, Gyuhyeon Sim, Hojoon Lee, Jaegul Choo

By exploiting the advantage of the game environment, we construct a gunshot dataset, namely BGG, for the firearm classification and gunshot localization tasks.

Classification Sound Classification

Morphology-Aware Interactive Keypoint Estimation

1 code implementation15 Sep 2022 Jinhee Kim, Taesung Kim, Taewoo Kim, Jaegul Choo, Dong-Wook Kim, Byungduk Ahn, In-Seok Song, Yoon-Ji Kim

To fully automate this procedure, deep-learning-based methods have been widely proposed and have achieved high performance in detecting keypoints in medical images.

Keypoint Estimation

Residual Correction in Real-Time Traffic Forecasting

no code implementations12 Sep 2022 Daejin Kim, Youngin Cho, Dongmin Kim, Cheonbok Park, Jaegul Choo

Extensive experiments on METR-LA and PEMS-BAY demonstrate that our ResCAL can correctly capture the correlation of errors and correct the failures of various traffic forecasting models in event situations.

Reweighting Strategy based on Synthetic Data Identification for Sentence Similarity

1 code implementation COLING 2022 Taehee Kim, ChaeHun Park, Jimin Hong, Radhika Dua, Edward Choi, Jaegul Choo

To analyze this, we first train a classifier that identifies machine-written sentences, and observe that the linguistic features of the sentences identified as written by a machine are significantly different from those of human-written sentences.

Sentence Sentence Embedding +2

A Visual Analytics System for Improving Attention-based Traffic Forecasting Models

no code implementations8 Aug 2022 Seungmin Jin, Hyunwook Lee, Cheonbok Park, Hyeshin Chu, Yunwon Tae, Jaegul Choo, Sungahn Ko

With deep learning (DL) outperforming conventional methods for different tasks, much effort has been devoted to utilizing DL in various domains.

Dynamic Time Warping

Towards Accurate Open-Set Recognition via Background-Class Regularization

no code implementations21 Jul 2022 Wonwoo Cho, Jaegul Choo

To effectively solve the OSR problem, previous studies attempted to limit latent feature space and reject data located outside the limited space via offline analyses, e. g., distance-based feature analyses, or complicated network architectures.

Open Set Learning

iColoriT: Towards Propagating Local Hint to the Right Region in Interactive Colorization by Leveraging Vision Transformer

1 code implementation14 Jul 2022 Jooyeol Yun, Sanghyeon Lee, Minho Park, Jaegul Choo

It is essential for point-interactive colorization methods to appropriately propagate user-provided colors (i. e., user hints) in the entire image to obtain a reasonably colorized image with minimal user effort.

Decoder Image Colorization +1

Mining Multi-Label Samples from Single Positive Labels

no code implementations12 Jun 2022 Youngin Cho, Daejin Kim, Mohammad Azam Khan, Jaegul Choo

Therefore, in this study we explore the practical setting called the single positive setting, where each data instance is annotated by only one positive label with no explicit negative labels.

Improving Evaluation of Debiasing in Image Classification

no code implementations8 Jun 2022 Jungsoo Lee, Juyoung Lee, Sanghun Jung, Jaegul Choo

Based on such issues, this paper 1) proposes an evaluation metric `Align-Conflict (AC) score' for the tuning criterion, 2) includes experimental settings with low bias severity and shows that they are yet to be explored, and 3) unifies the standardized experimental settings to promote fair comparisons between debiasing methods.

Classification Image Classification

CAFA: Class-Aware Feature Alignment for Test-Time Adaptation

no code implementations ICCV 2023 Sanghun Jung, Jungsoo Lee, Nanhee Kim, Amirreza Shaban, Byron Boots, Jaegul Choo

That is, a model does not have a chance to learn test data in a class-discriminative manner, which was feasible in other adaptation tasks (\textit{e. g.,} unsupervised domain adaptation) via supervised losses on the source data.

Test-time Adaptation Unsupervised Domain Adaptation

Revisiting the Importance of Amplifying Bias for Debiasing

1 code implementation29 May 2022 Jungsoo Lee, Jeonghoon Park, Daeyoung Kim, Juyoung Lee, Edward Choi, Jaegul Choo

$f_B$ is trained to focus on bias-aligned samples (i. e., overfitted to the bias) while $f_D$ is mainly trained with bias-conflicting samples by concentrating on samples which $f_B$ fails to learn, leading $f_D$ to be less susceptible to the dataset bias.

Attribute Image Classification

DraftRec: Personalized Draft Recommendation for Winning in Multi-Player Online Battle Arena Games

1 code implementation27 Apr 2022 Hojoon Lee, Dongyoon Hwang, Hyunseung Kim, Byungkun Lee, Jaegul Choo

To alleviate this problem, we propose DraftRec, a novel hierarchical model which recommends characters by considering each player's champion preferences and the interaction between the players.

3D-GIF: 3D-Controllable Object Generation via Implicit Factorized Representations

no code implementations12 Mar 2022 Minsoo Lee, Chaeyeon Chung, Hojun Cho, Minjung Kim, Sanghun Jung, Jaegul Choo, Minhyuk Sung

While NeRF-based 3D-aware image generation methods enable viewpoint control, limitations still remain to be adopted to various 3D applications.

3D geometry Image Generation +1

CG-NeRF: Conditional Generative Neural Radiance Fields

no code implementations7 Dec 2021 Kyungmin Jo, Gyumin Shim, Sanghun Jung, Soyoung Yang, Jaegul Choo

While recent NeRF-based generative models achieve the generation of diverse 3D-aware images, these approaches have limitations when generating images that contain user-specified characteristics.

3D-Aware Image Synthesis Diversity +2

Data Augmentation using Random Image Cropping for High-resolution Virtual Try-On (VITON-CROP)

no code implementations16 Nov 2021 Taewon Kang, Sunghyun Park, Seunghwan Choi, Jaegul Choo

Image-based virtual try-on provides the capacity to transfer a clothing item onto a photo of a given person, which is usually accomplished by warping the item to a given human pose and adjusting the warped item to the person.

Data Augmentation Image Cropping +1

AVocaDo: Strategy for Adapting Vocabulary to Downstream Domain

1 code implementation EMNLP 2021 Jimin Hong, Taehee Kim, Hyesu Lim, Jaegul Choo

During the fine-tuning phase of transfer learning, the pretrained vocabulary remains unchanged, while model parameters are updated.

Language Modeling Language Modelling +1

Improving Face Recognition with Large Age Gaps by Learning to Distinguish Children

1 code implementation22 Oct 2021 Jungsoo Lee, Jooyeol Yun, Sunghyun Park, Yonggyu Kim, Jaegul Choo

Despite the unprecedented improvement of face recognition, existing face recognition models still show considerably low performances in determining whether a pair of child and adult images belong to the same identity.

Face Recognition

Natural Attribute-based Shift Detection

no code implementations18 Oct 2021 Jeonghoon Park, Jimin Hong, Radhika Dua, Daehoon Gwak, Yixuan Li, Jaegul Choo, Edward Choi

Despite the impressive performance of deep networks in vision, language, and healthcare, unpredictable behaviors on samples from the distribution different than the training distribution cause severe problems in deployment.

Attribute Out of Distribution (OOD) Detection

Decoupled Kernel Neural Processes: Neural Network-Parameterized Stochastic Processes using Explicit Data-driven Kernel

no code implementations29 Sep 2021 Daehoon Gwak, Gyubok Lee, Jaehoon Lee, Jaesik Choi, Jaegul Choo, Edward Choi

To address this, we introduce a new neural stochastic processes, Decoupled Kernel Neural Processes (DKNPs), which explicitly learn a separate mean and kernel function to directly model the covariance between output variables in a data-driven manner.

Diversity Gaussian Processes

Distance-Based Background Class Regularization for Open-Set Recognition

no code implementations29 Sep 2021 Wonwoo Cho, Jaegul Choo

In this paper, we propose a novel distance-based BCR method suitable for OSR, which limits the feature space of known-class data in a class-wise manner and then makes background-class samples located far away from the limited feature space.

Open Set Learning

Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift

2 code implementations ICLR 2022 Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, Jaegul Choo

The former normalizes the input to fix its distribution in terms of the mean and variance, while the latter returns the output to the original distribution.

Time Series Time Series Forecasting

BiaSwap: Removing dataset bias with bias-tailored swapping augmentation

no code implementations ICCV 2021 Eungyeup Kim, Jihyeon Lee, Jaegul Choo

Although previous approaches pre-define the type of dataset bias to prevent the network from learning it, recognizing the bias type in the real dataset is often prohibitive.

Action Recognition Facial Attribute Classification

Standardized Max Logits: A Simple yet Effective Approach for Identifying Unexpected Road Obstacles in Urban-Scene Segmentation

1 code implementation ICCV 2021 Sanghun Jung, Jungsoo Lee, Daehoon Gwak, Sungha Choi, Jaegul Choo

However, the distribution of max logits of each predicted class is significantly different from each other, which degrades the performance of identifying unexpected objects in urban-scene segmentation.

Anomaly Detection Scene Segmentation +1

Deep Edge-Aware Interactive Colorization against Color-Bleeding Effects

1 code implementation ICCV 2021 Eungyeup Kim, Sanghyeon Lee, Jeonghoon Park, Somi Choi, Choonghyun Seo, Jaegul Choo

Deep neural networks for automatic image colorization often suffer from the color-bleeding artifact, a problematic color spreading near the boundaries between adjacent objects.

Colorization Image Colorization

Learning Debiased Representation via Disentangled Feature Augmentation

1 code implementation NeurIPS 2021 Jungsoo Lee, Eungyeup Kim, Juyoung Lee, Jihyeon Lee, Jaegul Choo

To this end, our method learns the disentangled representation of (1) the intrinsic attributes (i. e., those inherently defining a certain class) and (2) bias attributes (i. e., peripheral attributes causing the bias), from a large number of bias-aligned samples, the bias attributes of which have strong correlation with the target variable.

Data Augmentation Image Classification

Not Just Compete, but Collaborate: Local Image-to-Image Translation via Cooperative Mask Prediction

no code implementations CVPR 2021 Daejin Kim, Mohammad Azam Khan, Jaegul Choo

While the existing cycle-consistency loss ensures that the image can be translated back, our approach makes the model further preserve the attribute-irrelevant regions even in a single translation to another domain by using the Grad-CAM output computed from the discriminator.

Attribute Decoder +2

An Empirical Experiment on Deep Learning Models for Predicting Traffic Data

no code implementations12 May 2021 Hyunwook Lee, Cheonbok Park, Seungmin Jin, Hyeshin Chu, Jaegul Choo, Sungahn Ko

For example, it is difficult to figure out which models provide state-of-the-art performance, as recently proposed models have often been evaluated with different datasets and experiment environments.

VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization

1 code implementation CVPR 2021 Seunghwan Choi, Sunghyun Park, Minsoo Lee, Jaegul Choo

The task of image-based virtual try-on aims to transfer a target clothing item onto the corresponding region of a person, which is commonly tackled by fitting the item to the desired body part and fusing the warped item with the person.

Virtual Try-on Vocal Bursts Intensity Prediction

K-Hairstyle: A Large-scale Korean Hairstyle Dataset for Virtual Hair Editing and Hairstyle Classification

no code implementations11 Feb 2021 Taewoo Kim, Chaeyeon Chung, Sunghyun Park, Gyojung Gu, Keonmin Nam, Wonzo Choe, Jaesung Lee, Jaegul Choo

In response, we introduce a novel large-scale Korean hairstyle dataset, K-hairstyle, containing 500, 000 high-resolution images.

Translation

Stego Networks: Information Hiding on Deep Neural Networks

no code implementations1 Jan 2021 Youngwoo Cho, Beomsoo Kim, Jaegul Choo

This paper considers neural networks as novel steganographic cover media, which we call stego networks, that can be used to hide one's secret messages.

Cloud Computing

Learning Representations by Contrasting Clusters While Bootstrapping Instances

no code implementations1 Jan 2021 Junsoo Lee, Hojoon Lee, Inkyu Shin, Jaekyoung Bae, In So Kweon, Jaegul Choo

Learning visual representations using large-scale unlabelled images is a holy grail for most of computer vision tasks.

Clustering Contrastive Learning +5

Learning to Generate Questions by Recovering Answer-containing Sentences

no code implementations1 Jan 2021 Seohyun Back, Akhil Kedia, Sai Chetan Chinthakindi, Haejun Lee, Jaegul Choo

We evaluate our method against existing ones in terms of the quality of generated questions as well as the fine-tuned MRC model accuracy after training on the data synthetically generated by our method.

Ranked #3 on Question Generation on SQuAD1.1 (using extra training data)

Machine Reading Comprehension Question Answering +3

Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning

no code implementations ACL 2021 Cheonbok Park, Yunwon Tae, Taehee Kim, Soyoung Yang, Mohammad Azam Khan, Eunjeong Park, Jaegul Choo

To address this issue, this paper presents a novel meta-learning algorithm for unsupervised neural machine translation (UNMT) that trains the model to adapt to another domain by utilizing only a small amount of training data.

General Knowledge Meta-Learning +3

Neural Ordinary Differential Equations for Intervention Modeling

1 code implementation16 Oct 2020 Daehoon Gwak, Gyuhyeon Sim, Michael Poli, Stefano Massaroli, Jaegul Choo, Edward Choi

By interpreting the forward dynamics of the latent representation of neural networks as an ordinary differential equation, Neural Ordinary Differential Equation (Neural ODE) emerged as an effective framework for modeling a system dynamics in the continuous time domain.

Time Series Time Series Analysis

Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Differential Equation

1 code implementation16 Oct 2020 Sunghyun Park, Kangyeol Kim, Junsoo Lee, Jaegul Choo, Joonseok Lee, Sookyung Kim, Edward Choi

Video generation models often operate under the assumption of fixed frame rates, which leads to suboptimal performance when it comes to handling flexible frame rates (e. g., increasing the frame rate of the more dynamic portion of the video as well as handling missing video frames).

Decoder Video Generation

HyperTendril: Visual Analytics for User-Driven Hyperparameter Optimization of Deep Neural Networks

no code implementations4 Sep 2020 Heungseok Park, Yoonsoo Nam, Ji-Hoon Kim, Jaegul Choo

HyperTendril takes a novel approach to effectively steering hyperparameter optimization through an iterative, interactive tuning procedure that allows users to refine the search spaces and the configuration of the AutoML method based on their own insights from given results.

Hyperparameter Optimization

Towards Lightweight Lane Detection by Optimizing Spatial Embedding

1 code implementation arXiv.org 2020 Seokwoo Jung, Sungha Choi, Mohammad Azam Khan, Jaegul Choo

This paper addresses the problem that pixel embedding in proposal-free instance segmentation based lane detection is difficult to optimize.

Clustering Instance Segmentation +4

Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence

no code implementations CVPR 2020 Junsoo Lee, Eungyeup Kim, Yunsung Lee, Dongjun Kim, Jaehyuk Chang, Jaegul Choo

However, it is difficult to prepare for a training data set that has a sufficient amount of semantically meaningful pairs of images as well as the ground truth for a colored image reflecting a given reference (e. g., coloring a sketch of an originally blue car given a reference green car).

Colorization Image Colorization +1

NeurQuRI: Neural Question Requirement Inspector for Answerability Prediction in Machine Reading Comprehension

no code implementations ICLR 2020 Seohyun Back, Sai Chetan Chinthakindi, Akhil Kedia, Haejun Lee, Jaegul Choo

Real-world question answering systems often retrieve potentially relevant documents to a given question through a keyword search, followed by a machine reading comprehension (MRC) step to find the exact answer from them.

answerability prediction Machine Reading Comprehension +1

Cars Can't Fly up in the Sky: Improving Urban-Scene Segmentation via Height-driven Attention Networks

1 code implementation CVPR 2020 Sungha Choi, Joanne T. Kim, Jaegul Choo

This paper exploits the intrinsic features of urban-scene images and proposes a general add-on module, called height-driven attention networks (HANet), for improving semantic segmentation for urban-scene images.

Scene Segmentation Segmentation

Exploring Unlabeled Faces for Novel Attribute Discovery

1 code implementation CVPR 2020 Hyojin Bahng, Sunghyo Chung, Seungjoo Yoo, Jaegul Choo

Despite remarkable success in unpaired image-to-image translation, existing systems still require a large amount of labeled images.

Attribute Image-to-Image Translation +1

ST-GRAT: A Novel Spatio-temporal Graph Attention Network for Accurately Forecasting Dynamically Changing Road Speed

1 code implementation29 Nov 2019 Cheonbok Park, Chunggi Lee, Hyojin Bahng, Yunwon Tae, Kihwan Kim, Seungmin Jin, Sungahn Ko, Jaegul Choo

Predicting road traffic speed is a challenging task due to different types of roads, abrupt speed change and spatial dependencies between roads; it requires the modeling of dynamically changing spatial dependencies among roads and temporal patterns over long input sequences.

Graph Attention

Unpaired Image Translation via Adaptive Convolution-based Normalization

no code implementations29 Nov 2019 Wonwoong Cho, Kangyeol Kim, Eungyeup Kim, Hyunwoo J. Kim, Jaegul Choo

Disentangling content and style information of an image has played an important role in recent success in image translation.

Translation

NL2pSQL: Generating Pseudo-SQL Queries from Under-Specified Natural Language Questions

no code implementations IJCNLP 2019 Fuxiang Chen, Seung-won Hwang, Jaegul Choo, Jung-Woo Ha, Sunghun Kim

Here we describe a new NL2pSQL task to generate pSQL codes from natural language questions on under-specified database issues, NL2pSQL.

Denoising

Learning De-biased Representations with Biased Representations

3 code implementations ICML 2020 Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, Seong Joon Oh

This tactic is feasible in many scenarios where it is much easier to define a set of biased representations than to define and quantify bias.

ASGen: Answer-containing Sentence Generation to Pre-Train Question Generator for Scale-up Data in Question Answering

no code implementations25 Sep 2019 Akhil Kedia, Sai Chetan Chinthakindi, Seohyun Back, Haejun Lee, Jaegul Choo

We evaluate the question generation capability of our method by comparing the BLEU score with existing methods and test our method by fine-tuning the MRC model on the downstream MRC data after training on synthetic data.

Language Modeling Language Modelling +5

Progressive Upsampling Audio Synthesis via Effective Adversarial Training

no code implementations25 Sep 2019 Youngwoo Cho, Minwook Chang, Gerard Jounghyun Kim, Jaegul Choo

This paper proposes a novel generative model called PUGAN, which progressively synthesizes high-quality audio in a raw waveform.

Audio Generation Audio Synthesis +1

SANVis: Visual Analytics for Understanding Self-Attention Networks

no code implementations13 Sep 2019 Cheonbok Park, Inyoup Na, Yongjang Jo, Sungbok Shin, Jaehyo Yoo, Bum Chul Kwon, Jian Zhao, Hyungjong Noh, Yeonsoo Lee, Jaegul Choo

Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications.

Image Captioning Machine Translation +2

Coloring With Limited Data: Few-Shot Colorization via Memory-Augmented Networks

1 code implementation9 Jun 2019 Seungjoo Yoo, Hyojin Bahng, Sunghyo Chung, Junsoo Lee, Jaehyuk Chang, Jaegul Choo

Despite recent advancements in deep learning-based automatic colorization, they are still limited when it comes to few-shot learning.

Colorization Few-Shot Learning +1

What and Where to Translate: Local Mask-based Image-to-Image Translation

no code implementations9 Jun 2019 Wonwoong Cho, Seunghwan Choi, Junwoo Park, David Keetae Park, Tao Qin, Jaegul Choo

First, those methods extract style from an entire exemplar which includes noisy information, which impedes a translation model from properly extracting the intended style of the exemplar.

Image-to-Image Translation Translation

Image-to-Image Translation via Group-wise Deep Whitening-and-Coloring Transformation

2 code implementations CVPR 2019 Wonwoong Cho, Sungha Choi, David Keetae Park, Inkyu Shin, Jaegul Choo

However, applying this approach in image translation is computationally intensive and error-prone due to the expensive time complexity and its non-trivial backpropagation.

Image-to-Image Translation Style Transfer +1

Interpreting Models by Allowing to Ask

no code implementations13 Nov 2018 Sungmin Kang, David Keetae Park, Jaehyuk Chang, Jaegul Choo

Questions convey information about the questioner, namely what one does not know.

Colorization