1 code implementation • ECCV 2020 • Wonho Bae, Junhyug Noh, Gunhee Kim
Weakly supervised object localization (WSOL) is a task of localizing an object in an image only using image-level labels.
1 code implementation • ECCV 2020 • Jaekyeom Kim, Hyoungseok Kim, Gunhee Kim
Few-shot learning is an important research problem that tackles one of the greatest challenges of machine learning: learning a new task from a limited amount of labeled data.
no code implementations • ECCV 2020 • Youngjae Yu, Jongseok Kim, Heeseung Yun, Jiwan Chung, Gunhee Kim
We address character grounding and re-identification in multiple story-based videos like movies and associated text descriptions.
1 code implementation • 10 Feb 2025 • Seokwon Song, Taehyun Lee, Jaewoo Ahn, Jae Hyuk Sung, Gunhee Kim
To address this gap, we introduce the Conceptual Combination with Property Type dataset (CCPT), which consists of 12. 3K annotated triplets of noun phrases, properties, and property types.
1 code implementation • 15 Oct 2024 • Jinyoung Kim, Dayoon Ko, Gunhee Kim
Our benchmark includes dynamic entity mention resolution and entity-centric knowledge-intensive QA task, evaluating entity linking and RAG model's adaptability to new expressions, respectively.
1 code implementation • 5 Oct 2024 • Fatemeh Pesaran Zadeh, Juyeon Kim, Jin-Hwa Kim, Gunhee Kim
Large language models (LLMs) have demonstrated strong capabilities across various language tasks, notably through instruction-tuning methods.
no code implementations • 14 Aug 2024 • Minjung Kim, Hyung Suk Lim, Seung Hwan Kim, Soonyoung Lee, Bumsoo Kim, Gunhee Kim
SIA simultaneously decodes two sets of queries-context query and instance query.
no code implementations • 13 Aug 2024 • Minjung Kim, Hyung Suk Lim, Soonyoung Lee, Bumsoo Kim, Gunhee Kim
3D dense captioning is a task involving the localization of objects and the generation of descriptions for each object in a 3D scene.
no code implementations • 9 Aug 2024 • Heeseung Yun, Ruohan Gao, Ishwarya Ananthabhotla, Anurag Kumar, Jacob Donley, Chao Li, Gunhee Kim, Vamsi Krishna Ithapu, Calvin Murdock
Egocentric videos provide comprehensive contexts for user and scene understanding, spanning multisensory perception to behavioral interaction.
1 code implementation • 9 Jun 2024 • Dayoon Ko, Jinyoung Kim, Hahyeon Choi, Gunhee Kim
In the real world, knowledge is constantly evolving, which can render existing knowledge-based datasets outdated.
1 code implementation • 29 May 2024 • Soochan Lee, Hyeonseong Jeon, Jaehyeon Son, Gunhee Kim
On the other hand, in the more classical literature of statistical machine learning, many models have sequential Bayesian update rules that yield the same learning outcome as the batch training, i. e., they are completely immune to catastrophic forgetting.
1 code implementation • 28 May 2024 • Jaewoo Ahn, Taehyun Lee, Junyoung Lim, Jin-Hwa Kim, Sangdoo Yun, Hwaran Lee, Gunhee Kim
While Large Language Models (LLMs) can serve as agents to simulate human behaviors (i. e., role-playing agents), we emphasize the importance of point-in-time role-playing.
no code implementations • CVPR 2024 • Jinseo Jeong, Junseo Koo, Qimeng Zhang, Gunhee Kim
Existing NeRF-based inverse rendering methods suppose that scenes are exclusively illuminated by distant light sources, neglecting the potential influence of emissive sources within a scene.
1 code implementation • 6 Apr 2024 • Yeda Song, Dongwook Lee, Gunhee Kim
Our COCOA seeks both in-distribution anchors and differences by utilizing the learned reverse dynamics model, encouraging conservatism in the compositional input space for the policy or value function.
no code implementations • 9 Nov 2023 • Jaehyeon Son, Soochan Lee, Gunhee Kim
Over the past decade, deep neural networks have demonstrated significant success using the training scheme that involves mini-batch stochastic gradient descent on extensive datasets.
no code implementations • 24 Oct 2023 • Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi, Maarten Sap
Theory of mind (ToM) evaluations currently focus on testing models using passive narratives that inherently lack interactivity.
1 code implementation • 22 Oct 2023 • Dayoon Ko, Sangho Lee, Gunhee Kim
Our ExFunTube is unique over existing datasets in that our videos cover a wide range of domains with various types of humor that necessitate a multimodal understanding of the content.
1 code implementation • NeurIPS 2023 • Soochan Lee, Jaehyeon Son, Gunhee Kim
That is, we propose to formulate continual learning as a sequence modeling problem, allowing advanced sequence models to be utilized for continual learning.
1 code implementation • ICCV 2023 • Heeseung Yun, Joonil Na, Gunhee Kim
Sound can convey significant information for spatial reasoning in our daily lives.
1 code implementation • ICCV 2023 • Minjung Kim, Junseo Koo, Gunhee Kim
Visual localization is the task of estimating a 6-DoF camera pose of a query image within a provided 3D reference map.
1 code implementation • 12 Jun 2023 • Soochan Lee, Gunhee Kim
Generating intermediate steps, or Chain of Thought (CoT), is an effective way to significantly improve language models' (LM) multi-step reasoning capability.
1 code implementation • 28 May 2023 • Hwaran Lee, Seokhee Hong, Joonsuk Park, Takyoung Kim, Gunhee Kim, Jung-Woo Ha
Large language models (LLMs) learn not only natural text generation abilities but also social biases against different demographic groups from real-world data.
1 code implementation • 28 May 2023 • Hwaran Lee, Seokhee Hong, Joonsuk Park, Takyoung Kim, Meeyoung Cha, Yejin Choi, Byoung Pil Kim, Gunhee Kim, Eun-Ju Lee, Yong Lim, Alice Oh, Sangchul Park, Jung-Woo Ha
The potential social harms that large language models pose, such as generating offensive content and reinforcing biases, are steeply rising.
1 code implementation • 27 May 2023 • Jaewoo Ahn, Yeda Song, Sangdoo Yun, Gunhee Kim
In order to build self-consistent personalized dialogue agents, previous research has mostly focused on textual persona that delivers personal facts or personalities.
1 code implementation • 24 May 2023 • Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, Gunhee Kim
Since the remarkable generation performance of large language models raised ethical and legal concerns, approaches to detect machine-generated text by embedding watermarks are being developed.
1 code implementation • CVPR 2023 • Youngjae Yu, Jiwan Chung, Heeseung Yun, Jack Hessel, Jae Sung Park, Ximing Lu, Rowan Zellers, Prithviraj Ammanabrolu, Ronan Le Bras, Gunhee Kim, Yejin Choi
Language models are capable of commonsense reasoning: while domain-specific models can learn from explicit knowledge (e. g. commonsense graphs [6], ethical norms [25]), and larger models like GPT-3 manifest broad commonsense reasoning capacity.
1 code implementation • 20 Dec 2022 • Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, Yejin Choi
Data scarcity has been a long standing issue in the field of open-domain social dialogue.
no code implementations • 30 Nov 2022 • Yookoon Park, Chris Dongjoo Kim, Gunhee Kim
Based on the Laplace approximation of the latent variable posterior, VLAEs enhance the expressiveness of the posterior while reducing the amortization error.
1 code implementation • 19 Sep 2022 • Heeseung Yun, Sehun Lee, Gunhee Kim
360$^\circ$ video saliency detection is one of the challenging benchmarks for 360$^\circ$ video understanding since non-negligible distortion and discontinuity occur in the projection of any format of 360$^\circ$ videos, and capture-worthy viewpoint in the omnidirectional sphere is ambiguous by nature.
no code implementations • 15 Sep 2022 • Jongbin Won, Minhyuk Song, Gunhee Kim, Jong-Woong Park, Haemin Jeon
A jig for the four beams of structured light is designed and a corresponding alignment process is proposed.
1 code implementation • 25 May 2022 • Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, Maarten Sap
With this dataset, we introduce a dialogue safety detection module, Canary, capable of generating RoTs given conversational context, and a socially-informed dialogue agent, Prost.
Ranked #1 on
Dialogue Safety Prediction
on ProsocialDialog
1 code implementation • 25 May 2022 • Youngjae Yu, Jiwan Chung, Heeseung Yun, Jack Hessel, JaeSung Park, Ximing Lu, Prithviraj Ammanabrolu, Rowan Zellers, Ronan Le Bras, Gunhee Kim, Yejin Choi
Large language models readily adapt to novel settings, even without task-specific training data.
1 code implementation • ICLR 2022 • Seohong Park, Jongwook Choi, Jaekyeom Kim, Honglak Lee, Gunhee Kim
To address this issue, we propose Lipschitz-constrained Skill Discovery (LSD), which encourages the agent to discover more diverse, dynamic, and far-reaching skills.
1 code implementation • 7 Dec 2021 • Yookoon Park, Sangho Lee, Gunhee Kim, David M. Blei
We argue that the deep encoder should maximize its nonlinear expressivity on the data for downstream predictors to take full advantage of its representation power.
1 code implementation • NeurIPS 2021 • Seohong Park, Jaekyeom Kim, Gunhee Kim
SAR can handle the stochasticity of environments by adaptively reacting to changes in states during action repetition.
no code implementations • ICCV 2021 • Chris Dongjoo Kim, Jinseo Jeong, Sangwoo Moon, Gunhee Kim
Continually learning in the real world must overcome many challenges, among which noisy labels are a common and inevitable issue.
1 code implementation • 11 Oct 2021 • Heeseung Yun, Youngjae Yu, Wonsuk Yang, Kangil Lee, Gunhee Kim
However, previous benchmark tasks for panoramic videos are still limited to evaluate the semantic understanding of audio-visual relationships or spherical spatial property in surroundings.
no code implementations • ICLR 2022 • Insu Jeon, YoungJin Park, Gunhee Kim
Learning to infer the conditional posterior model is a key step for robust meta-learning.
1 code implementation • EMNLP 2021 • Hyunwoo Kim, Byeongchang Kim, Gunhee Kim
Empathy is a complex cognitive ability based on the reasoning of others' affective states.
1 code implementation • 27 Jun 2021 • Jaekyeom Kim, Seohong Park, Gunhee Kim
Having the ability to acquire inherent skills from environments without any external rewards or supervision like humans is an important problem.
no code implementations • CVPR 2021 • Youngjae Yu, Jiwan Chung, Heeseung Yun, Jongseok Kim, Gunhee Kim
In this work, we claim that a transitional adaptation task is required between pretraining and finetuning to harmonize the visual encoder and the language model for challenging downstream target tasks like visual storytelling.
Ranked #1 on
Visual Storytelling
on VIST
(ROUGE-L metric, using extra
training data)
1 code implementation • CVPR 2021 • Minui Hong, Jinwoo Choi, Gunhee Kim
In spite of the great success of deep neural networks for many challenging classification tasks, the learned networks are vulnerable to overfitting and adversarial attacks.
no code implementations • NAACL 2021 • Byeongchang Kim, Hyunwoo Kim, Seokhee Hong, Gunhee Kim
In this work, we ask: How robust are fact checking systems on claims in colloquial style?
1 code implementation • ICLR 2021 • Jaekyeom Kim, Minjung Kim, Dongyeon Woo, Gunhee Kim
We propose a novel information bottleneck (IB) method named Drop-Bottleneck, which discretely drops features that are irrelevant to the target variable.
1 code implementation • ICCV 2021 • Sangho Lee, Jiwan Chung, Youngjae Yu, Gunhee Kim, Thomas Breuel, Gal Chechik, Yale Song
We demonstrate that our approach finds videos with high audio-visual correspondence and show that self-supervised models trained on our data achieve competitive performances compared to models trained on existing manually curated datasets.
no code implementations • ICLR 2021 • Youngjae Yu, Sangho Lee, Gunhee Kim, Yale Song
We show that our approach achieves competitive performance on self-supervised learning of video representations with a considerable improvement in speed compared to the traditional methods.
1 code implementation • ICCV 2021 • Hoeseong Kim, Jongseok Kim, Hyungseok Lee, Hyunsung Park, Gunhee Kim
In addition, we propose a cycle consistency module that can potentially improve the performance of any change captioning networks in general by matching the composite feature of the generated caption and before image with the after image feature.
no code implementations • 1 Jan 2021 • Junsoo Ha, Gunhee Kim
As multi-agent systems proliferate in machine learning research, games have attracted much attention as a framework to understand optimization of multiple interacting objectives.
1 code implementation • ICLR 2021 • Myeongjang Pyeon, Jihwan Moon, Taeyoung Hahn, Gunhee Kim
Backward locking and update locking are well-known sources of inefficiency in backpropagation that prevent from concurrently updating layers.
Ranked #25 on
Neural Architecture Search
on ImageNet
1 code implementation • ICCV 2021 • Heeseung Yun, Youngjae Yu, Wonsuk Yang, Kangil Lee, Gunhee Kim
However, previous benchmark tasks for panoramic videos are still limited to evaluate the semantic understanding of audio-visual relationships or spherical spatial property in surroundings.
no code implementations • ICLR 2021 • Sangho Lee, Youngjae Yu, Gunhee Kim, Thomas Breuel, Jan Kautz, Yale Song
The recent success of Transformers in the language domain has motivated adapting it to a multimodal setting, where a new visual model is trained in tandem with an already pretrained language model.
no code implementations • 17 Oct 2020 • Yunchao Wei, Shuai Zheng, Ming-Ming Cheng, Hang Zhao, LiWei Wang, Errui Ding, Yi Yang, Antonio Torralba, Ting Liu, Guolei Sun, Wenguan Wang, Luc van Gool, Wonho Bae, Junhyug Noh, Jinhwan Seo, Gunhee Kim, Hao Zhao, Ming Lu, Anbang Yao, Yiwen Guo, Yurong Chen, Li Zhang, Chuangchuang Tan, Tao Ruan, Guanghua Gu, Shikui Wei, Yao Zhao, Mariia Dobko, Ostap Viniavskyi, Oles Dobosevych, Zhendong Wang, Zhenyuan Chen, Chen Gong, Huanqing Yan, Jun He
The purpose of the Learning from Imperfect Data (LID) workshop is to inspire and facilitate the research in developing novel approaches that would harness the imperfect data and improve the data-efficiency during training.
1 code implementation • ECCV 2020 • Chris Dongjoo Kim, Jinseo Jeong, Gunhee Kim
Continual learning from a sequential stream of data is a crucial challenge for machine learning research.
no code implementations • WS 2020 • Hankyol Lee, Youngjae Yu, Gunhee Kim
We present a novel data augmentation technique, CRA (Contextual Response Augmentation), which utilizes conversational context to generate meaningful samples for training.
1 code implementation • EMNLP 2020 • Hyunwoo Kim, Byeongchang Kim, Gunhee Kim
Results on Dialogue NLI (Welleck et al., 2019) and PersonaChat (Zhang et al., 2018) dataset show that our approach reduces contradiction and improves consistency of existing dialogue models.
1 code implementation • 27 Mar 2020 • Youngjae Yu, Seunghwan Lee, Yuncheol Choi, Gunhee Kim
In order to learn an effective image-text composition for the data in the fashion domain, our model proposes two key components as follows.
Ranked #18 on
Image Retrieval
on Fashion IQ
3 code implementations • ICLR 2020 • Byeongchang Kim, Jaewoo Ahn, Gunhee Kim
Knowledge-grounded dialogue is a task of generating an informative response based on both discourse context and external knowledge.
1 code implementation • ICLR 2020 • Soochan Lee, Junsoo Ha, Dongsu Zhang, Gunhee Kim
Despite the growing interest in continual learning, most of its contemporary works have been studied in a rather restricted setting where tasks are clearly distinguishable, and task boundaries are known during training.
1 code implementation • NeurIPS 2019 • Taeyoung Hahn, Myeongjang Pyeon, Gunhee Kim
Capsule networks have recently gained a great deal of interest as a new architecture of neural networks that can be more robust to input perturbations than similar-sized CNNs.
no code implementations • NAACL 2019 • Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, Gunhee Kim
We explore the problem of Audio Captioning: generating natural language description for any kind of audio in the wild, which has been surprisingly unexplored in previous research.
Ranked #14 on
Audio captioning
on AudioCaps
2 code implementations • ICLR 2019 • Insu Jeon, Wonkwang Lee, Gunhee Kim
IB-GAN objective is similar to that of InfoGAN but has a crucial difference; a capacity regularization for mutual information is adopted, thanks to which the generator of IB-GAN can harness a latent representation in disentangled and interpretable manner.
no code implementations • ICLR 2019 • Soochan Lee, Junsoo Ha, Gunhee Kim
Recent advances in conditional image generation tasks, such as image-to-image translation and image inpainting, are largely accounted to the success of conditional GAN models, which are often optimized by the joint use of the GAN loss with the reconstruction loss.
1 code implementation • ICLR 2019 • Seil Na, Yo Joong Choe, Dong-Hyun Lee, Gunhee Kim
Although deep convolutional networks have achieved improved performance in many natural language tasks, they have been treated as black boxes because they are difficult to interpret.
1 code implementation • NAACL 2019 • Byeongchang Kim, Hyunwoo Kim, Gunhee Kim
We address the problem of abstractive summarization in two directions: proposing a novel dataset and a new model.
2 code implementations • ECCV 2018 • Youngjae Yu, Jongseok Kim, Gunhee Kim
We present an approach named JSFusion (Joint Sequence Fusion) that can measure semantic similarity between any pairs of multimodal sequence data (e. g. a video clip and a language sentence).
Ranked #34 on
Video Retrieval
on LSMDC
no code implementations • ICML 2018 • Yunseok Jang, Gunhee Kim, Yale Song
Video prediction aims to generate realistic future frames by learning dynamic visual patterns.
no code implementations • CVPR 2018 • Sang-ho Lee, Jinyoung Sung, Youngjae Yu, Gunhee Kim
Second, we evaluate the temporal summarization with a newly collected 360° video dataset.
no code implementations • CVPR 2018 • Junhyug Noh, Soochan Lee, Beomsu Kim, Gunhee Kim
We propose methods of addressing two critical issues of pedestrian detection: (i) occlusion of target objects as false negative failure, and (ii) confusion with hard negative examples like vertical structures as false positive failure.
no code implementations • CVPR 2018 • Sang-ho Lee, Jinyoung Sung, Youngjae Yu, Gunhee Kim
Second, we evaluate the temporal summarization with a newly collected 360{\deg} video dataset.
4 code implementations • NAACL 2018 • Yookoon Park, Jaemin Cho, Gunhee Kim
To solve the degeneration problem, we propose a novel model named Variational Hierarchical Conversation RNNs (VHCR), involving two key ideas of (1) using a hierarchical structure of latent variables, and (2) exploiting an utterance drop regularization.
1 code implementation • ICLR 2018 • Youngjin Kim, Minjung Kim, Gunhee Kim
We propose an approach to address two issues that commonly occur during training of unsupervised GANs.
no code implementations • 31 Jan 2018 • Youngjae Yu, Sang-ho Lee, Joonil Na, Jaeyun Kang, Gunhee Kim
We address the problem of highlight detection from a 360 degree video by summarizing it both spatially and temporally.
1 code implementation • ICCV 2017 • Seil Na, Sang-ho Lee, Ji-Sung Kim, Gunhee Kim
We propose a novel memory network model named Read-Write Memory Network (RWMN) to perform question and answering tasks for large-scale, multimodal movie story understanding.
Ranked #3 on
Video Story QA
on MovieQA
no code implementations • ICML 2017 • Juyong Kim, Yookoon Park, Gunhee Kim, Sung Ju Hwang
We propose a novel deep neural network that is both lightweight and effectively structured for model parallelization.
no code implementations • CVPR 2017 • Youngjae Yu, Jongwook Choi, Yeonhwa Kim, Kyung Yoo, Sang-Hun Lee, Gunhee Kim
The attention mechanisms in deep neural networks are inspired by human's attention that sequentially focuses on the most relevant parts of the information over time to generate prediction output.
1 code implementation • 24 Jun 2017 • Seil Na, Youngjae Yu, Sang-ho Lee, Ji-Sung Kim, Gunhee Kim
YouTube-8M is the largest video dataset for multi-label video classification.
2 code implementations • CVPR 2017 • Cesc Chunseong Park, Byeongchang Kim, Gunhee Kim
We address personalization issues of image captioning, which have not been discussed yet in previous research.
2 code implementations • CVPR 2017 • Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, Gunhee Kim
In this paper, we focus on extending VQA to the video domain and contribute to the literature in three important ways.
Ranked #33 on
Visual Question Answering (VQA)
on MSRVTT-QA
no code implementations • CVPR 2017 • Youngjae Yu, Hyungjin Ko, Jongwook Choi, Gunhee Kim
We propose a high-level concept word detector that can be integrated with any video-to-language models.
Ranked #37 on
Video Retrieval
on LSMDC
no code implementations • 19 Dec 2015 • Hao Zhang, Zhiting Hu, Jinliang Wei, Pengtao Xie, Gunhee Kim, Qirong Ho, Eric Xing
To investigate how to adapt existing frameworks to efficiently support distributed GPUs, we propose Poseidon, a scalable system architecture for distributed inter-machine communication in existing DL frameworks.
no code implementations • ICCV 2015 • Bo Xiong, Gunhee Kim, Leonid Sigal
To address this, we propose a storyline representation that expresses an egocentric video as a set of jointly inferred, through MRF inference, story elements comprising of actors, locations, supporting objects and events, depicted on a timeline.
1 code implementation • NeurIPS 2015 • Cesc C. Park, Gunhee Kim
We propose an approach for generating a sequence of natural sentences for an image stream.
no code implementations • CVPR 2015 • Gunhee Kim, Seungwhan Moon, Leonid Sigal
We alternate between solving the two coupled latent SVM problems, by first fixing the summarization and solving for the alignment from blog images to photo streams and vice versa.
no code implementations • CVPR 2015 • Gunhee Kim, Seungwhan Moon, Leonid Sigal
While most previous work has dealt with the relations between a natural language sentence and an image or a video, our work extends to the relations between paragraphs and image sequences.
no code implementations • CVPR 2014 • Gunhee Kim, Leonid Sigal, Eric P. Xing
The reconstruction of storyline graphs is formulated as the inference of sparse time-varying directed graphs from a set of photo streams with assistance of videos.
no code implementations • CVPR 2014 • Gunhee Kim, Eric P. Xing
In this paper, we investigate an approach for reconstructing storyline graphs from large-scale collections of Internet images, and optionally other side information such as friendship graphs.
no code implementations • CVPR 2013 • Gunhee Kim, Eric P. Xing
To this end, we design a scalable message-passing based optimization framework to jointly achieve both tasks for the whole input image set at once.
no code implementations • NeurIPS 2009 • Gunhee Kim, Antonio Torralba
This paper proposes a fast and scalable alternating optimization technique to detect regions of interest (ROIs) in cluttered Web images without labels.