Search Results for author: Kihyuk Sohn

Found 61 papers, 26 papers with code

DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flow

no code implementations22 Mar 2024 Kyungmin Lee, Kihyuk Sohn, Jinwoo Shin

Recent progress in text-to-3D generation has been achieved through the utilization of score distillation methods: they make use of the pre-trained text-to-image (T2I) diffusion models by distilling via the diffusion model training objective.

Image-to-Image Translation Text to 3D

Direct Consistency Optimization for Compositional Text-to-Image Personalization

no code implementations19 Feb 2024 Kyungmin Lee, Sangkyung Kwak, Kihyuk Sohn, Jinwoo Shin

In particular, our method results in a superior Pareto frontier to the baselines.

Unsupervised LLM Adaptation for Question Answering

no code implementations16 Feb 2024 Kuniaki Saito, Kihyuk Sohn, Chen-Yu Lee, Yoshitaka Ushiku

In this task, we leverage a pre-trained LLM, a publicly available QA dataset (source data), and unlabeled documents from the target domain.

Question Answering

Instruct-Imagen: Image Generation with Multi-modal Instruction

no code implementations3 Jan 2024 Hexiang Hu, Kelvin C. K. Chan, Yu-Chuan Su, Wenhu Chen, Yandong Li, Kihyuk Sohn, Yang Zhao, Xue Ben, Boqing Gong, William Cohen, Ming-Wei Chang, Xuhui Jia

We introduce *multi-modal instruction* for image generation, a task representation articulating a range of generation intents with precision.

Image Generation Retrieval

Label Budget Allocation in Multi-Task Learning

no code implementations24 Aug 2023 Ximeng Sun, Kihyuk Sohn, Kate Saenko, Clayton Mellina, Xiao Bian

How should the label budget (i. e. the amount of money spent on labeling) be allocated among different tasks to achieve optimal multi-task performance?

Multi-Task Learning

Collaborative Score Distillation for Consistent Visual Synthesis

no code implementations4 Jul 2023 Subin Kim, Kyungmin Lee, June Suk Choi, Jongheon Jeong, Kihyuk Sohn, Jinwoo Shin

Generative priors of large-scale text-to-image diffusion models enable a wide range of new generation and editing applications on diverse visual modalities.

Learning Disentangled Prompts for Compositional Image Synthesis

no code implementations1 Jun 2023 Kihyuk Sohn, Albert Shaw, Yuan Hao, Han Zhang, Luisa Polania, Huiwen Chang, Lu Jiang, Irfan Essa

We study domain-adaptive image synthesis, the problem of teaching pretrained image generative models a new style or concept from as few as one image to synthesize novel images, to better understand the compositional image synthesis.

Domain Adaptation Image Generation +1

Video Probabilistic Diffusion Models in Projected Latent Space

1 code implementation CVPR 2023 Sihyun Yu, Kihyuk Sohn, Subin Kim, Jinwoo Shin

Specifically, PVDM is composed of two components: (a) an autoencoder that projects a given video as 2D-shaped latent vectors that factorize the complex cubic structure of video pixels and (b) a diffusion model architecture specialized for our new factorized latent space and the training/sampling procedure to synthesize videos of arbitrary length with a single model.

Video Generation

MaskSketch: Unpaired Structure-guided Masked Image Generation

2 code implementations CVPR 2023 Dina Bashkirova, Jose Lezama, Kihyuk Sohn, Kate Saenko, Irfan Essa

We show that intermediate self-attention maps of a masked generative transformer encode important structural information of the input image, such as scene layout and object shape, and we propose a novel sampling method based on this observation to enable structure-guided generation.

Conditional Image Generation Image-to-Image Translation +2

Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval

1 code implementation CVPR 2023 Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, Tomas Pfister

Existing methods rely on supervised learning of CIR models using labeled triplets consisting of the query image, text specification, and the target image.

Attribute Retrieval +2

Visual Prompt Tuning for Generative Transfer Learning

1 code implementation CVPR 2023 Kihyuk Sohn, Yuan Hao, José Lezama, Luisa Polania, Huiwen Chang, Han Zhang, Irfan Essa, Lu Jiang

We base our framework on state-of-the-art generative vision transformers that represent an image as a sequence of visual tokens to the autoregressive or non-autoregressive transformers.

Image Generation Transfer Learning +1

Prefix Conditioning Unifies Language and Label Supervision

no code implementations CVPR 2023 Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, Tomas Pfister

In experiments, we show that this simple technique improves the performance in zero-shot image recognition accuracy and robustness to the image-level distribution shift.

Classification Contrastive Learning +2

Federated Semi-Supervised Learning with Prototypical Networks

1 code implementation27 May 2022 Woojung Kim, Keondo Park, Kihyuk Sohn, Raphael Shu, Hyung-Sin Kim

Compared to a FSSL approach based on weight sharing, the prototype-based inter-client knowledge sharing significantly reduces both communication and computation costs, enabling more frequent knowledge sharing between more clients for better accuracy.

Federated Learning

Towards Group Robustness in the presence of Partial Group Labels

no code implementations10 Jan 2022 Vishnu Suresh Lokhande, Kihyuk Sohn, Jinsung Yoon, Madeleine Udell, Chen-Yu Lee, Tomas Pfister

Such a requirement is impractical in situations where the data labeling efforts for minority or rare groups are significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information.

Anomaly Clustering: Grouping Images into Coherent Clusters of Anomaly Types

2 code implementations21 Dec 2021 Kihyuk Sohn, Jinsung Yoon, Chun-Liang Li, Chen-Yu Lee, Tomas Pfister

We define a distance function between images, each of which is represented as a bag of embeddings, by the Euclidean distance between weighted averaged embeddings.

Anomaly Detection Clustering +2

Unifying Distribution Alignment as a Loss for Imbalanced Semi-supervised Learning

no code implementations29 Sep 2021 Justin Lazarow, Kihyuk Sohn, Chun-Liang Li, Zizhao Zhang, Chen-Yu Lee, Tomas Pfister

While remarkable progress in imbalanced supervised learning has been made recently, less attention has been given to the setting of imbalanced semi-supervised learning (SSL) where not only is a few labeled data provided, but the underlying data distribution can be severely imbalanced.

Pseudo Label

Invariant Learning with Partial Group Labels

no code implementations29 Sep 2021 Vishnu Suresh Lokhande, Kihyuk Sohn, Jinsung Yoon, Madeleine Udell, Chen-Yu Lee, Tomas Pfister

Such a requirement is impractical in situations where the data labelling efforts for minority or rare groups is significantly laborious or where the individuals comprising the dataset choose to conceal sensitive information.

Object-aware Contrastive Learning for Debiased Scene Representation

1 code implementation NeurIPS 2021 Sangwoo Mo, Hyunwoo Kang, Kihyuk Sohn, Chun-Liang Li, Jinwoo Shin

Contrastive self-supervised learning has shown impressive results in learning visual representations from unlabeled images by enforcing invariance against different data augmentations.

Contrastive Learning Object +2

Controlling Neural Networks with Rule Representations

1 code implementation NeurIPS 2021 Sungyong Seo, Sercan O. Arik, Jinsung Yoon, Xiang Zhang, Kihyuk Sohn, Tomas Pfister

The key aspect of DeepCTRL is that it does not require retraining to adapt the rule strength -- at inference, the user can adjust it based on the desired operation point on accuracy vs. rule verification ratio.

Decision Making

AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation

5 code implementations ICLR 2022 David Berthelot, Rebecca Roelofs, Kihyuk Sohn, Nicholas Carlini, Alex Kurakin

We extend semi-supervised learning to the problem of domain adaptation to learn significantly higher-accuracy models that train on one data distribution and test on a different one.

Semi-supervised Domain Adaptation Unsupervised Domain Adaptation

CutPaste: Self-Supervised Learning for Anomaly Detection and Localization

2 code implementations CVPR 2021 Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, Tomas Pfister

We aim at constructing a high performance model for defect detection that detects unknown anomalous patterns of an image without anomalous data.

Data Augmentation Defect Detection +4

Assessing Post-Disaster Damage from Satellite Imagery using Semi-Supervised Learning Techniques

no code implementations24 Nov 2020 Jihyeon Lee, Joseph Z. Xu, Kihyuk Sohn, Wenhan Lu, David Berthelot, Izzeddin Gur, Pranav Khaitan, Ke-Wei, Huang, Kyriacos Koupparis, Bernhard Kowatsch

To respond to disasters such as earthquakes, wildfires, and armed conflicts, humanitarian organizations require accurate and timely data in the form of damage assessments, which indicate what buildings and population centers have been most affected.

BIG-bench Machine Learning Disaster Response +1

Improving Face Recognition by Clustering Unlabeled Faces in the Wild

no code implementations ECCV 2020 Aruni RoyChowdhury, Xiang Yu, Kihyuk Sohn, Erik Learned-Miller, Manmohan Chandraker

While deep face recognition has benefited significantly from large-scale labeled data, current research is focused on leveraging unlabeled data to further boost performance, reducing the cost of human annotation.

Clustering Face Clustering +3

A Simple Semi-Supervised Learning Framework for Object Detection

7 code implementations10 May 2020 Kihyuk Sohn, Zizhao Zhang, Chun-Liang Li, Han Zhang, Chen-Yu Lee, Tomas Pfister

Semi-supervised learning (SSL) has a potential to improve the predictive performance of machine learning models using unlabeled data.

Ranked #13 on Semi-Supervised Object Detection on COCO 100% labeled data (using extra training data)

Data Augmentation Image Classification +4

ReMixMatch: Semi-Supervised Learning with Distribution Matching and Augmentation Anchoring

1 code implementation ICLR 2020 David Berthelot, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, Colin Raffel

We improve the recently-proposed ``MixMatch semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring.

Adversarial Learning of Privacy-Preserving and Task-Oriented Representations

no code implementations22 Nov 2019 Taihong Xiao, Yi-Hsuan Tsai, Kihyuk Sohn, Manmohan Chandraker, Ming-Hsuan Yang

For instance, there could be a potential privacy risk of machine learning systems via the model inversion attack, whose goal is to reconstruct the input data from the latent representation of deep networks.

Attribute BIG-bench Machine Learning +2

Adaptation Across Extreme Variations using Unlabeled Domain Bridges

no code implementations5 Jun 2019 Shuyang Dai, Kihyuk Sohn, Yi-Hsuan Tsai, Lawrence Carin, Manmohan Chandraker

We tackle an unsupervised domain adaptation problem for which the domain discrepancy between labeled source and unlabeled target domains is large, due to many factors of inter and intra-domain variation.

Object Recognition Semantic Segmentation +1

Domain Adaptation for Structured Output via Disentangled Patch Representations

no code implementations ICLR 2019 Yi-Hsuan Tsai, Kihyuk Sohn, Samuel Schulter, Manmohan Chandraker

To this end, we propose to learn discriminative feature representations of patches based on label histograms in the source domain, through the construction of a disentangled space.

Domain Adaptation Semantic Segmentation

Unsupervised Domain Adaptation for Distance Metric Learning

no code implementations ICLR 2019 Kihyuk Sohn, Wenling Shang, Xiang Yu, Manmohan Chandraker

Unsupervised domain adaptation is a promising avenue to enhance the performance of deep neural networks on a target domain, using labels only from a source domain.

Face Recognition Metric Learning +1

Active Adversarial Domain Adaptation

no code implementations16 Apr 2019 Jong-Chyi Su, Yi-Hsuan Tsai, Kihyuk Sohn, Buyu Liu, Subhransu Maji, Manmohan Chandraker

Our approach, active adversarial domain adaptation (AADA), explores a duality between two related problems: adversarial domain alignment and importance sampling for adapting models across domains.

Active Learning Domain Adaptation +3

Domain Adaptation for Structured Output via Discriminative Patch Representations

8 code implementations ICCV 2019 Yi-Hsuan Tsai, Kihyuk Sohn, Samuel Schulter, Manmohan Chandraker

Predicting structured outputs such as semantic segmentation relies on expensive per-pixel annotations to learn supervised models like convolutional neural networks.

Domain Adaptation Segmentation +2

Learning Gibbs-regularized GANs with variational discriminator reparameterization

no code implementations27 Sep 2018 Nicholas Rhinehart, Anqi Liu, Kihyuk Sohn, Paul Vernaza

We propose a novel approach to regularizing generative adversarial networks (GANs) leveraging learned {\em structured Gibbs distributions}.

Trajectory Forecasting

Feature Transfer Learning for Deep Face Recognition with Under-Represented Data

no code implementations23 Mar 2018 Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, Manmohan Chandraker

In this paper, we propose a center-based feature transfer framework to augment the feature space of under-represented subjects from the regular subjects that have sufficiently diverse samples.

Disentanglement Face Recognition +1

Gotta Adapt 'Em All: Joint Pixel and Feature-Level Domain Adaptation for Recognition in the Wild

1 code implementation CVPR 2019 Luan Tran, Kihyuk Sohn, Xiang Yu, Xiaoming Liu, Manmohan Chandraker

Recent developments in deep domain adaptation have allowed knowledge transfer from a labeled source domain to an unlabeled target domain at the level of intermediate features or input pixels.

Attribute Domain Adaptation +2

Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos

no code implementations ICCV 2017 Kihyuk Sohn, Sifei Liu, Guangyu Zhong, Xiang Yu, Ming-Hsuan Yang, Manmohan Chandraker

Despite rapid advances in face recognition, there remains a clear gap between the performance of still image-based face recognition and video-based face recognition, due to the vast difference in visual quality between the domains and the difficulty of curating diverse large-scale video datasets.

Data Augmentation Face Recognition +1

Channel-Recurrent Autoencoding for Image Modeling

no code implementations12 Jun 2017 Wenling Shang, Kihyuk Sohn, Yuandong Tian

Despite recent successes in synthesizing faces and bedrooms, existing generative models struggle to capture more complex image types, potentially due to the oversimplification of their latent space constructions.

Towards Large-Pose Face Frontalization in the Wild

no code implementations ICCV 2017 Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, Manmohan Chandraker

Despite recent advances in face recognition using deep learning, severe accuracy drops are observed for large pose variations in unconstrained environments.

3D Reconstruction Face Recognition +1

Reconstruction-Based Disentanglement for Pose-invariant Face Recognition

no code implementations ICCV 2017 Xi Peng, Xiang Yu, Kihyuk Sohn, Dimitris Metaxas, Manmohan Chandraker

Finally, we propose a new feature reconstruction metric learning to explicitly disentangle identity and pose, by demanding alignment between the feature reconstructions through various combinations of identity and pose features, which is obtained from two images of the same subject.

Disentanglement Face Recognition +2

Improved Deep Metric Learning with Multi-class N-pair Loss Objective

1 code implementation NeurIPS 2016 Kihyuk Sohn

Deep metric learning has gained much popularity in recent years, following the success of deep learning.

Clustering Face Verification +3

Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units

2 code implementations16 Mar 2016 Wenling Shang, Kihyuk Sohn, Diogo Almeida, Honglak Lee

Recently, convolutional neural networks (CNNs) have been used as a powerful tool to solve many problems of machine learning and computer vision.

Learning Structured Output Representation using Deep Conditional Generative Models

1 code implementation NeurIPS 2015 Kihyuk Sohn, Honglak Lee, Xinchen Yan

The model is trained efficiently in the framework of stochastic gradient variational Bayes, and allows a fast prediction using stochastic feed-forward inference.

Semantic Segmentation Structured Prediction

Improving Object Detection with Deep Convolutional Networks via Bayesian Optimization and Structured Prediction

no code implementations CVPR 2015 Yuting Zhang, Kihyuk Sohn, Ruben Villegas, Gang Pan, Honglak Lee

Object detection systems based on the deep convolutional neural network (CNN) have recently made ground- breaking advances on several object detection benchmarks.

Bayesian Optimization Object +3

Improved Multimodal Deep Learning with Variation of Information

no code implementations NeurIPS 2014 Kihyuk Sohn, Wenling Shang, Honglak Lee

Deep learning has been successfully applied to multimodal representation learning problems, with a common strategy to learning joint representations that are shared across multiple modalities on top of layers of modality-specific networks.

Multimodal Deep Learning Representation Learning

Augmenting CRFs with Boltzmann Machine Shape Priors for Image Labeling

no code implementations CVPR 2013 Andrew Kae, Kihyuk Sohn, Honglak Lee, Erik Learned-Miller

Although the CRF is a good baseline labeler, we show how an RBM can be added to the architecture to provide a global shape bias that complements the local modeling provided by the CRF.

Attribute Superpixels

Cannot find the paper you are looking for? You can Submit a new open access paper.