Search Results for author: Chunyuan Li

Found 71 papers, 34 papers with code

Rethinking Sentiment Style Transfer

no code implementations Findings (EMNLP) 2021 Ping Yu, Yang Zhao, Chunyuan Li, Changyou Chen

To overcome this issue, we propose a graph-based method to extract attribute content and attribute-independent content from input sentences in the YELP dataset and IMDB dataset.

Style Transfer Text Style Transfer

RegionCLIP: Region-based Language-Image Pretraining

no code implementations16 Dec 2021 Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, Jianfeng Gao

However, we show that directly applying such models to recognize image regions for object detection leads to poor performance due to a domain shift: CLIP was trained to match an image as a whole to a text description, without capturing the fine-grained alignment between image regions and text spans.

Image Classification Object Detection +1

A Generic Approach for Enhancing GANs by Regularized Latent Optimization

no code implementations7 Dec 2021 Yufan Zhou, Chunyuan Li, Changyou Chen, Jinhui Xu

With the rapidly growing model complexity and data volume, training deep generative models (DGMs) for better performance has becoming an increasingly more important challenge.

Image Inpainting Text-to-Image Generation +1

Grounded Language-Image Pre-training

1 code implementation7 Dec 2021 Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, Jianfeng Gao

The unification brings two benefits: 1) it allows GLIP to learn from both detection and grounding data to improve both tasks and bootstrap a good grounding model; 2) GLIP can leverage massive image-text pairs by generating grounding boxes in a self-training fashion, making the learned representation semantic-rich.

 Ranked #1 on Phrase Grounding on Flickr30k Entities Test (using extra training data)

Object Detection Phrase Grounding

Focal Attention for Long-Range Interactions in Vision Transformers

no code implementations NeurIPS 2021 Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiyang Dai, Bin Xiao, Lu Yuan, Jianfeng Gao

With focal attention, we propose a new variant of Vision Transformer models, called Focal Transformers, which achieve superior performance over the state-of-the-art (SoTA) Vision Transformers on a range of public image classification and object detection benchmarks.

Image Classification Object Detection +1

LAFITE: Towards Language-Free Training for Text-to-Image Generation

no code implementations27 Nov 2021 Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, Tong Sun

One of the major challenges in training text-to-image generation models is the need of a large number of high-quality image-text pairs.

Zero-Shot Text-to-Image Generation

Florence: A New Foundation Model for Computer Vision

no code implementations22 Nov 2021 Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, Zicheng Liu, Yumao Lu, Yu Shi, Lijuan Wang, JianFeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Luowei Zhou, Pengchuan Zhang

Computer vision foundation models, which are trained on diverse, large-scale dataset and can be adapted to a wide range of downstream tasks, are critical for this mission to solve real-world computer vision applications.

Action Classification Action Recognition +9

SYNERGY: Building Task Bots at Scale Using Symbolic Knowledge and Machine Teaching

no code implementations21 Oct 2021 Baolin Peng, Chunyuan Li, Zhu Zhang, Jinchao Li, Chenguang Zhu, Jianfeng Gao

We propose SYNERGY, a hybrid learning framework where a task bot is developed in two steps: (i) Symbolic knowledge to neural networks: Large amounts of simulated dialog sessions are generated based on task-specific symbolic knowledge which is represented as a task schema consisting of dialog flows and task-oriented databases.

Focal Self-attention for Local-Global Interactions in Vision Transformers

3 code implementations1 Jul 2021 Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiyang Dai, Bin Xiao, Lu Yuan, Jianfeng Gao

With focal self-attention, we propose a new variant of Vision Transformer models, called Focal Transformer, which achieves superior performance over the state-of-the-art vision Transformers on a range of public image classification and object detection benchmarks.

Image Classification Instance Segmentation +2

Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation

1 code implementation ICCV 2021 Jinyu Yang, Chunyuan Li, Weizhi An, Hehuan Ma, Yuzhi Guo, Yu Rong, Peilin Zhao, Junzhou Huang

Recent studies imply that deep neural networks are vulnerable to adversarial examples -- inputs with a slight but intentional perturbation are incorrectly classified by the network.

Semantic Segmentation Unsupervised Domain Adaptation

Contrastive Attraction and Contrastive Repulsion for Representation Learning

no code implementations8 May 2021 Huangjie Zheng, Xu Chen, Jiangchao Yao, Hongxia Yang, Chunyuan Li, Ya zhang, Hao Zhang, Ivor Tsang, Jingren Zhou, Mingyuan Zhou

Extensive large-scale experiments on standard vision tasks show that CACR not only consistently outperforms existing CL methods on benchmark datasets in representation learning, but also provides interpretable contrastive weights, demonstrating the efficacy of the proposed doubly contrastive strategy.

Contrastive Learning Representation Learning

Partition-Guided GANs

1 code implementation CVPR 2021 Mohammadreza Armandpour, Ali Sadeghian, Chunyuan Li, Mingyuan Zhou

We formulate two desired criteria for the space partitioner that aid the training of our mixture of generators: 1) to produce connected partitions and 2) provide a proxy of distance between partitions and data samples, along with a direction for reducing that distance.

Leveraging User Behavior History for Personalized Email Search

no code implementations15 Feb 2021 Keping Bi, Pavel Metrikov, Chunyuan Li, Byungki Byun

Given these observations, we propose to leverage user search history as query context to characterize users and build a context-aware ranking model for email search.


SDA: Improving Text Generation with Self Data Augmentation

no code implementations2 Jan 2021 Ping Yu, Ruiyi Zhang, Yang Zhao, Yizhe Zhang, Chunyuan Li, Changyou Chen

Data augmentation has been widely used to improve deep neural networks in many research fields, such as computer vision.

Data Augmentation Imitation Learning +1

RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems

no code implementations ACL 2021 Baolin Peng, Chunyuan Li, Zhu Zhang, Chenguang Zhu, Jinchao Li, Jianfeng Gao

For task-oriented dialog systems to be maximally useful, it must be able to process conversations in a way that is (1) generalizable with a small number of training examples for new task domains, and (2) robust to user input in various styles, modalities or domains.

Few-Shot Named Entity Recognition: A Comprehensive Study

2 code implementations29 Dec 2020 Jiaxin Huang, Chunyuan Li, Krishan Subudhi, Damien Jose, Shobana Balakrishnan, Weizhu Chen, Baolin Peng, Jianfeng Gao, Jiawei Han

This paper presents a comprehensive study to efficiently build named entity recognition (NER) systems when a small number of in-domain labeled data is available.

Few-Shot Learning Named Entity Recognition +1

Hierarchical Graph Capsule Network

1 code implementation16 Dec 2020 Jinyu Yang, Peilin Zhao, Yu Rong, Chaochao Yan, Chunyuan Li, Hehuan Ma, Junzhou Huang

Graph Neural Networks (GNNs) draw their strength from explicitly modeling the topological information of structured data.

Graph Classification

ReMP: Rectified Metric Propagation for Few-Shot Learning

no code implementations2 Dec 2020 Yang Zhao, Chunyuan Li, Ping Yu, Changyou Chen

Few-shot learning features the capability of generalizing from a few examples.

Few-Shot Learning

Robust Conversational AI with Grounded Text Generation

no code implementations7 Sep 2020 Jianfeng Gao, Baolin Peng, Chunyuan Li, Jinchao Li, Shahin Shayandeh, Lars Liden, Heung-Yeung Shum

This article provides an overview of this progress and discusses related methods and technologies that can be incorporated for building robust conversational AI systems.

Text Generation

Weakly supervised cross-domain alignment with optimal transport

no code implementations14 Aug 2020 Siyang Yuan, Ke Bai, Liqun Chen, Yizhe Zhang, Chenyang Tao, Chunyuan Li, Guoyin Wang, Ricardo Henao, Lawrence Carin

Cross-domain alignment between image objects and text sequences is key to many visual-language tasks, and it poses a fundamental challenge to both computer vision and natural language processing.

Structure-Aware Human-Action Generation

1 code implementation ECCV 2020 Ping Yu, Yang Zhao, Chunyuan Li, Junsong Yuan, Changyou Chen

Generating long-range skeleton-based human actions has been a challenging problem since small deviations of one frame can cause a malformed action sequence.

Action Generation graph construction +1

POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training

1 code implementation EMNLP 2020 Yizhe Zhang, Guoyin Wang, Chunyuan Li, Zhe Gan, Chris Brockett, Bill Dolan

Large-scale pre-trained language models, such as BERT and GPT-2, have achieved excellent performance in language representation learning and free-form text generation.

Language Modelling Representation Learning +1

Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space

1 code implementation EMNLP 2020 Chunyuan Li, Xiang Gao, Yuan Li, Baolin Peng, Xiujun Li, Yizhe Zhang, Jianfeng Gao

We hope that our first pre-trained big VAE language model itself and results can help the NLP community renew the interests of deep generative models in the era of large-scale pre-training, and make these principled methods more practical.

Language Modelling Representation Learning +1

Multi-View Learning for Vision-and-Language Navigation

no code implementations2 Mar 2020 Qiaolin Xia, Xiujun Li, Chunyuan Li, Yonatan Bisk, Zhifang Sui, Jianfeng Gao, Yejin Choi, Noah A. Smith

Learning to navigate in a visual environment following natural language instructions is a challenging task because natural language instructions are highly variable, ambiguous, and under-specified.

MULTI-VIEW LEARNING Vision and Language Navigation

Survival Cluster Analysis

1 code implementation29 Feb 2020 Paidamoyo Chapfuwa, Chunyuan Li, Nikhil Mehta, Lawrence Carin, Ricardo Henao

As a result, there is an unmet need in survival analysis for identifying subpopulations with distinct risk profiles, while jointly accounting for accurate individualized time-to-event predictions.

Survival Analysis

Few-shot Natural Language Generation for Task-Oriented Dialog

2 code implementations Findings of the Association for Computational Linguistics 2020 Baolin Peng, Chenguang Zhu, Chunyuan Li, Xiujun Li, Jinchao Li, Michael Zeng, Jianfeng Gao

It is pre-trained on a large set of annotated NLG corpus to acquire the controllable generation ability, and fine-tuned with only a few domain-specific labels to adapt to new domains.

Data-to-Text Generation Few-Shot Learning

Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training

1 code implementation CVPR 2020 Weituo Hao, Chunyuan Li, Xiujun Li, Lawrence Carin, Jianfeng Gao

By training on a large amount of image-text-action triplets in a self-supervised learning manner, the pre-trained model provides generic representations of visual environments and language instructions.

Self-Supervised Learning Vision and Language Navigation +1

Twin Auxilary Classifiers GAN

1 code implementation NeurIPS 2019 Mingming Gong, Yanwu Xu, Chunyuan Li, Kun Zhang, Kayhan Batmanghelich

One of the popular conditional models is Auxiliary Classifier GAN (AC-GAN) that generates highly discriminative images by extending the loss function of GAN with an auxiliary classifier.

Conditional Image Generation

Straight-Through Estimator as Projected Wasserstein Gradient Flow

no code implementations5 Oct 2019 Pengyu Cheng, Chang Liu, Chunyuan Li, Dinghan Shen, Ricardo Henao, Lawrence Carin

The Straight-Through (ST) estimator is a widely used technique for back-propagating gradients through discrete random variables.

Robust Navigation with Language Pretraining and Stochastic Sampling

1 code implementation IJCNLP 2019 Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah Smith, Yejin Choi

Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unseen instructions and environments.

Vision and Language Navigation

Implicit Deep Latent Variable Models for Text Generation

1 code implementation IJCNLP 2019 Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, Changyou Chen

Deep latent variable models (LVM) such as variational auto-encoder (VAE) have recently played an important role in text generation.

Language Modelling Style Transfer +1

Twin Auxiliary Classifiers GAN

4 code implementations5 Jul 2019 Mingming Gong, Yanwu Xu, Chunyuan Li, Kun Zhang, Kayhan Batmanghelich

One of the popular conditional models is Auxiliary Classifier GAN (AC-GAN), which generates highly discriminative images by extending the loss function of GAN with an auxiliary classifier.

Conditional Image Generation

Towards Amortized Ranking-Critical Training for Collaborative Filtering

1 code implementation10 Jun 2019 Sam Lobel, Chunyuan Li, Jianfeng Gao, Lawrence Carin

In this paper we investigate new methods for training collaborative filtering models based on actor-critic reinforcement learning, to directly optimize the non-differentiable quality metrics of interest.

Collaborative Filtering Learning-To-Rank +1

Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishing

2 code implementations NAACL 2019 Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, Lawrence Carin

Variational autoencoders (VAEs) with an auto-regressive decoder have been applied for many natural language processing (NLP) tasks.

Language Modelling

Adversarial Learning of a Sampler Based on an Unnormalized Distribution

1 code implementation3 Jan 2019 Chunyuan Li, Ke Bai, Jianqiao Li, Guoyin Wang, Changyou Chen, Lawrence Carin

We investigate adversarial learning in the case when only an unnormalized form of the density can be accessed, rather than samples.


Generative Adversarial Network Training is a Continual Learning Problem

no code implementations ICLR 2019 Kevin J Liang, Chunyuan Li, Guoyin Wang, Lawrence Carin

We hypothesize that this is at least in part due to the evolution of the generator distribution and the catastrophic forgetting tendency of neural networks, which leads to the discriminator losing the ability to remember synthesized samples from previous instantiations of the generator.

Continual Learning Text Generation

Policy Optimization as Wasserstein Gradient Flows

no code implementations ICML 2018 Ruiyi Zhang, Changyou Chen, Chunyuan Li, Lawrence Carin

Policy optimization is a core component of reinforcement learning (RL), and most existing RL methods directly optimize parameters of a policy based on maximizing the expected total reward, or its surrogate.

Joint Embedding of Words and Labels for Text Classification

2 code implementations ACL 2018 Guoyin Wang, Chunyuan Li, Wenlin Wang, Yizhe Zhang, Dinghan Shen, Xinyuan Zhang, Ricardo Henao, Lawrence Carin

Word embeddings are effective intermediate representations for capturing semantic regularities between words, when learning the representations of text sequences.

General Classification Sentiment Analysis +1

Measuring the Intrinsic Dimension of Objective Landscapes

3 code implementations ICLR 2018 Chunyuan Li, Heerad Farkhoor, Rosanne Liu, Jason Yosinski

A byproduct of this construction is a simple approach for compressing networks, in some cases by more than 100 times.

Adversarial Time-to-Event Modeling

4 code implementations ICML 2018 Paidamoyo Chapfuwa, Chenyang Tao, Chunyuan Li, Courtney Page, Benjamin Goldstein, Lawrence Carin, Ricardo Henao

Modern health data science applications leverage abundant molecular and electronic health data, providing opportunities for machine learning to build statistical models to support clinical practice.

Survival Analysis

Learning Structural Weight Uncertainty for Sequential Decision-Making

1 code implementation30 Dec 2017 Ruiyi Zhang, Chunyuan Li, Changyou Chen, Lawrence Carin

Learning probability distributions on the weights of neural networks (NNs) has recently proven beneficial in many applications.

Decision Making Multi-Armed Bandits

Adversarial Symmetric Variational Autoencoder

no code implementations NeurIPS 2017 Yunchen Pu, Wei-Yao Wang, Ricardo Henao, Liqun Chen, Zhe Gan, Chunyuan Li, Lawrence Carin

A new form of variational autoencoder (VAE) is developed, in which the joint distribution of data and codes is considered in two (symmetric) forms: ($i$) from observed data fed through the encoder to yield codes, and ($ii$) from latent codes drawn from a simple prior and propagated through the decoder to manifest data.

Triangle Generative Adversarial Networks

1 code implementation NeurIPS 2017 Zhe Gan, Liqun Chen, Wei-Yao Wang, Yunchen Pu, Yizhe Zhang, Hao liu, Chunyuan Li, Lawrence Carin

The generators are designed to learn the two-way conditional distributions between the two domains, while the discriminators implicitly define a ternary discriminative function, which is trained to distinguish real data pairs and two kinds of fake data pairs.

Image-to-Image Translation Semi-Supervised Image Classification +1

Symmetric Variational Autoencoder and Connections to Adversarial Learning

2 code implementations6 Sep 2017 Liqun Chen, Shuyang Dai, Yunchen Pu, Chunyuan Li, Qinliang Su, Lawrence Carin

A new form of the variational autoencoder (VAE) is proposed, based on the symmetric Kullback-Leibler divergence.

ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching

5 code implementations NeurIPS 2017 Chunyuan Li, Hao liu, Changyou Chen, Yunchen Pu, Liqun Chen, Ricardo Henao, Lawrence Carin

We investigate the non-identifiability issues associated with bidirectional adversarial training for joint distribution matching.

Continuous-Time Flows for Efficient Inference and Density Estimation

no code implementations ICML 2018 Changyou Chen, Chunyuan Li, Liqun Chen, Wenlin Wang, Yunchen Pu, Lawrence Carin

Distinct from normalizing flows and GANs, CTFs can be adopted to achieve the above two goals in one framework, with theoretical guarantees.

Density Estimation

VAE Learning via Stein Variational Gradient Descent

no code implementations NeurIPS 2017 Yunchen Pu, Zhe Gan, Ricardo Henao, Chunyuan Li, Shaobo Han, Lawrence Carin

A new method for learning variational autoencoders (VAEs) is developed, based on Stein variational gradient descent.

Seeds Cleansing CNMF for Spatiotemporal Neural Signals Extraction of Miniscope Imaging Data

1 code implementation3 Apr 2017 Jinghao Lu, Chunyuan Li, Fan Wang

Miniscope calcium imaging is increasingly being used to monitor large populations of neuronal activities in freely behaving animals.

Neurons and Cognition Quantitative Methods

Learning Generic Sentence Representations Using Convolutional Neural Networks

no code implementations EMNLP 2017 Zhe Gan, Yunchen Pu, Ricardo Henao, Chunyuan Li, Xiaodong He, Lawrence Carin

We propose a new encoder-decoder approach to learn distributed sentence representations that are applicable to multiple purposes.

Unsupervised Learning with Truncated Gaussian Graphical Models

no code implementations15 Nov 2016 Qinliang Su, Xuejun Liao, Chunyuan Li, Zhe Gan, Lawrence Carin

Gaussian graphical models (GGMs) are widely used for statistical modeling, because of ease of inference and the ubiquitous use of the normal distribution in practical approximations.

Unsupervised Pre-training

Stochastic Gradient MCMC with Stale Gradients

no code implementations NeurIPS 2016 Changyou Chen, Nan Ding, Chunyuan Li, Yizhe Zhang, Lawrence Carin

In this paper we develop theory to show that while the bias and MSE of an SG-MCMC algorithm depend on the staleness of stochastic gradients, its estimation variance (relative to the expected estimate, based on a prescribed number of samples) is independent of it.

Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization

1 code implementation25 Dec 2015 Changyou Chen, David Carlson, Zhe Gan, Chunyuan Li, Lawrence Carin

Stochastic gradient Markov chain Monte Carlo (SG-MCMC) methods are Bayesian analogs to popular stochastic optimization methods; however, this connection is not well studied.

Stochastic Optimization

Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks

no code implementations23 Dec 2015 Chunyuan Li, Changyou Chen, David Carlson, Lawrence Carin

Pytorch implementations of Bayes By Backprop, MC Dropout, SGLD, the Local Reparametrization Trick, KF-Laplace and more

A Deep Generative Deconvolutional Image Model

no code implementations23 Dec 2015 Yunchen Pu, Xin Yuan, Andrew Stevens, Chunyuan Li, Lawrence Carin

A deep generative model is developed for representation and analysis of images, based on a hierarchical convolutional dictionary-learning framework.

Dictionary Learning Image Generation

High-Order Stochastic Gradient Thermostats for Bayesian Learning of Deep Models

no code implementations23 Dec 2015 Chunyuan Li, Changyou Chen, Kai Fan, Lawrence Carin

Stochastic gradient MCMC algorithms (SG-MCMC) are a family of diffusion-based sampling methods for large-scale Bayesian learning.

Cannot find the paper you are looking for? You can Submit a new open access paper.