Search Results for author: Jianwen Xie

Found 49 papers, 10 papers with code

Dual-Space Optimization: Improved Molecule Sequence Design by Latent Prompt Transformer

no code implementations27 Feb 2024 Deqian Kong, Yuhao Huang, Jianwen Xie, Edouardo Honig, Ming Xu, Shuanghong Xue, Pei Lin, Sanping Zhou, Sheng Zhong, Nanning Zheng, Ying Nian Wu

Designing molecules with desirable properties, such as drug-likeliness and high binding affinities towards protein targets, is a challenging problem.

Latent Plan Transformer: Planning as Latent Variable Inference

no code implementations7 Feb 2024 Deqian Kong, Dehong Xu, Minglu Zhao, Bo Pang, Jianwen Xie, Andrew Lizarraga, Yuhao Huang, Sirui Xie, Ying Nian Wu

We introduce the Latent Plan Transformer (LPT), a novel model that leverages a latent space to connect a Transformer-based trajectory generator and the final return.

STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models

no code implementations19 Oct 2023 Belhal Karimi, Jianwen Xie, Ping Li

We propose in this paper, STANLEY, a STochastic gradient ANisotropic LangEvin dYnamics, for sampling high dimensional data.

Image Generation

Molecule Design by Latent Prompt Transformer

no code implementations5 Oct 2023 Deqian Kong, Yuhao Huang, Jianwen Xie, Ying Nian Wu

This paper proposes a latent prompt Transformer model for solving challenging optimization problems such as molecule design, where the goal is to find molecules with optimal values of a target chemical or biological property that can be computed by an existing software.

Property Prediction

Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood

no code implementations10 Sep 2023 Yaxuan Zhu, Jianwen Xie, YingNian Wu, Ruiqi Gao

Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming, and there exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models.

Image Inpainting Out-of-Distribution Detection

Progressive Energy-Based Cooperative Learning for Multi-Domain Image-to-Image Translation

no code implementations26 Jun 2023 Weinan Song, Yaxuan Zhu, Lei He, YingNian Wu, Jianwen Xie

The components of translator, style encoder, and style generator constitute a diversified image generator.

Image-to-Image Translation

Likelihood-Based Generative Radiance Field with Latent Space Energy-Based Model for 3D-Aware Disentangled Image Representation

no code implementations16 Apr 2023 Yaxuan Zhu, Jianwen Xie, Ping Li

We propose the NeRF-LEBM, a likelihood-based top-down 3D-aware 2D image generative model that incorporates 3D representation via Neural Radiance Fields (NeRF) and 2D imaging process via differentiable volume rendering.

Object Variational Inference

CoopInit: Initializing Generative Adversarial Networks via Cooperative Learning

no code implementations21 Mar 2023 Yang Zhao, Jianwen Xie, Ping Li

The proposed algorithm consists of two learning stages: (i) Cooperative initialization stage: The discriminator of GAN is treated as an energy-based model (EBM) and is optimized via maximum likelihood estimation (MLE), with the help of the GAN's generator to provide synthetic data to approximate the learning gradients.

Image-to-Image Translation

A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inference

no code implementations23 Jan 2023 Jianwen Xie, Yaxuan Zhu, Yifei Xu, Dingcheng Li, Ping Li

We study a normalizing flow in the latent space of a top-down generator model, in which the normalizing flow model plays the role of the informative prior model of the generator.

Anomaly Detection Image Inpainting +1

CoopHash: Cooperative Learning of Multipurpose Descriptor and Contrastive Pair Generator via Variational MCMC Teaching for Supervised Image Hashing

no code implementations9 Oct 2022 Khoa D. Doan, Jianwen Xie, Yaxuan Zhu, Yang Zhao, Ping Li

Leveraging supervised information can lead to superior retrieval performance in the image hashing domain but the performance degrades significantly without enough labeled data.


A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model

no code implementations ICLR 2022 Jianwen Xie, Yaxuan Zhu, Jun Li, Ping Li

Under the short-run non-mixing MCMC scenario, the estimation of the energy-based model is shown to follow the perturbation of maximum likelihood, and the short-run Langevin flow and the normalizing flow form a two-flow generator that we call CoopFlow.

An Energy-Based Prior for Generative Saliency

1 code implementation19 Apr 2022 Jing Zhang, Jianwen Xie, Nick Barnes, Ping Li

We propose a novel generative saliency prediction framework that adopts an informative energy-based model as a prior distribution.

object-detection RGB-D Salient Object Detection +3

Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction

no code implementations NeurIPS 2021 Jing Zhang, Jianwen Xie, Nick Barnes, Ping Li

In this paper, we take a step further by proposing a novel generative vision transformer with latent variables following an informative energy-based prior for salient object detection.

object-detection RGB-D Salient Object Detection +3

Unsupervised Contrastive Learning for Signal-Dependent Noise Synthesis

no code implementations29 Sep 2021 Nanqing Dong, Jianwen Xie, Ping Li

We present a simple yet robust noise synthesis framework based on unsupervised contrastive learning.

Contrastive Learning

Blind Image Super-resolution with Elaborate Degradation Modeling on Noise and Kernel

1 code implementation CVPR 2022 Zongsheng Yue, Qian Zhao, Jianwen Xie, Lei Zhang, Deyu Meng, Kwan-Yee K. Wong

To address the above issues, this paper proposes a model-based blind SISR method under the probabilistic framework, which elaborately models image degradation from the perspectives of noise and blur kernel.

Image Super-Resolution

Energy-Based Generative Cooperative Saliency Prediction

1 code implementation25 Jun 2021 Jing Zhang, Jianwen Xie, Zilong Zheng, Nick Barnes

In this paper, to model the uncertainty of visual saliency, we study the saliency prediction problem from the perspective of generative models by learning a conditional probability distribution over the saliency map given an input image, and treating the saliency prediction as a sampling process from the learned distribution.

Saliency Prediction

Patchwise Generative ConvNet: Training Energy-Based Models From a Single Natural Image for Internal Learning

no code implementations CVPR 2021 Zilong Zheng, Jianwen Xie, Ping Li

Exploiting internal statistics of a single natural image has long been recognized as a significant research paradigm where the goal is to learn the distribution of patches within the image without relying on external training data.

Descriptive Image Generation +1

Learning Deep Latent Variable Models by Short-Run MCMC Inference With Optimal Transport Correction

no code implementations CVPR 2021 Dongsheng An, Jianwen Xie, Ping Li

Learning latent variable models with deep top-down architectures typically requires inferring the latent variables for each training example based on the posterior distribution of these latent variables.

Semi-Supervised Video Deraining with Dynamical Rain Generator

1 code implementation CVPR 2021 Zongsheng Yue, Jianwen Xie, Qian Zhao, Deyu Meng

Firstly, most of them do not sufficiently model the characteristics of rain layers of rainy videos.

Rain Removal

Learning Cycle-Consistent Cooperative Networks via Alternating MCMC Teaching for Unsupervised Cross-Domain Translation

no code implementations7 Mar 2021 Jianwen Xie, Zilong Zheng, Xiaolin Fang, Song-Chun Zhu, Ying Nian Wu

This paper studies the unsupervised cross-domain translation problem by proposing a generative framework, in which the probability distribution of each domain is represented by a generative cooperative network that consists of an energy-based model and a latent variable model.

Translation Unsupervised Image-To-Image Translation

Learning Energy-Based Generative Models via Coarse-to-Fine Expanding and Sampling

no code implementations ICLR 2021 Yang Zhao, Jianwen Xie, Ping Li

Energy-based models (EBMs) for generative modeling parametrize a single net and can be directly trained by maximum likelihood estimation.

Translation Unsupervised Image-To-Image Translation

Learning Energy-Based Model with Variational Auto-Encoder as Amortized Sampler

no code implementations29 Dec 2020 Jianwen Xie, Zilong Zheng, Ping Li

In this paper, we propose to learn a variational auto-encoder (VAE) to initialize the finite-step MCMC, such as Langevin dynamics that is derived from the energy function, for efficient amortized sampling of the EBM.

A Representational Model of Grid Cells' Path Integration Based on Matrix Lie Algebras

no code implementations28 Sep 2020 Ruiqi Gao, Jianwen Xie, Xue-Xin Wei, Song-Chun Zhu, Ying Nian Wu

The grid cells in the mammalian medial entorhinal cortex exhibit striking hexagon firing patterns when the agent navigates in the open field.


Learning Noise-Aware Encoder-Decoder from Noisy Labels by Alternating Back-Propagation for Saliency Detection

no code implementations ECCV 2020 Jing Zhang, Jianwen Xie, Nick Barnes

The proposed model consists of two sub-models parameterized by neural networks: (1) a saliency predictor that maps input images to clean saliency maps, and (2) a noise generator, which is a latent variable model that produces noises from Gaussian latent vectors.

Saliency Detection

On Path Integration of Grid Cells: Group Representation and Isotropic Scaling

1 code implementation NeurIPS 2021 Ruiqi Gao, Jianwen Xie, Xue-Xin Wei, Song-Chun Zhu, Ying Nian Wu

In this paper, we conduct theoretical analysis of a general representation model of path integration by grid cells, where the 2D self-position is encoded as a higher dimensional vector, and the 2D self-motion is represented by a general transformation of the vector.

Dimensionality Reduction Position

Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification

1 code implementation CVPR 2021 Jianwen Xie, Yifei Xu, Zilong Zheng, Song-Chun Zhu, Ying Nian Wu

We propose a generative model of unordered point sets, such as point clouds, in the form of an energy-based model, where the energy function is parameterized by an input-permutation-invariant bottom-up neural network.

3D Generation General Classification +3

Learning Energy-based Spatial-Temporal Generative ConvNets for Dynamic Patterns

no code implementations26 Sep 2019 Jianwen Xie, Song-Chun Zhu, Ying Nian Wu

We show that an energy-based spatial-temporal generative ConvNet can be used to model and synthesize dynamic patterns.

Learning Feature-to-Feature Translator by Alternating Back-Propagation for Generative Zero-Shot Learning

1 code implementation ICCV 2019 Yizhe Zhu, Jianwen Xie, Bingchen Liu, Ahmed Elgammal

We investigate learning feature-to-feature translator networks by alternating back-propagation as a general-purpose solution to zero-shot learning (ZSL) problems.

Zero-Shot Learning

Energy-Based Continuous Inverse Optimal Control

no code implementations10 Apr 2019 Yifei Xu, Jianwen Xie, Tianyang Zhao, Chris Baker, Yibiao Zhao, Ying Nian Wu

The problem of continuous inverse optimal control (over finite time horizon) is to learn the unknown cost function over the sequence of continuous control variables from expert demonstrations.

Autonomous Driving Continuous Control +1

Semantic-Guided Multi-Attention Localization for Zero-Shot Learning

no code implementations NeurIPS 2019 Yizhe Zhu, Jianwen Xie, Zhiqiang Tang, Xi Peng, Ahmed Elgammal

Zero-shot learning extends the conventional object classification to the unseen class recognition by introducing semantic representations of classes.

Zero-Shot Learning

Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning

no code implementations7 Feb 2019 Jianwen Xie, Zilong Zheng, Xiaolin Fang, Song-Chun Zhu, Ying Nian Wu

This paper studies the problem of learning the conditional distribution of a high-dimensional output given an input, where the output and input may belong to two different domains, e. g., the output is a photo image and the input is a sketch image.

Image-to-Image Translation

Learning V1 Simple Cells with Vector Representation of Local Content and Matrix Representation of Local Motion

no code implementations24 Jan 2019 Ruiqi Gao, Jianwen Xie, Siyuan Huang, Yufan Ren, Song-Chun Zhu, Ying Nian Wu

This paper proposes a representational model for image pairs such as consecutive video frames that are related by local pixel displacements, in the hope that the model may shed light on motion perception in primary visual cortex (V1).

Optical Flow Estimation

Learning Dynamic Generator Model by Alternating Back-Propagation Through Time

no code implementations27 Dec 2018 Jianwen Xie, Ruiqi Gao, Zilong Zheng, Song-Chun Zhu, Ying Nian Wu

The non-linear transformation of this transition model can be parametrized by a feedforward neural network.

Learning Grid Cells as Vector Representation of Self-Position Coupled with Matrix Representation of Self-Motion

1 code implementation ICLR 2019 Ruiqi Gao, Jianwen Xie, Song-Chun Zhu, Ying Nian Wu

In this model, the 2D self-position of the agent is represented by a high-dimensional vector, and the 2D self-motion or displacement of the agent is represented by a matrix that transforms the vector.


Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Knowledge Transfer

1 code implementation CVPR 2018 Hao-Shu Fang, Guansong Lu, Xiaolin Fang, Jianwen Xie, Yu-Wing Tai, Cewu Lu

In this paper, we present a novel method to generate synthetic human part segmentation data using easily-obtained human keypoint annotations.

Ranked #4 on Human Part Segmentation on PASCAL-Part (using extra training data)

Human Parsing Human Part Segmentation +3

Learning Descriptor Networks for 3D Shape Synthesis and Analysis

1 code implementation CVPR 2018 Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, Ying Nian Wu

This paper proposes a 3D shape descriptor network, which is a deep convolutional energy-based model, for modeling volumetric shape patterns.


A Causal And-Or Graph Model for Visibility Fluent Reasoning in Tracking Interacting Objects

no code implementations CVPR 2018 Yuanlu Xu, Lei Qin, Xiaobai Liu, Jianwen Xie, Song-Chun Zhu

We introduce a Causal And-Or Graph (C-AOG) to represent the causal-effect relations between an object's visibility fluent and its activities, and develop a probabilistic graph model to jointly reason the visibility fluent change (e. g., from visible to invisible) and track humans in videos.

Visual Tracking

Super-Trajectory for Video Segmentation

no code implementations ICCV 2017 Wenguan Wang, Jianbing Shen, Jianwen Xie, Fatih Porikli

We introduce a novel semi-supervised video segmentation approach based on an efficient video representation, called as "super-trajectory".

Clustering Segmentation +2

Cooperative Training of Descriptor and Generator Networks

no code implementations29 Sep 2016 Jianwen Xie, Yang Lu, Ruiqi Gao, Song-Chun Zhu, Ying Nian Wu

Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finite-step MCMC that samples and trains the energy-based descriptor model.

Decoding the Encoding of Functional Brain Networks: an fMRI Classification Comparison of Non-negative Matrix Factorization (NMF), Independent Component Analysis (ICA), and Sparse Coding Algorithms

no code implementations1 Jul 2016 Jianwen Xie, Pamela K. Douglas, Ying Nian Wu, Arthur L. Brody, Ariana E. Anderson

Spatial sparse coding algorithms ($L1$ Regularized Learning and K-SVD) would impose local specialization and a discouragement of multitasking, where the total observed activity in a single voxel originates from a restricted number of possible brain networks.

Time Series Analysis

Synthesizing Dynamic Patterns by Spatial-Temporal Generative ConvNet

no code implementations CVPR 2017 Jianwen Xie, Song-Chun Zhu, Ying Nian Wu

We show that a spatial-temporal generative ConvNet can be used to model and synthesize dynamic patterns.

A Theory of Generative ConvNet

no code implementations10 Feb 2016 Jianwen Xie, Yang Lu, Song-Chun Zhu, Ying Nian Wu

If we further assume that the non-linearity in the ConvNet is Rectified Linear Unit (ReLU) and the reference distribution is Gaussian white noise, then we obtain a generative ConvNet model that is unique among energy-based models: The model is piecewise Gaussian, and the means of the Gaussian pieces are defined by an auto-encoder, where the filters in the bottom-up encoding become the basis functions in the top-down decoding, and the binary activation variables detected by the filters in the bottom-up convolution process become the coefficients of the basis functions in the top-down deconvolution process.

Learning Inhomogeneous FRAME Models for Object Patterns

no code implementations CVPR 2014 Jianwen Xie, Wenze Hu, Song-Chun Zhu, Ying Nian Wu

We investigate an inhomogeneous version of the FRAME (Filters, Random field, And Maximum Entropy) model and apply it to modeling object patterns.


Cannot find the paper you are looking for? You can Submit a new open access paper.