Search Results for author: Jun Zhu

Found 198 papers, 72 papers with code

Understanding and Stabilizing GANs' Training Dynamics Using Control Theory

no code implementations ICML 2020 Kun Xu, Chongxuan Li, Jun Zhu, Bo Zhang

There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods.

Defense Against Adversarial Attacks via Controlling Gradient Leaking on Embedded Manifolds

no code implementations ECCV 2020 Yueru Li, Shuyu Cheng, Hang Su, Jun Zhu

Based on our investigation, we further present a new robust learning algorithm which encourages a larger gradient component in the tangent space of data manifold, suppressing the gradient leaking phenomenon consequently.

Variance Reduction and Quasi-Newton for Particle-Based Variational Inference

no code implementations ICML 2020 Michael Zhu, Chang Liu, Jun Zhu

Particle-based Variational Inference methods (ParVIs), like Stein Variational Gradient Descent, are nonparametric variational inference methods that optimize a set of particles to best approximate a target distribution.

Bayesian Inference Riemannian optimization +1

Tianshou: a Highly Modularized Deep Reinforcement Learning Library

1 code implementation29 Jul 2021 Jiayi Weng, Huayu Chen, Dong Yan, Kaichao You, Alexis Duburcq, Minghao Zhang, Hang Su, Jun Zhu

We present Tianshou, a highly modularized python library for deep reinforcement learning (DRL) that uses PyTorch as its backend.

Query2Label: A Simple Transformer Way to Multi-Label Classification

1 code implementation22 Jul 2021 Shilong Liu, Lei Zhang, Xiao Yang, Hang Su, Jun Zhu

The use of Transformer is rooted in the need of extracting local discriminative features adaptively for different labels, which is a strongly desired property due to the existence of multiple objects in one image.

Classification Multi-Label Classification

On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithms

1 code implementation21 Jul 2021 Shuyu Cheng, Guoqiang Wu, Jun Zhu

Finally, our theoretical results are confirmed by experiments on several numerical benchmarks as well as adversarial attacks.

Boosting Transferability of Targeted Adversarial Examples via Hierarchical Generative Networks

no code implementations5 Jul 2021 Xiao Yang, Yinpeng Dong, Tianyu Pang, Hang Su, Jun Zhu

Transfer-based adversarial attacks can effectively evaluate model robustness in the black-box setting.

Understanding Adversarial Attacks on Observations in Deep Reinforcement Learning

no code implementations30 Jun 2021 You Qiaoben, Chengyang Ying, Xinning Zhou, Hang Su, Jun Zhu, Bo Zhang

Following the analysis of the function space, we design a generic two-stage framework in the subspace where the adversary lures the agent to a target trajectory or a deceptive policy.

Regularized OFU: an Efficient UCB Estimator forNon-linear Contextual Bandit

no code implementations29 Jun 2021 Yichi Zhou, Shihong Song, Huishuai Zhang, Jun Zhu, Wei Chen, Tie-Yan Liu

However, it is in general unknown how to deriveefficient and effective EE trade-off methods for non-linearcomplex tasks, suchas contextual bandit with deep neural network as the reward function.

Multi-Armed Bandits

Improving Transferability of Adversarial Patches on Face Recognition with Generative Models

no code implementations CVPR 2021 Zihao Xiao, Xianfeng Gao, Chilin Fu, Yinpeng Dong, Wei Gao, Xiaolu Zhang, Jun Zhou, Jun Zhu

However, deep CNNs are vulnerable to adversarial patches, which are physically realizable and stealthy, raising new security concerns on the real-world applications of these models.

Face Recognition

Accumulative Poisoning Attacks on Real-time Data

1 code implementation18 Jun 2021 Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, Jun Zhu

Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy.

Federated Learning

Scalable Quasi-Bayesian Inference for Instrumental Variable Regression

no code implementations16 Jun 2021 Ziyu Wang, Yuhao Zhou, Tongzheng Ren, Jun Zhu

Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking.

Bayesian Inference

Nonlinear Hawkes Processes in Time-Varying System

no code implementations9 Jun 2021 Feng Zhou, Quyu Kong, Yixuan Zhang, Cheng Feng, Jun Zhu

Hawkes processes are a class of point processes that have the ability to model the self- and mutual-exciting phenomena.

Bayesian Inference Point Processes +1

Understanding Softmax Confidence and Uncertainty

no code implementations9 Jun 2021 Tim Pearce, Alexandra Brintrup, Jun Zhu

It is often remarked that neural networks fail to increase their uncertainty when predicting on data far from the training distribution.

Stability and Generalization of Bilevel Programming in Hyperparameter Optimization

no code implementations8 Jun 2021 Fan Bao, Guoqiang Wu, Chongxuan Li, Jun Zhu, Bo Zhang

Our results can explain some mysterious behaviours of the bilevel programming in practice, for instance, overfitting to the validation set.

Hyperparameter Optimization

Exploring Memorization in Adversarial Training

no code implementations3 Jun 2021 Yinpeng Dong, Ke Xu, Xiao Yang, Tianyu Pang, Zhijie Deng, Hang Su, Jun Zhu

In this paper, we investigate the memorization effect in adversarial training (AT) for promoting a deeper understanding of capacity, convergence, generalization, and especially robust overfitting of adversarially trained classifiers.

KO-PDE: Kernel Optimized Discovery of Partial Differential Equations with Varying Coefficients

no code implementations2 Jun 2021 Yingtao Luo, Qiang Liu, Yuntian Chen, WenBo Hu, Jun Zhu

Previous algorithms can discover some simple instances of PDEs-VC but fail in the discovery of PDEs with coefficients of higher complexity, as a result of coefficient estimation inaccuracy.

Density Estimation

Adversarial Training with Rectified Rejection

1 code implementation31 May 2021 Tianyu Pang, Huishuai Zhang, Di He, Yinpeng Dong, Hang Su, Wei Chen, Jun Zhu, Tie-Yan Liu

Adversarial training (AT) is one of the most effective strategies for promoting model robustness, whereas even the state-of-the-art adversarially trained models struggle to exceed 60% robust test accuracy on CIFAR-10 without additional data, which is far from practical.

Unsupervised Part Segmentation through Disentangling Appearance and Shape

no code implementations CVPR 2021 Shilong Liu, Lei Zhang, Xiao Yang, Hang Su, Jun Zhu

We study the problem of unsupervised discovery and segmentation of object parts, which, as an intermediate local representation, are capable of finding intrinsic object structure and providing more explainable recognition results.

Semantic Segmentation

Rethinking and Reweighting the Univariate Losses for Multi-Label Ranking: Consistency and Generalization

no code implementations10 May 2021 Guoqiang Wu, Chongxuan Li, Kun Xu, Jun Zhu

Our results show that learning algorithms with the consistent univariate loss have an error bound of $O(c)$ ($c$ is the number of labels), while algorithms with the inconsistent pairwise loss depend on $O(\sqrt{c})$ as shown in prior work.

Multi-Label Classification

Automated Decision-based Adversarial Attacks

no code implementations9 May 2021 Qi-An Fu, Yinpeng Dong, Hang Su, Jun Zhu

Deep learning models are vulnerable to adversarial examples, which can fool a target classifier by imposing imperceptible perturbations onto natural examples.

Adversarial Attack Program Synthesis

MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering

1 code implementation ICLR 2021 Tsung Wei Tsai, Chongxuan Li, Jun Zhu

We present Mixture of Contrastive Experts (MiCE), a unified probabilistic clustering framework that simultaneously exploits the discriminative representations learned by contrastive learning and the semantic structures captured by a latent mixture model.

Contrastive Learning Image Clustering

Few-shot Continual Learning: a Brain-inspired Approach

no code implementations19 Apr 2021 Liyuan Wang, Qian Li, Yi Zhong, Jun Zhu

Our solution is based on the observation that continual learning of a task sequence inevitably interferes few-shot generalization, which makes it highly nontrivial to extend few-shot learning strategies to continual learning scenarios.

Continual Learning Few-Shot Learning

Counter-Strike Deathmatch with Large-Scale Behavioural Cloning

1 code implementation9 Apr 2021 Tim Pearce, Jun Zhu

This paper describes an AI agent that plays the popular first-person-shooter (FPS) video game `Counter-Strike; Global Offensive' (CSGO) from pixel input.

Behavioural cloning FPS Games

Accurate and Reliable Forecasting using Stochastic Differential Equations

no code implementations28 Mar 2021 Peng Cui, Zhijie Deng, WenBo Hu, Jun Zhu

It is critical yet challenging for deep learning models to properly characterize uncertainty that is pervasive in real-world environments.

Prediction Intervals

LiBRe: A Practical Bayesian Approach to Adversarial Detection

1 code implementation CVPR 2021 Zhijie Deng, Xiao Yang, Shizhen Xu, Hang Su, Jun Zhu

Despite their appealing flexibility, deep neural networks (DNNs) are vulnerable against adversarial examples.

Adversarial Defense

Black-box Detection of Backdoor Attacks with Limited Information and Data

no code implementations24 Mar 2021 Yinpeng Dong, Xiao Yang, Zhijie Deng, Tianyu Pang, Zihao Xiao, Hang Su, Jun Zhu

Although deep neural networks (DNNs) have made rapid progress in recent years, they are vulnerable in adversarial environments.

Implicit Normalizing Flows

1 code implementation ICLR 2021 Cheng Lu, Jianfei Chen, Chongxuan Li, Qiuhao Wang, Jun Zhu

Through theoretical analysis, we show that the function space of ImpFlow is strictly richer than that of ResFlows.

DNN2LR: Automatic Feature Crossing for Credit Scoring

no code implementations24 Feb 2021 Qiang Liu, Zhaocheng Liu, Haoli Zhang, Yuntian Chen, Jun Zhu

Accordingly, we can design an automatic feature crossing method to find feature interactions in DNN, and use them as cross features in LR.

Feature Engineering

Rethinking Natural Adversarial Examples for Classification Models

1 code implementation23 Feb 2021 Xiao Li, Jianmin Li, Ting Dai, Jie Shi, Jun Zhu, Xiaolin Hu

A detection model based on the classification model EfficientNet-B7 achieved a top-1 accuracy of 53. 95%, surpassing previous state-of-the-art classification models trained on ImageNet, suggesting that accurate localization information can significantly boost the performance of classification models on ImageNet-A.

Classification General Classification +1

High-fidelity Prediction of Megapixel Longitudinal Phase-space Images of Electron Beams using Encoder-Decoder Neural Networks

no code implementations25 Jan 2021 Jun Zhu, Ye Chen, Frank Brinker, Winfried Decking, Sergey Tomin, Holger Schlarb

We also show the scalability and interpretability of the model by sharing the same decoder with more than one encoder used for different setups of the photoinjector, and propose a pragmatic way to model a facility with various diagnostics and working points.

Cognitive Visual Inspection Service for LCD Manufacturing Industry

no code implementations11 Jan 2021 Yuanyuan Ding, Junchi Yan, Guoqiang Hu, Jun Zhu

This paper discloses a novel visual inspection system for liquid crystal display (LCD), which is currently a dominant type in the FPD industry.

Defect Detection

Relaxed Conditional Image Transfer for Semi-supervised Domain Adaptation

no code implementations5 Jan 2021 Qijun Luo, Zhili Liu, Lanqing Hong, Chongxuan Li, Kuo Yang, Liyuan Wang, Fengwei Zhou, Guilin Li, Zhenguo Li, Jun Zhu

Semi-supervised domain adaptation (SSDA), which aims to learn models in a partially labeled target domain with the assistance of the fully labeled source domain, attracts increasing attention in recent years.

Domain Adaptation

Ranking Cost: One-Stage Circuit Routing by Directly Optimizing Global Objective Function

no code implementations1 Jan 2021 Shiyu Huang, Bin Wang, Dong Li, Jianye Hao, Jun Zhu, Ting Chen

In our method, we introduce a new set of variables called cost maps, which can help the A* router to find out proper paths to achieve the global object.

Adaptive N-step Bootstrapping with Off-policy Data

no code implementations1 Jan 2021 Guan Wang, Dong Yan, Hang Su, Jun Zhu

In this work, we point out that the optimal value of n actually differs on each data point, while the fixed value n is a rough average of them.

Atari Games

Series Saliency: Temporal Interpretation for Multivariate Time Series Forecasting

no code implementations16 Dec 2020 Qingyi Pan, WenBo Hu, Jun Zhu

Though deep learning methods have recently been developed to give superior forecasting results, it is crucial to improve the interpretability of time series models.

Data Augmentation Multivariate Time Series Forecasting +1

Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings

1 code implementation14 Dec 2020 Qipeng Guo, Zhijing Jin, Ziyu Wang, Xipeng Qiu, Weinan Zhang, Jun Zhu, Zheng Zhang, David Wipf

Cycle-consistent training is widely used for jointly learning a forward and inverse mapping between two domains of interest without the cumbersome requirement of collecting matched pairs within each domain.

Knowledge Graphs Text Generation

Understanding and Exploring the Network with Stochastic Architectures

no code implementations NeurIPS 2020 Zhijie Deng, Yinpeng Dong, Shifeng Zhang, Jun Zhu

In this work, we decouple the training of a network with stochastic architectures (NSA) from NAS and provide a first systematical investigation on it as a stand-alone problem.

Neural Architecture Search

Multi-label classification: do Hamming loss and subset accuracy really conflict with each other?

1 code implementation NeurIPS 2020 Guoqiang Wu, Jun Zhu

On the other hand, when directly optimizing SA with its surrogate loss, it has learning guarantees that depend on $O(\sqrt{c})$ for both HL and SA measures.

General Classification Multi-Label Classification

Further Analysis of Outlier Detection with Deep Generative Models

1 code implementation NeurIPS 2020 Ziyu Wang, Bin Dai, David Wipf, Jun Zhu

The recent, counter-intuitive discovery that deep generative models (DGMs) can frequently assign a higher likelihood to outliers has implications for both outlier detection applications as well as our overall understanding of generative modeling.

Outlier Detection

Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models

1 code implementation16 Oct 2020 Fan Bao, Kun Xu, Chongxuan Li, Lanqing Hong, Jun Zhu, Bo Zhang

The learning and evaluation of energy-based latent variable models (EBLVMs) without any structural assumptions are highly challenging, because the true posteriors and the partition functions in such models are generally intractable.

Latent Variable Models

Bi-level Score Matching for Learning Energy-based Latent Variable Models

1 code implementation NeurIPS 2020 Fan Bao, Chongxuan Li, Kun Xu, Hang Su, Jun Zhu, Bo Zhang

This paper presents a bi-level score matching (BiSM) method to learn EBLVMs with general structures by reformulating SM as a bi-level optimization problem.

Latent Variable Models Stochastic Optimization

BayesAdapter: Being Bayesian, Inexpensively and Reliably, via Bayesian Fine-tuning

1 code implementation5 Oct 2020 Zhijie Deng, Hao Zhang, Xiao Yang, Yinpeng Dong, Jun Zhu

Despite their theoretical appealingness, Bayesian neural networks (BNNs) are left behind in real-world adoption, due to persistent concerns on their scalability, accessibility, and reliability.

Variational Inference

Bag of Tricks for Adversarial Training

2 code implementations ICLR 2021 Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, Jun Zhu

Adversarial training (AT) is one of the most effective strategies for promoting model robustness.

Switching Transferable Gradient Directions for Query-Efficient Black-Box Adversarial Attacks

no code implementations15 Sep 2020 Chen Ma, Shuyu Cheng, Li Chen, Jun Zhu, Junhai Yong

In each iteration, SWITCH first tries to update the current sample along the direction of $\hat{\mathbf{g}}$, but considers switching to its opposite direction $-\hat{\mathbf{g}}$ if our algorithm detects that it does not increase the value of the attack objective function.

Adversarial Attack

Training Interpretable Convolutional Neural Networks by Differentiating Class-specific Filters

1 code implementation ECCV 2020 Haoyu Liang, Zhihao Ouyang, Yuyuan Zeng, Hang Su, Zihao He, Shu-Tao Xia, Jun Zhu, Bo Zhang

Most existing works attempt post-hoc interpretation on a pre-trained model, while neglecting to reduce the entanglement underlying the model.

Object Localization

Delving into the Adversarial Robustness on Face Recognition

2 code implementations8 Jul 2020 Xiao Yang, Dingcheng Yang, Yinpeng Dong, Wenjian Yu, Hang Su, Jun Zhu

However, the lack of robustness in deep CNNs to adversarial examples has raised security concerns to enormous face recognition applications.

Face Recognition

Efficient Learning of Generative Models via Finite-Difference Score Matching

1 code implementation NeurIPS 2020 Tianyu Pang, Kun Xu, Chongxuan Li, Yang song, Stefano Ermon, Jun Zhu

Several machine learning applications involve the optimization of higher-order derivatives (e. g., gradients of gradients) during training, which can be expensive in respect to memory and computation even with automatic differentiation.

Efficient Inference of Flexible Interaction in Spiking-neuron Networks

no code implementations ICLR 2021 Feng Zhou, Yixuan Zhang, Jun Zhu

Hawkes process provides an effective statistical framework for analyzing the time-dependent interaction of neuronal spiking activities.

Calibrated Reliable Regression using Maximum Mean Discrepancy

no code implementations NeurIPS 2020 Peng Cui, Wen-Bo Hu, Jun Zhu

Accurate quantification of uncertainty is crucial for real-world applications of machine learning.

Prediction Intervals

Dynamic Window-level Granger Causality of Multi-channel Time Series

no code implementations14 Jun 2020 Zhiheng Zhang, Wen-Bo Hu, Tian Tian, Jun Zhu

In this paper, we present the dynamic window-level Granger causality method (DWGC) for multi-channel time series data.

Time Series

Brain-inspired global-local learning incorporated with neuromorphic computing

no code implementations5 Jun 2020 Yujie Wu, Rong Zhao, Jun Zhu, Feng Chen, Mingkun Xu, Guoqi Li, Sen Song, Lei Deng, Guanrui Wang, Hao Zheng, Jing Pei, Youhui Zhang, Mingguo Zhao, Luping Shi

We demonstrate the advantages of this model in multiple different tasks, including few-shot learning, continual learning, and fault-tolerance learning in neuromorphic vision sensors.

Continual Learning Few-Shot Learning

Nonparametric Score Estimators

1 code implementation ICML 2020 Yuhao Zhou, Jiaxin Shi, Jun Zhu

Estimating the score, i. e., the gradient of log density function, from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models that involve flexible yet intractable densities.

Lazy-CFR: fast and near-optimal regret minimization for extensive games with imperfect information

no code implementations ICLR 2020 Yichi Zhou, Tongzheng Ren, Jialian Li, Dong Yan, Jun Zhu

In this paper, we present Lazy-CFR, a CFR algorithm that adopts a lazy update strategy to avoid traversing the whole game tree in each round.

Towards Face Encryption by Generating Adversarial Identity Masks

1 code implementation15 Mar 2020 Xiao Yang, Yinpeng Dong, Tianyu Pang, Hang Su, Jun Zhu, Yuefeng Chen, Hui Xue

As billions of personal data being shared through social media and network, the data privacy and security have drawn an increasing attention.

Face Recognition

Triple Memory Networks: a Brain-Inspired Method for Continual Learning

no code implementations6 Mar 2020 Liyuan Wang, Bo Lei, Qian Li, Hang Su, Jun Zhu, Yi Zhong

Continual acquisition of novel experience without interfering previously learned knowledge, i. e. continual learning, is critical for artificial neural networks, but limited by catastrophic forgetting.

class-incremental learning Hippocampus +1

VFlow: More Expressive Generative Flows with Variational Data Augmentation

1 code implementation ICML 2020 Jianfei Chen, Cheng Lu, Biqi Chenli, Jun Zhu, Tian Tian

Generative flows are promising tractable models for density modeling that define probabilistic distributions with invertible transformations.

Ranked #19 on Image Generation on CIFAR-10 (bits/dimension metric)

Density Estimation Image Generation +2

Boosting Adversarial Training with Hypersphere Embedding

1 code implementation NeurIPS 2020 Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, Hang Su

Adversarial training (AT) is one of the most effective defenses against adversarial attacks for deep learning models.

Representation Learning

A Wasserstein Minimum Velocity Approach to Learning Unnormalized Models

1 code implementation18 Feb 2020 Ziyu Wang, Shuyu Cheng, Yueru Li, Jun Zhu, Bo Zhang

Score matching provides an effective approach to learning flexible unnormalized models, but its scalability is limited by the need to evaluate a second-order derivative.

Adversarial Distributional Training for Robust Deep Learning

1 code implementation NeurIPS 2020 Yinpeng Dong, Zhijie Deng, Tianyu Pang, Hang Su, Jun Zhu

Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples.

Analyzing the Noise Robustness of Deep Neural Networks

no code implementations26 Jan 2020 Kelei Cao, Mengchen Liu, Hang Su, Jing Wu, Jun Zhu, Shixia Liu

The key is to compare and analyze the datapaths of both the adversarial and normal examples.

Adversarial Attack

SVQN: Sequential Variational Soft Q-Learning Networks

no code implementations ICLR 2020 Shiyu Huang, Hang Su, Jun Zhu, Ting Chen

Partially Observable Markov Decision Processes (POMDPs) are popular and flexible models for real-world decision-making applications that demand the information from past observations to make optimal decisions.

Decision Making Q-Learning

Benchmarking Adversarial Robustness

no code implementations26 Dec 2019 Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Hang Su, Zihao Xiao, Jun Zhu

Deep neural networks are vulnerable to adversarial examples, which becomes one of the most important research problems in the development of deep learning.

Adversarial Attack Image Classification

Triple Generative Adversarial Networks

1 code implementation20 Dec 2019 Chongxuan Li, Kun Xu, Jiashuo Liu, Jun Zhu, Bo Zhang

It is formulated as a three-player minimax game consisting of a generator, a classifier and a discriminator, and therefore is referred to as Triple Generative Adversarial Network (Triple-GAN).

Classification Conditional Image Generation +3

The Search for Sparse, Robust Neural Networks

1 code implementation5 Dec 2019 Justin Cosentino, Federico Zaiter, Dan Pei, Jun Zhu

Recent work on deep neural network pruning has shown there exist sparse subnetworks that achieve equal or improved accuracy, training time, and loss using fewer network parameters when compared to their dense counterparts.

Network Pruning

Design and Interpretation of Universal Adversarial Patches in Face Detection

no code implementations ECCV 2020 Xiao Yang, Fangyun Wei, Hongyang Zhang, Jun Zhu

We consider universal adversarial patches for faces -- small visual elements whose addition to a face image reliably destroys the performance of face detectors.

Face Detection

Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structure

1 code implementation22 Nov 2019 Zhijie Deng, Yucen Luo, Jun Zhu, Bo Zhang

Bayesian neural networks (BNNs) augment deep networks with uncertainty quantification by Bayesian treatment of the network weights.

Bayesian Inference Neural Architecture Search +1

Generative Well-intentioned Networks

no code implementations NeurIPS 2019 Justin Cosentino, Jun Zhu

We propose Generative Well-intentioned Networks (GWINs), a novel framework for increasing the accuracy of certainty-based, closed-world classifiers.

Understanding and Stabilizing GANs' Training Dynamics with Control Theory

1 code implementation29 Sep 2019 Kun Xu, Chongxuan Li, Jun Zhu, Bo Zhang

There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods.

Ranked #22 on Image Generation on CIFAR-10 (Inception score metric)

Image Generation L2 Regularization

Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks

1 code implementation ICLR 2020 Tianyu Pang, Kun Xu, Jun Zhu

Our experiments on CIFAR-10 and CIFAR-100 demonstrate that MI can further improve the adversarial robustness for the models trained by mixup and its variants.

A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels

no code implementations20 Sep 2019 Yucen Luo, Jun Zhu, Tomas Pfister

Recently deep neural networks have shown their capacity to memorize training data, even with noisy labels, which hurts generalization performance.

Learning with noisy labels

DashNet: A Hybrid Artificial and Spiking Neural Network for High-speed Object Tracking

no code implementations15 Sep 2019 Zheyu Yang, Yujie Wu, Guanrui Wang, Yukuan Yang, Guoqi Li, Lei Deng, Jun Zhu, Luping Shi

To the best of our knowledge, DashNet is the first framework that can integrate and process ANNs and SNNs in a hybrid paradigm, which provides a novel solution to achieve both effectiveness and efficiency for high-speed object tracking.

Object Tracking

Improving Black-box Adversarial Attacks with a Transfer-based Prior

2 code implementations NeurIPS 2019 Shuyu Cheng, Yinpeng Dong, Tianyu Pang, Hang Su, Jun Zhu

We consider the black-box adversarial setting, where the adversary has to generate adversarial perturbations without access to the target models to compute gradients.

Scalable Training of Inference Networks for Gaussian-Process Models

2 code implementations27 May 2019 Jiaxin Shi, Mohammad Emtiyaz Khan, Jun Zhu

Inference in Gaussian process (GP) models is computationally challenging for large data, and often difficult to approximate with a small number of inducing points.

Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness

2 code implementations ICLR 2020 Tianyu Pang, Kun Xu, Yinpeng Dong, Chao Du, Ning Chen, Jun Zhu

Previous work shows that adversarially robust generalization requires larger sample complexity, and the same dataset, e. g., CIFAR-10, which enables good standard accuracy may not suffice to train robust models.

Countering Noisy Labels By Learning From Auxiliary Clean Labels

no code implementations23 May 2019 Tsung Wei Tsai, Chongxuan Li, Jun Zhu

We consider the learning from noisy labels (NL) problem which emerges in many real-world applications.

Boosting Generative Models by Leveraging Cascaded Meta-Models

1 code implementation11 May 2019 Fan Bao, Hang Su, Jun Zhu

Besides, our framework can be extended to semi-supervised boosting, where the boosted model learns a joint distribution of data and labels.

$A^*$ sampling with probability matching

no code implementations ICLR 2019 Yichi Zhou, Jun Zhu

We provide insights into the relationship between $A^*$ sampling and probability matching by analyzing a nontrivial special case in which the state space is partitioned into two subsets.

Decision Making

Efficient Decision-based Black-box Adversarial Attacks on Face Recognition

no code implementations CVPR 2019 Yinpeng Dong, Hang Su, Baoyuan Wu, Zhifeng Li, Wei Liu, Tong Zhang, Jun Zhu

In this paper, we evaluate the robustness of state-of-the-art face recognition models in the decision-based black-box attack setting, where the attackers have no access to the model parameters and gradients, but can only acquire hard-label predictions by sending queries to the target model.

Face Recognition

Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks

1 code implementation CVPR 2019 Yinpeng Dong, Tianyu Pang, Hang Su, Jun Zhu

In this paper, we propose a translation-invariant attack method to generate more transferable adversarial examples against the defense models.

Cluster Alignment with a Teacher for Unsupervised Domain Adaptation

1 code implementation ICCV 2019 Zhijie Deng, Yucen Luo, Jun Zhu

Deep learning methods have shown promise in unsupervised domain adaptation, which aims to leverage a labeled source domain to learn a classifier for the unlabeled target domain with a different distribution.

Unsupervised Domain Adaptation

Function Space Particle Optimization for Bayesian Neural Networks

1 code implementation ICLR 2019 Ziyu Wang, Tongzheng Ren, Jun Zhu, Bo Zhang

While Bayesian neural networks (BNNs) have drawn increasing attention, their posterior inference remains challenging, due to the high-dimensional and over-parameterized nature.

Variational Inference

Batch Virtual Adversarial Training for Graph Convolutional Networks

no code implementations25 Feb 2019 Zhijie Deng, Yinpeng Dong, Jun Zhu

We present batch virtual adversarial training (BVAT), a novel regularization method for graph convolutional networks (GCNs).

General Classification Node Classification

Understanding MCMC Dynamics as Flows on the Wasserstein Space

1 code implementation1 Feb 2019 Chang Liu, Jingwei Zhuo, Jun Zhu

It is known that the Langevin dynamics used in MCMC is the gradient flow of the KL divergence on the Wasserstein space, which helps convergence analysis and inspires recent particle-based variational inference methods (ParVIs).

Variational Inference

Reward Shaping via Meta-Learning

no code implementations27 Jan 2019 Haosheng Zou, Tongzheng Ren, Dong Yan, Hang Su, Jun Zhu

Reward shaping is one of the most effective methods to tackle the crucial yet challenging problem of credit assignment in Reinforcement Learning (RL).

Meta-Learning

Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples

no code implementations25 Jan 2019 Yinpeng Dong, Fan Bao, Hang Su, Jun Zhu

3) We propose to improve the consistency of neurons on adversarial example subset by an adversarial training algorithm with a consistent loss.

Improving Adversarial Robustness via Promoting Ensemble Diversity

4 code implementations25 Jan 2019 Tianyu Pang, Kun Xu, Chao Du, Ning Chen, Jun Zhu

Though deep neural networks have achieved significant progress on various tasks, often enhanced by model ensemble, existing high-performance models can be vulnerable to adversarial attacks.

To Relieve Your Headache of Training an MRF, Take AdVIL

no code implementations ICLR 2020 Chongxuan Li, Chao Du, Kun Xu, Max Welling, Jun Zhu, Bo Zhang

We propose a black-box algorithm called {\it Adversarial Variational Inference and Learning} (AdVIL) to perform inference and learning on a general Markov random field (MRF).

Variational Inference

Stochastic Expectation Maximization with Variance Reduction

no code implementations NeurIPS 2018 Jianfei Chen, Jun Zhu, Yee Whye Teh, Tong Zhang

However, sEM has a slower asymptotic convergence rate than batch EM, and requires a decreasing sequence of step sizes, which is difficult to tune.

Latent Variable Models

Composite Binary Decomposition Networks

no code implementations16 Nov 2018 You Qiaoben, Zheng Wang, Jianguo Li, Yinpeng Dong, Yu-Gang Jiang, Jun Zhu

Binary neural networks have great resource and computing efficiency, while suffer from long training procedure and non-negligible accuracy drops, when comparing to the full-precision counterparts.

General Classification Image Classification +2

Semi-crowdsourced Clustering with Deep Generative Models

1 code implementation NeurIPS 2018 Yucen Luo, Tian Tian, Jiaxin Shi, Jun Zhu, Bo Zhang

We propose a new approach that includes a deep generative model (DGM) to characterize low-level features of the data, and a statistical relational model for noisy pairwise annotations on its subset.

Variational Inference

Lazy-CFR: fast and near optimal regret minimization for extensive games with imperfect information

no code implementations10 Oct 2018 Yichi Zhou, Tongzheng Ren, Jialian Li, Dong Yan, Jun Zhu

In this paper, we present a novel technique, lazy update, which can avoid traversing the whole game tree in CFR, as well as a novel analysis on the regret of CFR with lazy update.

Analyzing the Noise Robustness of Deep Neural Networks

no code implementations9 Oct 2018 Mengchen Liu, Shixia Liu, Hang Su, Kelei Cao, Jun Zhu

Deep neural networks (DNNs) are vulnerable to maliciously generated adversarial examples.

Direct Training for Spiking Neural Networks: Faster, Larger, Better

no code implementations16 Sep 2018 Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Luping Shi

Spiking neural networks (SNNs) that enables energy efficient implementation on emerging neuromorphic hardware are gaining more attention.

Deep Structured Generative Models

no code implementations10 Jul 2018 Kun Xu, Haoyu Liang, Jun Zhu, Hang Su, Bo Zhang

Deep generative models have shown promising results in generating realistic images, but it is still non-trivial to generate images with complicated structures.

Understanding and Accelerating Particle-Based Variational Inference

1 code implementation4 Jul 2018 Chang Liu, Jingwei Zhuo, Pengyu Cheng, Ruiyi Zhang, Jun Zhu, Lawrence Carin

Particle-based variational inference methods (ParVIs) have gained attention in the Bayesian inference literature, for their capacity to yield flexible and accurate approximations.

Bayesian Inference Variational Inference

A Spectral Approach to Gradient Estimation for Implicit Distributions

2 code implementations ICML 2018 Jiaxin Shi, Shengyang Sun, Jun Zhu

Recently there have been increasing interests in learning and inference with implicit distributions (i. e., distributions without tractable densities).

Variational Inference

Graphical Generative Adversarial Networks

1 code implementation NeurIPS 2018 Chongxuan Li, Max Welling, Jun Zhu, Bo Zhang

We propose Graphical Generative Adversarial Networks (Graphical-GAN) to model structured data.

Towards Training Probabilistic Topic Models on Neuromorphic Multi-chip Systems

no code implementations10 Apr 2018 Zihao Xiao, Jianfei Chen, Jun Zhu

We also propose an extension to train pLSI and a method to prune the network to obey the limited fan-in of some NMSs.

Stochastic Optimization Topic Models

Adversarial Attacks and Defences Competition

1 code implementation31 Mar 2018 Alexey Kurakin, Ian Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, Jun Zhu, Xiaolin Hu, Cihang Xie, Jian-Yu Wang, Zhishuai Zhang, Zhou Ren, Alan Yuille, Sangxia Huang, Yao Zhao, Yuzhe Zhao, Zhonglin Han, Junjiajia Long, Yerkebulan Berdibekov, Takuya Akiba, Seiya Tokui, Motoki Abe

To accelerate research on adversarial examples and robustness of machine learning classifiers, Google Brain organized a NIPS 2017 competition that encouraged researchers to develop new methods to generate adversarial examples as well as to develop new ways to defend against them.

Stochastic Gradient Hamiltonian Monte Carlo with Variance Reduction for Bayesian Inference

no code implementations29 Mar 2018 Zhize Li, Tianyi Zhang, Shuyu Cheng, Jun Zhu, Jian Li

In this paper, we apply the variance reduction tricks on Hamiltonian Monte Carlo and achieve better theoretical convergence results compared with the variance-reduced Langevin dynamics.

Bayesian Inference

Sparse Adversarial Perturbations for Videos

1 code implementation7 Mar 2018 Xingxing Wei, Jun Zhu, Hang Su

Although adversarial samples of deep neural networks (DNNs) have been intensively studied on static images, their extensions in videos are never explored.

Action Recognition

Max-Mahalanobis Linear Discriminant Analysis Networks

2 code implementations ICML 2018 Tianyu Pang, Chao Du, Jun Zhu

In this paper, we show that a properly designed classifier can improve robustness to adversarial attacks and lead to better prediction results.

Understanding Human Behaviors in Crowds by Imitating the Decision-Making Process

no code implementations25 Jan 2018 Haosheng Zou, Hang Su, Shihong Song, Jun Zhu

Crowd behavior understanding is crucial yet challenging across a wide range of applications, since crowd behavior is inherently determined by a sequential decision-making process based on various factors, such as the pedestrians' own destinations, interaction with nearby pedestrians and anticipation of upcoming events.

Decision Making Imitation Learning

Stochastic Training of Graph Convolutional Networks

no code implementations ICLR 2018 Jianfei Chen, Jun Zhu

Previous attempts on reducing the receptive field size by subsampling neighbors do not have any convergence guarantee, and their receptive field size per node is still in the order of hundreds.

A Hierarchical Recurrent Neural Network for Symbolic Melody Generation

2 code implementations14 Dec 2017 Jian Wu, Changran Hu, Yulong Wang, Xiaolin Hu, Jun Zhu

In this paper, we present a hierarchical recurrent neural network for melody generation, which consists of three Long-Short-Term-Memory (LSTM) subnetworks working in a coarse-to-fine manner along time.

Sound Multimedia

Learning Random Fourier Features by Hybrid Constrained Optimization

no code implementations7 Dec 2017 Jianqiao Wangni, Jingwei Zhuo, Jun Zhu

Since the algorithm consumes a major computation cost in the testing phase, we propose a novel teacher-learner framework of learning computation-efficient kernel embeddings from specific data.

Learning to Write Stylized Chinese Characters by Reading a Handful of Examples

no code implementations6 Dec 2017 Danyang Sun, Tongzheng Ren, Chongxun Li, Hang Su, Jun Zhu

Automatically writing stylized Chinese characters is an attractive yet challenging task due to its wide applicabilities.

Population Matching Discrepancy and Applications in Deep Learning

no code implementations NeurIPS 2017 Jianfei Chen, Chongxuan Li, Yizhong Ru, Jun Zhu

In this paper, we propose population matching discrepancy (PMD) for estimating the distribution distance based on samples, as well as an algorithm to learn the parameters of the distributions using PMD as an objective.

Domain Adaptation

Riemannian Stein Variational Gradient Descent for Bayesian Inference

1 code implementation30 Nov 2017 Chang Liu, Jun Zhu

The benefits are two-folds: (i) for inference tasks in Euclidean spaces, RSVGD has the advantage over SVGD of utilizing information geometry, and (ii) for inference tasks on Riemann manifolds, RSVGD brings the unique advantages of SVGD to the Riemannian world.

Bayesian Inference

Diversity-Promoting Bayesian Learning of Latent Variable Models

no code implementations23 Nov 2017 Pengtao Xie, Jun Zhu, Eric P. Xing

We also extend our approach to "diversify" Bayesian nonparametric models where the number of components is infinite.

Latent Variable Models Variational Inference

Visual Concepts and Compositional Voting

no code implementations13 Nov 2017 Jianyu Wang, Zhishuai Zhang, Cihang Xie, Yuyin Zhou, Vittal Premachandran, Jun Zhu, Lingxi Xie, Alan Yuille

We use clustering algorithms to study the population activities of the features and extract a set of visual concepts which we show are visually tight and correspond to semantic parts of vehicles.

Semantic Part Detection

Message Passing Stein Variational Gradient Descent

no code implementations ICML 2018 Jingwei Zhuo, Chang Liu, Jiaxin Shi, Jun Zhu, Ning Chen, Bo Zhang

Stein variational gradient descent (SVGD) is a recently proposed particle-based Bayesian inference method, which has attracted a lot of interest due to its remarkable approximation ability and particle efficiency compared to traditional variational inference and Markov Chain Monte Carlo methods.

Bayesian Inference Variational Inference

Smooth Neighbors on Teacher Graphs for Semi-supervised Learning

1 code implementation CVPR 2018 Yucen Luo, Jun Zhu, Mengxi Li, Yong Ren, Bo Zhang

In SNTG, a graph is constructed based on the predictions of the teacher model, i. e., the implicit self-ensemble of models.

Stochastic Training of Graph Convolutional Networks with Variance Reduction

1 code implementation ICML 2018 Jianfei Chen, Jun Zhu, Le Song

Previous attempts on reducing the receptive field size by subsampling neighbors do not have a convergence guarantee, and their receptive field size per node is still in the order of hundreds.

Boosting Adversarial Attacks with Momentum

5 code implementations CVPR 2018 Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, Jianguo Li

To further improve the success rates for black-box attacks, we apply momentum iterative algorithms to an ensemble of models, and show that the adversarially trained models with a strong defense ability are also vulnerable to our black-box attacks.

Adversarial Attack

ZhuSuan: A Library for Bayesian Deep Learning

1 code implementation18 Sep 2017 Jiaxin Shi, Jianfei Chen, Jun Zhu, Shengyang Sun, Yucen Luo, Yihong Gu, Yuhao Zhou

In this paper we introduce ZhuSuan, a python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and deep learning.

Probabilistic Programming

Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples

no code implementations18 Aug 2017 Yinpeng Dong, Hang Su, Jun Zhu, Fan Bao

We find that: (1) the neurons in DNNs do not truly detect semantic objects/parts, but respond to objects/parts only as recurrent discriminative patches; (2) deep visual representations are not robust distributed codes of visual concepts because the representations of adversarial images are largely not consistent with those of real images, although they have similar visual appearance, both of which are different from previous findings.

Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization

1 code implementation3 Aug 2017 Yinpeng Dong, Renkun Ni, Jianguo Li, Yurong Chen, Jun Zhu, Hang Su

This procedure can greatly compensate the quantization error and thus yield better accuracy for low-bit DNNs.

Quantization

Identify the Nash Equilibrium in Static Games with Random Payoffs

no code implementations ICML 2017 Yichi Zhou, Jialian Li, Jun Zhu

We study the problem on how to learn the pure Nash Equilibrium of a two-player zero-sum static game with random payoffs under unknown distributions via efficient payoff queries.

Detecting Semantic Parts on Partially Occluded Objects

no code implementations25 Jul 2017 Jianyu Wang, Cihang Xie, Zhishuai Zhang, Jun Zhu, Lingxi Xie, Alan Yuille

Our approach detects semantic parts by accumulating the confidence of local visual cues.

Semantic Part Detection

The YouTube-8M Kaggle Competition: Challenges and Methods

1 code implementation28 Jun 2017 Haosheng Zou, Kun Xu, Jialian Li, Jun Zhu

We took part in the YouTube-8M Video Understanding Challenge hosted on Kaggle, and achieved the 10th place within less than one month's time.

General Classification Video Classification +1

Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks

1 code implementation8 Jun 2017 Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Luping Shi

By simultaneously considering the layer-by-layer spatial domain (SD) and the timing-dependent temporal domain (TD) in the training phase, as well as an approximated derivative for the spike activity, we propose a spatio-temporal backpropagation (STBP) training framework without using any complicated technology.

Object Detection

Towards Robust Detection of Adversarial Examples

1 code implementation NeurIPS 2018 Tianyu Pang, Chao Du, Yinpeng Dong, Jun Zhu

Although the recent progress is substantial, deep learning methods can be vulnerable to the maliciously generated adversarial examples.

Kernel Implicit Variational Inference

no code implementations ICLR 2018 Jiaxin Shi, Shengyang Sun, Jun Zhu

Recent progress in variational inference has paid much attention to the flexibility of variational posteriors.

General Classification Variational Inference

Improving Interpretability of Deep Neural Networks with Semantic Information

no code implementations CVPR 2017 Yinpeng Dong, Hang Su, Jun Zhu, Bo Zhang

Interpretability of deep neural networks (DNNs) is essential since it enables users to understand the overall strengths and weaknesses of the models, conveys an understanding of how the models will behave in the future, and how to diagnose and correct potential problems.

Action Recognition Video Captioning

Triple Generative Adversarial Nets

1 code implementation NeurIPS 2017 Chongxuan Li, Kun Xu, Jun Zhu, Bo Zhang

Generative Adversarial Nets (GANs) have shown promise in image generation and semi-supervised learning (SSL).

Image Generation

Scalable Inference for Nested Chinese Restaurant Process Topic Models

no code implementations23 Feb 2017 Jianfei Chen, Jun Zhu, Jie Lu, Shixia Liu

Finally, we propose an efficient distributed implementation of PCGS through vectorization, pre-processing, and a careful design of the concurrent data structures and communication strategy.

Hierarchical structure Topic Models +1

Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspective

no code implementations4 Feb 2017 Shixia Liu, Xiting Wang, Mengchen Liu, Jun Zhu

Interactive model analysis, the process of understanding, diagnosing, and refining a machine learning model with the help of interactive visualization, is very important for users to efficiently solve real-world artificial intelligence and data mining problems.

Collaborative Filtering with User-Item Co-Autoregressive Models

1 code implementation21 Dec 2016 Chao Du, Chongxuan Li, Yin Zheng, Jun Zhu, Bo Zhang

Deep neural networks have shown promise in collaborative filtering (CF).

A Communication-Efficient Parallel Method for Group-Lasso

no code implementations7 Dec 2016 Binghong Chen, Jun Zhu

Group-Lasso (gLasso) identifies important explanatory factors in predicting the response variable by considering the grouping structure over input variables.

Stochastic Gradient Geodesic MCMC Methods

no code implementations NeurIPS 2016 Chang Liu, Jun Zhu, Yang song

We propose two stochastic gradient MCMC methods for sampling from Bayesian posterior distributions defined on Riemann manifolds with a known geodesic flow, e. g. hyperspheres.

Topic Models

SeDMiD for Confusion Detection: Uncovering Mind State from Time Series Brain Wave Data

no code implementations29 Nov 2016 Jingkang Yang, Haohan Wang, Jun Zhu, Eric P. Xing

In this paper, we propose an extension of State Space Model to work with different sources of information together with its learning and inference algorithms.

Time Series

Max-Margin Deep Generative Models for (Semi-)Supervised Learning

1 code implementation22 Nov 2016 Chongxuan Li, Jun Zhu, Bo Zhang

Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability.

SaberLDA: Sparsity-Aware Learning of Topic Models on GPUs

no code implementations8 Oct 2016 Kaiwei Li, Jianfei Chen, WenGuang Chen, Jun Zhu

Latent Dirichlet Allocation (LDA) is a popular tool for analyzing discrete count data such as text and images.

Topic Models

Kernel Bayesian Inference with Posterior Regularization

no code implementations NeurIPS 2016 Yang Song, Jun Zhu, Yong Ren

We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution.

Bayesian Inference

Conditional Generative Moment-Matching Networks

no code implementations NeurIPS 2016 Yong Ren, Jialian Li, Yucen Luo, Jun Zhu

Maximum mean discrepancy (MMD) has been successfully applied to learn deep generative models for characterizing a joint distribution of variables via kernel mean embedding.

PSDVec: a Toolbox for Incremental and Scalable Word Embedding

no code implementations10 Jun 2016 Shaohua Li, Jun Zhu, Chunyan Miao

PSDVec is a Python/Perl toolbox that learns word embeddings, i. e. the mapping of words in a natural language to continuous vectors which encode the semantic/syntactic regularities between the words.

Word Embeddings Word Similarity

Generative Topic Embedding: a Continuous Representation of Documents (Extended Version with Proofs)

1 code implementation9 Jun 2016 Shaohua Li, Tat-Seng Chua, Jun Zhu, Chunyan Miao

Word embedding maps words into a low-dimensional continuous embedding space by exploiting the local word collocation patterns in a small context window.

Document Classification Variational Inference

Towards Better Analysis of Deep Convolutional Neural Networks

no code implementations24 Apr 2016 Mengchen Liu, Jiaxin Shi, Zhen Li, Chongxuan Li, Jun Zhu, Shixia Liu

Deep convolutional neural networks (CNNs) have achieved breakthrough performance in many pattern recognition tasks such as image classification.

Image Classification

Learning to Generate with Memory

1 code implementation24 Feb 2016 Chongxuan Li, Jun Zhu, Bo Zhang

Memory units have been widely used to enrich the capabilities of deep networks on capturing long-term dependencies in reasoning and prediction tasks, but little investigation exists on deep generative models (DGMs) which are good at inferring high-level invariant representations from unlabeled data.

Density Estimation Image Generation +2

Max-Margin Nonparametric Latent Feature Models for Link Prediction

no code implementations24 Feb 2016 Jun Zhu, Jiaming Song, Bei Chen

Our approach attempts to unite the ideas of max-margin learning and Bayesian nonparametrics to discover discriminative latent features for link prediction.

Link Prediction Variational Inference

Spectral Learning for Supervised Topic Models

no code implementations19 Feb 2016 Yong Ren, Yining Wang, Jun Zhu

Spectral methods have been applied to learn unsupervised topic models, such as latent Dirichlet allocation (LDA), with provable guarantees.

Topic Models

Scaling up Dynamic Topic Models

1 code implementation19 Feb 2016 Arnab Bhadury, Jianfei Chen, Jun Zhu, Shixia Liu

Dynamic topic models (DTMs) are very effective in discovering topics and capturing their evolution trends in time series data.

Time Series Topic Models

Streaming Gibbs Sampling for LDA Model

no code implementations6 Jan 2016 Yang Gao, Jianfei Chen, Jun Zhu

Streaming variational Bayes (SVB) is successful in learning LDA models in an online manner.

Fast Parallel SVM using Data Augmentation

no code implementations24 Dec 2015 Hugh Perkins, Minjie Xu, Jun Zhu, Bo Zhang

As one of the most popular classifiers, linear SVMs still have challenges in dealing with very large-scale problems, even though linear or sub-linear algorithms have been developed recently on single machines.

Bayesian Inference Data Augmentation

Jointly Modeling Topics and Intents with Global Order Structure

no code implementations7 Dec 2015 Bei Chen, Jun Zhu, Nan Yang, Tian Tian, Ming Zhou, Bo Zhang

Modeling document structure is of great importance for discourse analysis and related applications.

Discriminative Nonparametric Latent Feature Relational Models with Data Augmentation

no code implementations7 Dec 2015 Bei Chen, Ning Chen, Jun Zhu, Jiaming Song, Bo Zhang

We present a discriminative nonparametric latent feature relational model (LFRM) for link prediction to automatically infer the dimensionality of latent features.

Bayesian Inference Data Augmentation +1

Building Memory with Concept Learning Capabilities from Large-scale Knowledge Base

no code implementations3 Dec 2015 Jiaxin Shi, Jun Zhu

We present a new perspective on neural knowledge base (KB) embeddings, from which we build a framework that can model symbolic knowledge in the KB together with its learning process.

Bayesian Matrix Completion via Adaptive Relaxed Spectral Regularization

1 code implementation3 Dec 2015 Yang Song, Jun Zhu

Bayesian matrix completion has been studied based on a low-rank matrix factorization formulation with promising results.

Bayesian Inference Matrix Completion

Max-Margin Majority Voting for Learning from Crowds

no code implementations NeurIPS 2015 Tian Tian, Jun Zhu

Learning-from-crowds aims to design proper aggregation strategies to infer the unknown true labels from the noisy labels provided by ordinary web workers.

Bayesian Inference

DeePM: A Deep Part-Based Model for Object Detection and Semantic Part Localization

no code implementations23 Nov 2015 Jun Zhu, Xianjie Chen, Alan L. Yuille

In this paper, we propose a deep part-based model (DeePM) for symbiotic object detection and semantic part localization.

Object Detection

WarpLDA: a Cache Efficient O(1) Algorithm for Latent Dirichlet Allocation

no code implementations29 Oct 2015 Jianfei Chen, Kaiwei Li, Jun Zhu, WenGuang Chen

We then develop WarpLDA, an LDA sampler which achieves both the best O(1) time complexity per token and the best O(K) scope of random access.

Pose-Guided Human Parsing with Deep Learned Features

no code implementations17 Aug 2015 Fangting Xia, Jun Zhu, Peng Wang, Alan Yuille

Parsing human body into semantic regions is crucial to human-centric analysis.

Human Parsing

A Generative Word Embedding Model and its Low Rank Positive Semidefinite Solution

1 code implementation EMNLP 2015 Shaohua Li, Jun Zhu, Chunyan Miao

Most existing word embedding methods can be categorized into Neural Embedding Models and Matrix Factorization (MF)-based methods.

Dropout Training for SVMs with Data Augmentation

no code implementations10 Aug 2015 Ning Chen, Jun Zhu, Jianfei Chen, Ting Chen

Empirical results on several real datasets demonstrate the effectiveness of dropout training on significantly boosting the classification accuracy of both linear and nonlinear SVMs.

Data Augmentation Representation Learning

Learning Deep Generative Models with Doubly Stochastic MCMC

no code implementations15 Jun 2015 Chao Du, Jun Zhu, Bo Zhang

We present doubly stochastic gradient MCMC, a simple and generic method for (approximate) Bayesian inference of deep generative models (DGMs) in a collapsed continuous parameter space.

Bayesian Inference Density Estimation +1

Bounded-Distortion Metric Learning

no code implementations10 May 2015 Renjie Liao, Jianping Shi, Ziyang Ma, Jun Zhu, Jiaya Jia

Metric learning aims to embed one metric space into another to benefit tasks like classification and clustering.

General Classification Metric Learning

Fast Sampling for Bayesian Max-Margin Models

no code implementations27 Apr 2015 Wenbo Hu, Jun Zhu, Bo Zhang

Bayesian max-margin models have shown superiority in various practical applications, such as text categorization, collaborative prediction, social network link prediction and crowdsourcing, and they conjoin the flexibility of Bayesian modeling and predictive strengths of max-margin learning.

Link Prediction Text Categorization

Max-margin Deep Generative Models

2 code implementations NeurIPS 2015 Chongxuan Li, Jun Zhu, Tianlin Shi, Bo Zhang

Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability.

Spectral Methods for Supervised Topic Models

no code implementations NeurIPS 2014 Yining Wang, Jun Zhu

Supervised topic models simultaneously model the latent topic structure of large collections of documents and a response variable associated with each document.

Topic Models

Robust Bayesian Max-Margin Clustering

no code implementations NeurIPS 2014 Changyou Chen, Jun Zhu, Xinhua Zhang

We present max-margin Bayesian clustering (BMC), a general and robust framework that incorporates the max-margin criterion into Bayesian clustering models, as well as two concrete models of BMC to demonstrate its flexibility and effectiveness in dealing with different clustering tasks.

Learning From Weakly Supervised Data by The Expectation Loss SVM (e-SVM) algorithm

no code implementations NeurIPS 2014 Jun Zhu, Junhua Mao, Alan L. Yuille

We propose a novel learning algorithm called \emph{expectation loss SVM} (e-SVM) that is devoted to the problems where only the ``positiveness" instead of a binary label of each training sample is available.

Object Detection Semantic Segmentation

Distributed Bayesian Posterior Sampling via Moment Sharing

no code implementations NeurIPS 2014 Minjie Xu, Balaji Lakshminarayanan, Yee Whye Teh, Jun Zhu, Bo Zhang

We propose a distributed Markov chain Monte Carlo (MCMC) inference algorithm for large scale Bayesian posterior simulation.

Big Learning with Bayesian Methods

no code implementations24 Nov 2014 Jun Zhu, Jianfei Chen, Wen-Bo Hu, Bo Zhang

Explosive growth in data and availability of cheap computing resources have sparked increasing interest in Big learning, an emerging subfield that studies scalable machine learning algorithms, systems, and applications with Big Data.

Bayesian Inference Distributed Computing

Contrastive Feature Induction for Efficient Structure Learning of Conditional Random Fields

no code implementations28 Jun 2014 Ni Lao, Jun Zhu

We prove that the gradient of candidate features can be represented solely as a function of signals and errors, and that CFI is an efficient approximation of gradient-based evaluation methods.

Feature Selection Relational Reasoning