Search Results for author: Jun Zhu

Found 325 papers, 158 papers with code

DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection

14 code implementations7 Mar 2022 Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M. Ni, Heung-Yeung Shum

Compared to other models on the leaderboard, DINO significantly reduces its model size and pre-training data size while achieving better results.

Real-Time Object Detection

Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

5 code implementations9 Mar 2023 Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang

To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion.

Referring Expression Referring Expression Comprehension +2

Stochastic Training of Graph Convolutional Networks with Variance Reduction

2 code implementations ICML 2018 Jianfei Chen, Jun Zhu, Le Song

Previous attempts on reducing the receptive field size by subsampling neighbors do not have a convergence guarantee, and their receptive field size per node is still in the order of hundreds.

Tianshou: a Highly Modularized Deep Reinforcement Learning Library

1 code implementation29 Jul 2021 Jiayi Weng, Huayu Chen, Dong Yan, Kaichao You, Alexis Duburcq, Minghao Zhang, Yi Su, Hang Su, Jun Zhu

In this paper, we present Tianshou, a highly modularized Python library for deep reinforcement learning (DRL) that uses PyTorch as its backend.

reinforcement-learning Reinforcement Learning (RL)

One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale

3 code implementations12 Mar 2023 Fan Bao, Shen Nie, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yue, Yue Cao, Hang Su, Jun Zhu

Inspired by the unified view, UniDiffuser learns all distributions simultaneously with a minimal modification to the original diffusion model -- perturbs data in all modalities instead of a single modality, inputs individual timesteps in different modalities, and predicts the noise of all modalities instead of a single modality.

Text-to-Image Generation

ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation

2 code implementations NeurIPS 2023 Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, Jun Zhu

In comparison, VSD works well with various CFG weights as ancestral sampling from diffusion models and simultaneously improves the diversity and sample quality with a common CFG weight (i. e., $7. 5$).

Text to 3D

Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning

3 code implementations25 Apr 2023 Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, Jun Zhu

The main challenge for this setting is that the intermediate guidance during the diffusion sampling procedure, which is jointly defined by the sampling distribution and the energy function, is unknown and is hard to estimate.

D4RL Image Generation +1

ZhuSuan: A Library for Bayesian Deep Learning

1 code implementation18 Sep 2017 Jiaxin Shi, Jianfei Chen, Jun Zhu, Shengyang Sun, Yucen Luo, Yihong Gu, Yuhao Zhou

In this paper we introduce ZhuSuan, a python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and deep learning.

Probabilistic Programming regression

DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR

7 code implementations ICLR 2022 Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, Lei Zhang

We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR.

Object Detection

DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models

1 code implementation2 Nov 2022 Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu

The commonly-used fast sampler for guided sampling is DDIM, a first-order diffusion ODE solver that generally needs 100 to 250 steps for high-quality samples.

Text-to-Image Generation

All are Worth Words: A ViT Backbone for Diffusion Models

3 code implementations CVPR 2023 Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, Jun Zhu

We evaluate U-ViT in unconditional and class-conditional image generation, as well as text-to-image generation tasks, where U-ViT is comparable if not superior to a CNN-based U-Net of a similar size.

Conditional Image Generation Text-to-Image Generation

Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels

2 code implementations NeurIPS 2023 Zebin You, Yong Zhong, Fan Bao, Jiacheng Sun, Chongxuan Li, Jun Zhu

In an effort to further advance semi-supervised generative and classification tasks, we propose a simple yet effective training strategy called dual pseudo training (DPT), built upon strong semi-supervised learners and diffusion models.

Classification

Bag of Tricks for Adversarial Training

2 code implementations ICLR 2021 Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, Jun Zhu

Adversarial training (AT) is one of the most effective strategies for promoting model robustness.

Adversarial Robustness Benchmarking

RobFR: Benchmarking Adversarial Robustness on Face Recognition

2 code implementations8 Jul 2020 Xiao Yang, Dingcheng Yang, Yinpeng Dong, Hang Su, Wenjian Yu, Jun Zhu

Based on large-scale evaluations, the commercial FR API services fail to exhibit acceptable performance on robustness evaluation, and we also draw several important conclusions for understanding the adversarial robustness of FR models and providing insights for the design of robust FR models.

Adversarial Robustness Benchmarking +1

Query2Label: A Simple Transformer Way to Multi-Label Classification

2 code implementations22 Jul 2021 Shilong Liu, Lei Zhang, Xiao Yang, Hang Su, Jun Zhu

The use of Transformer is rooted in the need of extracting local discriminative features adaptively for different labels, which is a strongly desired property due to the existence of multiple objects in one image.

Classification Multi-Label Classification

Counter-Strike Deathmatch with Large-Scale Behavioural Cloning

2 code implementations9 Apr 2021 Tim Pearce, Jun Zhu

This paper describes an AI agent that plays the popular first-person-shooter (FPS) video game `Counter-Strike; Global Offensive' (CSGO) from pixel input.

Behavioural cloning FPS Games

Boosting Adversarial Attacks with Momentum

7 code implementations CVPR 2018 Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, Jianguo Li

To further improve the success rates for black-box attacks, we apply momentum iterative algorithms to an ensemble of models, and show that the adversarially trained models with a strong defense ability are also vulnerable to our black-box attacks.

Adversarial Attack

Triple Generative Adversarial Nets

1 code implementation NeurIPS 2017 Chongxuan Li, Kun Xu, Jun Zhu, Bo Zhang

Generative Adversarial Nets (GANs) have shown promise in image generation and semi-supervised learning (SSL).

Image Generation

ControlVideo: Conditional Control for One-shot Text-driven Video Editing and Beyond

1 code implementation26 May 2023 Min Zhao, Rongzhen Wang, Fan Bao, Chongxuan Li, Jun Zhu

This paper presents \emph{ControlVideo} for text-driven video editing -- generating a video that aligns with a given text while preserving the structure of the source video.

Text-to-Video Editing Video Editing

Generative Topic Embedding: a Continuous Representation of Documents (Extended Version with Proofs)

1 code implementation9 Jun 2016 Shaohua Li, Tat-Seng Chua, Jun Zhu, Chunyan Miao

Word embedding maps words into a low-dimensional continuous embedding space by exploiting the local word collocation patterns in a small context window.

Document Classification Variational Inference

A Generative Word Embedding Model and its Low Rank Positive Semidefinite Solution

1 code implementation EMNLP 2015 Shaohua Li, Jun Zhu, Chunyan Miao

Most existing word embedding methods can be categorized into Neural Embedding Models and Matrix Factorization (MF)-based methods.

EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations

1 code implementation14 Jul 2022 Min Zhao, Fan Bao, Chongxuan Li, Jun Zhu

Further, we provide an alternative explanation of the EGSDE as a product of experts, where each of the three experts (corresponding to the SDE and two feature extractors) solely contributes to faithfulness or realism.

Image-to-Image Translation Translation

Equivariant Energy-Guided SDE for Inverse Molecular Design

2 code implementations30 Sep 2022 Fan Bao, Min Zhao, Zhongkai Hao, Peiyao Li, Chongxuan Li, Jun Zhu

Inverse molecular design is critical in material science and drug discovery, where the generated molecules should satisfy certain desirable properties.

3D Molecule Generation Drug Discovery

Detection Transformer with Stable Matching

1 code implementation ICCV 2023 Shilong Liu, Tianhe Ren, Jiayu Chen, Zhaoyang Zeng, Hao Zhang, Feng Li, Hongyang Li, Jun Huang, Hang Su, Jun Zhu, Lei Zhang

We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR.

Position

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

2 code implementations ICLR 2022 Fan Bao, Chongxuan Li, Jun Zhu, Bo Zhang

In this work, we present a surprising result that both the optimal reverse variance and the corresponding optimal KL divergence of a DPM have analytic forms w. r. t.

Adversarial Attacks and Defences Competition

1 code implementation31 Mar 2018 Alexey Kurakin, Ian Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, Jun Zhu, Xiaolin Hu, Cihang Xie, Jian-Yu Wang, Zhishuai Zhang, Zhou Ren, Alan Yuille, Sangxia Huang, Yao Zhao, Yuzhe Zhao, Zhonglin Han, Junjiajia Long, Yerkebulan Berdibekov, Takuya Akiba, Seiya Tokui, Motoki Abe

To accelerate research on adversarial examples and robustness of machine learning classifiers, Google Brain organized a NIPS 2017 competition that encouraged researchers to develop new methods to generate adversarial examples as well as to develop new ways to defend against them.

BIG-bench Machine Learning

Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks

2 code implementations CVPR 2019 Yinpeng Dong, Tianyu Pang, Hang Su, Jun Zhu

In this paper, we propose a translation-invariant attack method to generate more transferable adversarial examples against the defense models.

Translation

Training Transformers with 4-bit Integers

1 code implementation NeurIPS 2023 Haocheng Xi, Changhao Li, Jianfei Chen, Jun Zhu

To achieve this, we carefully analyze the specific structures of activation and gradients in transformers to propose dedicated quantizers for them.

Image Classification Machine Translation +1

Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications

1 code implementation15 Nov 2022 Zhongkai Hao, Songming Liu, Yichi Zhang, Chengyang Ying, Yao Feng, Hang Su, Jun Zhu

Recent work shows that it provides potential benefits for machine learning models by incorporating the physical prior and collected data, which makes the intersection of machine learning and physics become a prevailing paradigm.

Physics-informed machine learning

PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs

1 code implementation15 Jun 2023 Zhongkai Hao, Jiachen Yao, Chang Su, Hang Su, Ziao Wang, Fanzhi Lu, Zeyu Xia, Yichi Zhang, Songming Liu, Lu Lu, Jun Zhu

In addition to providing a standardized means of assessing performance, PINNacle also offers an in-depth analysis to guide future research, particularly in areas such as domain decomposition methods and loss reweighting for handling multi-scale problems and complex geometry.

Benchmarking

TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations

1 code implementation9 Oct 2021 Shiyu Huang, Wenze Chen, Longfei Zhang, Shizhen Xu, Ziyang Li, Fengming Zhu, Deheng Ye, Ting Chen, Jun Zhu

To the best of our knowledge, Tikick is the first learning-based AI system that can take over the multi-agent Google Research Football full game, while previous work could either control a single agent or experiment on toy academic scenarios.

Starcraft Starcraft II

Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models

1 code implementation15 Jun 2022 Fan Bao, Chongxuan Li, Jiacheng Sun, Jun Zhu, Bo Zhang

Thus, the generation performance on a subset of timesteps is crucial, which is greatly influenced by the covariance design in DPMs.

Computational Efficiency

Max-Mahalanobis Linear Discriminant Analysis Networks

2 code implementations ICML 2018 Tianyu Pang, Chao Du, Jun Zhu

In this paper, we show that a properly designed classifier can improve robustness to adversarial attacks and lead to better prediction results.

Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness

2 code implementations ICLR 2020 Tianyu Pang, Kun Xu, Yinpeng Dong, Chao Du, Ning Chen, Jun Zhu

Previous work shows that adversarially robust generalization requires larger sample complexity, and the same dataset, e. g., CIFAR-10, which enables good standard accuracy may not suffice to train robust models.

Adversarial Robustness

Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings

1 code implementation14 Dec 2020 Qipeng Guo, Zhijing Jin, Ziyu Wang, Xipeng Qiu, Weinan Zhang, Jun Zhu, Zheng Zhang, David Wipf

Cycle-consistent training is widely used for jointly learning a forward and inverse mapping between two domains of interest without the cumbersome requirement of collecting matched pairs within each domain.

Knowledge Graphs Text Generation

How Robust is Google's Bard to Adversarial Image Attacks?

1 code implementation21 Sep 2023 Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu

By attacking white-box surrogate vision encoders or MLLMs, the generated adversarial examples can mislead Bard to output wrong image descriptions with a 22% success rate based solely on the transferability.

Adversarial Robustness Chatbot +1

DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics

1 code implementation NeurIPS 2023 Kaiwen Zheng, Cheng Lu, Jianfei Chen, Jun Zhu

In this work, we propose a novel formulation towards the optimal parameterization during sampling that minimizes the first-order discretization error of the ODE solution.

Image Generation

Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization

1 code implementation3 Aug 2017 Yinpeng Dong, Renkun Ni, Jianguo Li, Yurong Chen, Jun Zhu, Hang Su

This procedure can greatly compensate the quantization error and thus yield better accuracy for low-bit DNNs.

Quantization

Graphical Generative Adversarial Networks

1 code implementation NeurIPS 2018 Chongxuan Li, Max Welling, Jun Zhu, Bo Zhang

We propose Graphical Generative Adversarial Networks (Graphical-GAN) to model structured data.

Robustness and Accuracy Could Be Reconcilable by (Proper) Definition

1 code implementation21 Feb 2022 Tianyu Pang, Min Lin, Xiao Yang, Jun Zhu, Shuicheng Yan

The trade-off between robustness and accuracy has been widely studied in the adversarial literature.

Inductive Bias

Improving Adversarial Robustness via Promoting Ensemble Diversity

6 code implementations25 Jan 2019 Tianyu Pang, Kun Xu, Chao Du, Ning Chen, Jun Zhu

Though deep neural networks have achieved significant progress on various tasks, often enhanced by model ensemble, existing high-performance models can be vulnerable to adversarial attacks.

Adversarial Robustness

Adversarial Distributional Training for Robust Deep Learning

1 code implementation NeurIPS 2020 Yinpeng Dong, Zhijie Deng, Tianyu Pang, Hang Su, Jun Zhu

Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples.

Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks

1 code implementation ICLR 2020 Tianyu Pang, Kun Xu, Jun Zhu

Our experiments on CIFAR-10 and CIFAR-100 demonstrate that MI can further improve the adversarial robustness for the models trained by mixup and its variants.

Adversarial Robustness

Towards a General Framework for Continual Learning with Pre-training

1 code implementation21 Oct 2023 Liyuan Wang, Jingyi Xie, Xingxing Zhang, Hang Su, Jun Zhu

In this work, we present a general framework for continual learning of sequentially arrived tasks with the use of pre-training, which has emerged as a promising direction for artificial intelligence systems to accommodate real-world dynamics.

Continual Learning

Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality

1 code implementation NeurIPS 2023 Liyuan Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, Jun Zhu

Following these empirical and theoretical insights, we propose Hierarchical Decomposition (HiDe-)Prompt, an innovative approach that explicitly optimizes the hierarchical components with an ensemble of task-specific prompts and statistics of both uninstructed and instructed representations, further with the coordination of a contrastive regularization strategy.

Continual Learning

DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding

1 code implementation28 Nov 2022 Shilong Liu, Yaoyuan Liang, Feng Li, Shijia Huang, Hao Zhang, Hang Su, Jun Zhu, Lei Zhang

As phrase extraction can be regarded as a $1$D text segmentation problem, we formulate PEG as a dual detection problem and propose a novel DQ-DETR model, which introduces dual queries to probe different features from image and text for object prediction and phrase mask prediction.

object-detection Object Detection +4

MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering

1 code implementation ICLR 2021 Tsung Wei Tsai, Chongxuan Li, Jun Zhu

We present Mixture of Contrastive Experts (MiCE), a unified probabilistic clustering framework that simultaneously exploits the discriminative representations learned by contrastive learning and the semantic structures captured by a latent mixture model.

Clustering Contrastive Learning +1

Towards Robust Detection of Adversarial Examples

1 code implementation NeurIPS 2018 Tianyu Pang, Chao Du, Yinpeng Dong, Jun Zhu

Although the recent progress is substantial, deep learning methods can be vulnerable to the maliciously generated adversarial examples.

Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matching

1 code implementation16 Jun 2022 Cheng Lu, Kaiwen Zheng, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu

To fill up this gap, we show that the negative likelihood of the ODE can be bounded by controlling the first, second, and third-order score matching errors; and we further present a novel high-order denoising score matching method to enable maximum likelihood training of score-based diffusion ODEs.

Denoising

Smooth Neighbors on Teacher Graphs for Semi-supervised Learning

1 code implementation CVPR 2018 Yucen Luo, Jun Zhu, Mengxi Li, Yong Ren, Bo Zhang

In SNTG, a graph is constructed based on the predictions of the teacher model, i. e., the implicit self-ensemble of models.

CodeS: Towards Building Open-source Language Models for Text-to-SQL

1 code implementation26 Feb 2024 Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang, Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, Hong Chen

To address the limitations, we introduce CodeS, a series of pre-trained language models with parameters ranging from 1B to 15B, specifically designed for the text-to-SQL task.

Data Augmentation Domain Adaptation +2

Scalable Training of Inference Networks for Gaussian-Process Models

2 code implementations27 May 2019 Jiaxin Shi, Mohammad Emtiyaz Khan, Jun Zhu

Inference in Gaussian process (GP) models is computationally challenging for large data, and often difficult to approximate with a small number of inducing points.

GNOT: A General Neural Operator Transformer for Operator Learning

2 code implementations28 Feb 2023 Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, Jun Zhu

However, there are several challenges for learning operators in practical applications like the irregular mesh, multiple input functions, and complexity of the PDEs' solution.

Operator learning

Improving Black-box Adversarial Attacks with a Transfer-based Prior

2 code implementations NeurIPS 2019 Shuyu Cheng, Yinpeng Dong, Tianyu Pang, Hang Su, Jun Zhu

We consider the black-box adversarial setting, where the adversary has to generate adversarial perturbations without access to the target models to compute gradients.

Query-Efficient Black-box Adversarial Attacks Guided by a Transfer-based Prior

1 code implementation13 Mar 2022 Yinpeng Dong, Shuyu Cheng, Tianyu Pang, Hang Su, Jun Zhu

However, the existing methods inevitably suffer from low attack success rates or poor query efficiency since it is difficult to estimate the gradient in a high-dimensional input space with limited information.

A Comprehensive Survey of Continual Learning: Theory, Method and Application

1 code implementation31 Jan 2023 Liyuan Wang, Xingxing Zhang, Hang Su, Jun Zhu

To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime.

Continual Learning Learning Theory

Triple Generative Adversarial Networks

1 code implementation20 Dec 2019 Chongxuan Li, Kun Xu, Jiashuo Liu, Jun Zhu, Bo Zhang

It is formulated as a three-player minimax game consisting of a generator, a classifier and a discriminator, and therefore is referred to as Triple Generative Adversarial Network (Triple-GAN).

Classification Conditional Image Generation +4

Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling

1 code implementation29 Sep 2022 Huayu Chen, Cheng Lu, Chengyang Ying, Hang Su, Jun Zhu

To address this problem, we adopt a generative approach by decoupling the learned policy into two parts: an expressive generative behavior model and an action evaluation model.

Computational Efficiency D4RL +4

Towards Face Encryption by Generating Adversarial Identity Masks

1 code implementation ICCV 2021 Xiao Yang, Yinpeng Dong, Tianyu Pang, Hang Su, Jun Zhu, Yuefeng Chen, Hui Xue

As billions of personal data being shared through social media and network, the data privacy and security have drawn an increasing attention.

Face Recognition

VFlow: More Expressive Generative Flows with Variational Data Augmentation

1 code implementation ICML 2020 Jianfei Chen, Cheng Lu, Biqi Chenli, Jun Zhu, Tian Tian

Generative flows are promising tractable models for density modeling that define probabilistic distributions with invertible transformations.

Ranked #30 on Image Generation on CIFAR-10 (bits/dimension metric)

Density Estimation Image Generation +2

Nonparametric Score Estimators

1 code implementation ICML 2020 Yuhao Zhou, Jiaxin Shi, Jun Zhu

Estimating the score, i. e., the gradient of log density function, from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models that involve flexible yet intractable densities.

Implicit Normalizing Flows

1 code implementation ICLR 2021 Cheng Lu, Jianfei Chen, Chongxuan Li, Qiuhao Wang, Jun Zhu

Through theoretical analysis, we show that the function space of ImpFlow is strictly richer than that of ResFlows.

INT: Towards Infinite-frames 3D Detection with An Efficient Framework

1 code implementation30 Sep 2022 Jianyun Xu, Zhenwei Miao, Da Zhang, Hongyu Pan, Kaixuan Liu, Peihan Hao, Jun Zhu, Zhengyang Sun, Hongmin Li, Xin Zhan

By employing INT on CenterPoint, we can get around 7% (Waymo) and 15% (nuScenes) performance boost with only 2~4ms latency overhead, and currently SOTA on the Waymo 3D Detection leaderboard.

Boosting Adversarial Training with Hypersphere Embedding

1 code implementation NeurIPS 2020 Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, Hang Su

Adversarial training (AT) is one of the most effective defenses against adversarial attacks for deep learning models.

Representation Learning

A Closer Look at Parameter-Efficient Tuning in Diffusion Models

1 code implementation31 Mar 2023 Chendong Xiang, Fan Bao, Chongxuan Li, Hang Su, Jun Zhu

Large-scale diffusion models like Stable Diffusion are powerful and find various real-world applications while customizing such models by fine-tuning is both memory and time inefficient.

Efficient Diffusion Personalization Position

Robust Classification via a Single Diffusion Model

2 code implementations24 May 2023 Huanran Chen, Yinpeng Dong, Zhengyi Wang, Xiao Yang, Chengqi Duan, Hang Su, Jun Zhu

Since our method does not require training on particular adversarial attacks, we demonstrate that it is more generalizable to defend against multiple unseen threats.

Adversarial Defense Adversarial Robustness +2

A Spectral Approach to Gradient Estimation for Implicit Distributions

3 code implementations ICML 2018 Jiaxin Shi, Shengyang Sun, Jun Zhu

Recently there have been increasing interests in learning and inference with implicit distributions (i. e., distributions without tractable densities).

Variational Inference

Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart

1 code implementation CVPR 2022 Tianyu Pang, Huishuai Zhang, Di He, Yinpeng Dong, Hang Su, Wei Chen, Jun Zhu, Tie-Yan Liu

Along with this routine, we find that confidence and a rectified confidence (R-Con) can form two coupled rejection metrics, which could provably distinguish wrongly classified inputs from correctly classified ones.

Vocal Bursts Valence Prediction

Towards Effective Adversarial Textured 3D Meshes on Physical Face Recognition

1 code implementation CVPR 2023 Xiao Yang, Chang Liu, Longlong Xu, Yikai Wang, Yinpeng Dong, Ning Chen, Hang Su, Jun Zhu

The goal of this work is to develop a more reliable technique that can carry out an end-to-end evaluation of adversarial robustness for commercial systems.

Adversarial Robustness Face Recognition

BayesAdapter: Being Bayesian, Inexpensively and Reliably, via Bayesian Fine-tuning

1 code implementation5 Oct 2020 Zhijie Deng, Jun Zhu

Despite their theoretical appealingness, Bayesian neural networks (BNNs) are left behind in real-world adoption, mainly due to persistent concerns on their scalability, accessibility, and reliability.

Variational Inference

Training Interpretable Convolutional Neural Networks by Differentiating Class-specific Filters

1 code implementation ECCV 2020 Haoyu Liang, Zhihao Ouyang, Yuyuan Zeng, Hang Su, Zihao He, Shu-Tao Xia, Jun Zhu, Bo Zhang

Most existing works attempt post-hoc interpretation on a pre-trained model, while neglecting to reduce the entanglement underlying the model.

Object Localization

Cluster Alignment with a Teacher for Unsupervised Domain Adaptation

1 code implementation ICCV 2019 Zhijie Deng, Yucen Luo, Jun Zhu

Deep learning methods have shown promise in unsupervised domain adaptation, which aims to leverage a labeled source domain to learn a classifier for the unlabeled target domain with a different distribution.

Clustering Unsupervised Domain Adaptation

Score Regularized Policy Optimization through Diffusion Behavior

1 code implementation11 Oct 2023 Huayu Chen, Cheng Lu, Zhengyi Wang, Hang Su, Jun Zhu

Recent developments in offline reinforcement learning have uncovered the immense potential of diffusion modeling, which excels at representing heterogeneous behavior policies.

D4RL

AFEC: Active Forgetting of Negative Transfer in Continual Learning

1 code implementation NeurIPS 2021 Liyuan Wang, Mingtian Zhang, Zhongfan Jia, Qian Li, Chenglong Bao, Kaisheng Ma, Jun Zhu, Yi Zhong

Without accessing to the old training samples, knowledge transfer from the old tasks to each new task is difficult to determine, which might be either positive or negative.

Continual Learning Transfer Learning

Towards Efficient and Exact Optimization of Language Model Alignment

1 code implementation1 Feb 2024 Haozhe Ji, Cheng Lu, Yilin Niu, Pei Ke, Hongning Wang, Jun Zhu, Jie Tang, Minlie Huang

We prove that EXO is guaranteed to optimize in the same direction as the RL algorithms asymptotically for arbitary parametrization of the policy, while enables efficient optimization by circumventing the complexities associated with RL algorithms.

Language Modelling Reinforcement Learning (RL)

Understanding and Accelerating Particle-Based Variational Inference

1 code implementation4 Jul 2018 Chang Liu, Jingwei Zhuo, Pengyu Cheng, Ruiyi Zhang, Jun Zhu, Lawrence Carin

Particle-based variational inference methods (ParVIs) have gained attention in the Bayesian inference literature, for their capacity to yield flexible and accurate approximations.

Bayesian Inference Variational Inference

A Hierarchical Recurrent Neural Network for Symbolic Melody Generation

2 code implementations14 Dec 2017 Jian Wu, Changran Hu, Yulong Wang, Xiaolin Hu, Jun Zhu

In this paper, we present a hierarchical recurrent neural network for melody generation, which consists of three Long-Short-Term-Memory (LSTM) subnetworks working in a coarse-to-fine manner along time.

Sound Multimedia

Max-margin Deep Generative Models

2 code implementations NeurIPS 2015 Chongxuan Li, Jun Zhu, Tianlin Shi, Bo Zhang

Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability.

Exploring Memorization in Adversarial Training

1 code implementation ICLR 2022 Yinpeng Dong, Ke Xu, Xiao Yang, Tianyu Pang, Zhijie Deng, Hang Su, Jun Zhu

In this paper, we explore the memorization effect in adversarial training (AT) for promoting a deeper understanding of model capacity, convergence, generalization, and especially robust overfitting of the adversarially trained models.

Memorization

Accumulative Poisoning Attacks on Real-time Data

1 code implementation NeurIPS 2021 Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, Jun Zhu

Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy.

Federated Learning

Learning CLIP Guided Visual-Text Fusion Transformer for Video-based Pedestrian Attribute Recognition

1 code implementation20 Apr 2023 Jun Zhu, Jiandong Jin, Zihan Yang, Xiaohao Wu, Xiao Wang

The averaged visual tokens and text tokens are concatenated and fed into a fusion Transformer for multi-modal interactive learning.

Attribute Pedestrian Attribute Recognition +1

Fast Lossless Neural Compression with Integer-Only Discrete Flows

1 code implementation17 Jun 2022 Siyu Wang, Jianfei Chen, Chongxuan Li, Jun Zhu, Bo Zhang

In this work, we propose Integer-only Discrete Flows (IODF), an efficient neural compressor with integer-only arithmetic.

Quantization

Accelerated Linearized Laplace Approximation for Bayesian Deep Learning

1 code implementation23 Oct 2022 Zhijie Deng, Feng Zhou, Jun Zhu

Laplace approximation (LA) and its linearized variant (LLA) enable effortless adaptation of pretrained deep neural networks to Bayesian neural networks.

PREIM3D: 3D Consistent Precise Image Attribute Editing from a Single Image

1 code implementation CVPR 2023 Jianhui Li, Jianmin Li, Haoji Zhang, Shilong Liu, Zhengyi Wang, Zihao Xiao, Kaiwen Zheng, Jun Zhu

As for imprecise image editing, we attribute the problem to the gap between the latent space of real images and that of generated images.

Attribute

Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence

1 code implementation29 Aug 2023 Liyuan Wang, Xingxing Zhang, Qian Li, Mingtian Zhang, Hang Su, Jun Zhu, Yi Zhong

Continual learning aims to empower artificial intelligence (AI) with strong adaptability to the real world.

Continual Learning

Function Space Particle Optimization for Bayesian Neural Networks

1 code implementation ICLR 2019 Ziyu Wang, Tongzheng Ren, Jun Zhu, Bo Zhang

While Bayesian neural networks (BNNs) have drawn increasing attention, their posterior inference remains challenging, due to the high-dimensional and over-parameterized nature.

Variational Inference

Ranking Cost: Building An Efficient and Scalable Circuit Routing Planner with Evolution-Based Optimization

1 code implementation8 Oct 2021 Shiyu Huang, Bin Wang, Dong Li, Jianye Hao, Ting Chen, Jun Zhu

In this work, we propose a new algorithm for circuit routing, named Ranking Cost, which innovatively combines search-based methods (i. e., A* algorithm) and learning-based methods (i. e., Evolution Strategies) to form an efficient and trainable router.

NeuralEF: Deconstructing Kernels by Deep Neural Networks

2 code implementations30 Apr 2022 Zhijie Deng, Jiaxin Shi, Jun Zhu

Learning the principal eigenfunctions of an integral operator defined by a kernel and a data distribution is at the core of many machine learning problems.

Image Classification

NUNO: A General Framework for Learning Parametric PDEs with Non-Uniform Data

1 code implementation30 May 2023 Songming Liu, Zhongkai Hao, Chengyang Ying, Hang Su, Ze Cheng, Jun Zhu

The neural operator has emerged as a powerful tool in learning mappings between function spaces in PDEs.

Operator learning

Preserving Pre-trained Features Helps Calibrate Fine-tuned Language Models

1 code implementation30 May 2023 Guande He, Jianfei Chen, Jun Zhu

In light of these observations, we evaluate the calibration of several methods that preserve pre-trained features and show that preserving pre-trained features can improve the calibration of fine-tuned language models.

Language Modelling Masked Language Modeling +1

Sparse Adversarial Perturbations for Videos

1 code implementation7 Mar 2018 Xingxing Wei, Jun Zhu, Hang Su

Although adversarial samples of deep neural networks (DNNs) have been intensively studied on static images, their extensions in videos are never explored.

Action Recognition Temporal Action Localization

Memory Replay with Data Compression for Continual Learning

1 code implementation ICLR 2022 Liyuan Wang, Xingxing Zhang, Kuo Yang, Longhui Yu, Chongxuan Li, Lanqing Hong, Shifeng Zhang, Zhenguo Li, Yi Zhong, Jun Zhu

In this work, we propose memory replay with data compression (MRDC) to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer.

Autonomous Driving Class Incremental Learning +5

Revisiting Discriminative vs. Generative Classifiers: Theory and Implications

1 code implementation5 Feb 2023 Chenyu Zheng, Guoqiang Wu, Fan Bao, Yue Cao, Chongxuan Li, Jun Zhu

Theoretically, the paper considers the surrogate loss instead of the zero-one loss in analyses and generalizes the classical results from binary cases to multiclass ones.

Few-Shot Learning Image Classification +1

InstructPix2NeRF: Instructed 3D Portrait Editing from a Single Image

1 code implementation6 Nov 2023 Jianhui Li, Shilong Liu, Zidong Liu, Yikai Wang, Kaiwen Zheng, Jinghui Xu, Jianmin Li, Jun Zhu

With the success of Neural Radiance Field (NeRF) in 3D-aware portrait editing, a variety of works have achieved promising results regarding both quality and 3D consistency.

Semi-crowdsourced Clustering with Deep Generative Models

1 code implementation NeurIPS 2018 Yucen Luo, Tian Tian, Jiaxin Shi, Jun Zhu, Bo Zhang

We propose a new approach that includes a deep generative model (DGM) to characterize low-level features of the data, and a statistical relational model for noisy pairwise annotations on its subset.

Clustering Variational Inference

Understanding MCMC Dynamics as Flows on the Wasserstein Space

1 code implementation1 Feb 2019 Chang Liu, Jingwei Zhuo, Jun Zhu

It is known that the Langevin dynamics used in MCMC is the gradient flow of the KL divergence on the Wasserstein space, which helps convergence analysis and inspires recent particle-based variational inference methods (ParVIs).

Novel Concepts Variational Inference

Efficient Learning of Generative Models via Finite-Difference Score Matching

1 code implementation NeurIPS 2020 Tianyu Pang, Kun Xu, Chongxuan Li, Yang song, Stefano Ermon, Jun Zhu

Several machine learning applications involve the optimization of higher-order derivatives (e. g., gradients of gradients) during training, which can be expensive in respect to memory and computation even with automatic differentiation.

Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks

1 code implementation8 Jun 2017 Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Luping Shi

By simultaneously considering the layer-by-layer spatial domain (SD) and the timing-dependent temporal domain (TD) in the training phase, as well as an approximated derivative for the spike activity, we propose a spatio-temporal backpropagation (STBP) training framework without using any complicated technology.

object-detection Object Detection +1

Max-Margin Deep Generative Models for (Semi-)Supervised Learning

1 code implementation22 Nov 2016 Chongxuan Li, Jun Zhu, Bo Zhang

Deep generative models (DGMs) are effective on learning multilayered representations of complex data and performing inference of input data by exploring the generative ability.

Missing Labels

The Search for Sparse, Robust Neural Networks

1 code implementation5 Dec 2019 Justin Cosentino, Federico Zaiter, Dan Pei, Jun Zhu

Recent work on deep neural network pruning has shown there exist sparse subnetworks that achieve equal or improved accuracy, training time, and loss using fewer network parameters when compared to their dense counterparts.

Network Pruning

Bi-level Score Matching for Learning Energy-based Latent Variable Models

1 code implementation NeurIPS 2020 Fan Bao, Chongxuan Li, Kun Xu, Hang Su, Jun Zhu, Bo Zhang

This paper presents a bi-level score matching (BiSM) method to learn EBLVMs with general structures by reformulating SM as a bi-level optimization problem.

Rolling Shutter Correction Stochastic Optimization

Neural Eigenfunctions Are Structured Representation Learners

1 code implementation23 Oct 2022 Zhijie Deng, Jiaxin Shi, Hao Zhang, Peng Cui, Cewu Lu, Jun Zhu

Unlike prior spectral methods such as Laplacian Eigenmap that operate in a nonparametric manner, Neural Eigenmap leverages NeuralEF to parametrically model eigenfunctions using a neural network.

Contrastive Learning Data Augmentation +7

A Unified Hard-Constraint Framework for Solving Geometrically Complex PDEs

1 code implementation6 Oct 2022 Songming Liu, Zhongkai Hao, Chengyang Ying, Hang Su, Jun Zhu, Ze Cheng

We present a unified hard-constraint framework for solving geometrically complex PDEs with neural networks, where the most commonly used Dirichlet, Neumann, and Robin boundary conditions (BCs) are considered.

Understanding and Stabilizing GANs' Training Dynamics with Control Theory

1 code implementation29 Sep 2019 Kun Xu, Chongxuan Li, Jun Zhu, Bo Zhang

There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods.

Ranked #37 on Image Generation on CIFAR-10 (Inception score metric)

Image Generation L2 Regularization

Censored Quantile Regression Neural Networks for Distribution-Free Survival Analysis

1 code implementation26 May 2022 Tim Pearce, Jong-Hyeon Jeong, Yichen Jia, Jun Zhu

To offer theoretical insight into our algorithm, we show firstly that it can be interpreted as a form of expectation-maximisation, and secondly that it exhibits a desirable `self-correcting' property.

regression Survival Analysis

Further Analysis of Outlier Detection with Deep Generative Models

1 code implementation NeurIPS 2020 Ziyu Wang, Bin Dai, David Wipf, Jun Zhu

The recent, counter-intuitive discovery that deep generative models (DGMs) can frequently assign a higher likelihood to outliers has implications for both outlier detection applications as well as our overall understanding of generative modeling.

Outlier Detection

Cluster Attack: Query-based Adversarial Attacks on Graphs with Graph-Dependent Priors

1 code implementation ICML Workshop AML 2021 Zhengyi Wang, Zhongkai Hao, Ziqiao Wang, Hang Su, Jun Zhu

In this work, we propose Cluster Attack -- a Graph Injection Attack (GIA) on node classification, which injects fake nodes into the original graph to degenerate the performance of graph neural networks (GNNs) on certain victim nodes while affecting the other nodes as little as possible.

Adversarial Attack Clustering +3

Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Risk

1 code implementation9 Jun 2022 Chengyang Ying, Xinning Zhou, Hang Su, Dong Yan, Ning Chen, Jun Zhu

Though deep reinforcement learning (DRL) has obtained substantial success, it may encounter catastrophic failures due to the intrinsic uncertainty of both transition and observation.

Continuous Control reinforcement-learning +2

Noise Contrastive Alignment of Language Models with Explicit Rewards

1 code implementation8 Feb 2024 Huayu Chen, Guande He, Hang Su, Jun Zhu

Existing alignment methods, such as Direct Preference Optimization (DPO), are mainly tailored for pairwise preference data where rewards are implicitly defined rather than explicitly given.

Language Modelling

DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training

1 code implementation6 Mar 2024 Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandkumar, Jian Song, Jun Zhu

Pre-training has been investigated to improve the efficiency and performance of training neural operators in data-scarce settings.

Denoising

Riemannian Stein Variational Gradient Descent for Bayesian Inference

1 code implementation30 Nov 2017 Chang Liu, Jun Zhu

The benefits are two-folds: (i) for inference tasks in Euclidean spaces, RSVGD has the advantage over SVGD of utilizing information geometry, and (ii) for inference tasks on Riemann manifolds, RSVGD brings the unique advantages of SVGD to the Riemannian world.

Bayesian Inference

Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models

1 code implementation NeurIPS Workshop ICBINB 2020 Fan Bao, Kun Xu, Chongxuan Li, Lanqing Hong, Jun Zhu, Bo Zhang

The learning and evaluation of energy-based latent variable models (EBLVMs) without any structural assumptions are highly challenging, because the true posteriors and the partition functions in such models are generally intractable.

Multi-label classification: do Hamming loss and subset accuracy really conflict with each other?

1 code implementation NeurIPS 2020 Guoqiang Wu, Jun Zhu

On the other hand, when directly optimizing SA with its surrogate loss, it has learning guarantees that depend on $O(\sqrt{c})$ for both HL and SA measures.

General Classification Multi-Label Classification

DualTeacher: Bridging Coexistence of Unlabelled Classes for Semi-supervised Incremental Object Detection

1 code implementation13 Dec 2023 Ziqi Yuan, Liyuan Wang, Wenbo Ding, Xingxing Zhang, Jiachen Zhong, Jianyong Ai, Jianmin Li, Jun Zhu

A commonly-used strategy for supervised IOD is to encourage the current model (as a student) to mimic the behavior of the old model (as a teacher), but it generally fails in SSIOD because a dominant number of object instances from old and new classes are coexisting and unlabelled, with the teacher only recognizing a fraction of them.

Object object-detection +1

Learning to Generate with Memory

1 code implementation24 Feb 2016 Chongxuan Li, Jun Zhu, Bo Zhang

Memory units have been widely used to enrich the capabilities of deep networks on capturing long-term dependencies in reasoning and prediction tasks, but little investigation exists on deep generative models (DGMs) which are good at inferring high-level invariant representations from unlabeled data.

Density Estimation Image Generation +2

Bayesian Matrix Completion via Adaptive Relaxed Spectral Regularization

1 code implementation3 Dec 2015 Yang Song, Jun Zhu

Bayesian matrix completion has been studied based on a low-rank matrix factorization formulation with promising results.

Bayesian Inference Collaborative Filtering +1

Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structure

1 code implementation22 Nov 2019 Zhijie Deng, Yucen Luo, Jun Zhu, Bo Zhang

Bayesian neural networks (BNNs) augment deep networks with uncertainty quantification by Bayesian treatment of the network weights.

Bayesian Inference Neural Architecture Search +2

Rethinking Natural Adversarial Examples for Classification Models

1 code implementation23 Feb 2021 Xiao Li, Jianmin Li, Ting Dai, Jie Shi, Jun Zhu, Xiaolin Hu

A detection model based on the classification model EfficientNet-B7 achieved a top-1 accuracy of 53. 95%, surpassing previous state-of-the-art classification models trained on ImageNet, suggesting that accurate localization information can significantly boost the performance of classification models on ImageNet-A.

Classification General Classification +2

Deep Bayesian Structure Networks

1 code implementation25 Sep 2019 Zhijie Deng, Yucen Luo, Jun Zhu, Bo Zhang

Bayesian neural networks (BNNs) introduce uncertainty estimation to deep networks by performing Bayesian inference on network weights.

Bayesian Inference Neural Architecture Search +1

Towards Job-Transition-Tag Graph for a Better Job Title Representation Learning

1 code implementation Findings (NAACL) 2022 Jun Zhu, Céline Hudelot

Works on learning job title representation are mainly based on \textit{Job-Transition Graph}, built from the working history of talents.

Representation Learning TAG

Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs

1 code implementation6 May 2023 Kaiwen Zheng, Cheng Lu, Jianfei Chen, Jun Zhu

The probability flow ordinary differential equation (ODE) of diffusion models (i. e., diffusion ODEs) is a particular case of continuous normalizing flows (CNFs), which enables deterministic inference and exact likelihood evaluation.

 Ranked #1 on Image Generation on ImageNet 32x32 (bpd metric)

Image Generation

The YouTube-8M Kaggle Competition: Challenges and Methods

1 code implementation28 Jun 2017 Haosheng Zou, Kun Xu, Jialian Li, Jun Zhu

We took part in the YouTube-8M Video Understanding Challenge hosted on Kaggle, and achieved the 10th place within less than one month's time.

General Classification Video Classification +1

A Wasserstein Minimum Velocity Approach to Learning Unnormalized Models

1 code implementation pproximateinference AABI Symposium 2019 Ziyu Wang, Shuyu Cheng, Yueru Li, Jun Zhu, Bo Zhang

Score matching provides an effective approach to learning flexible unnormalized models, but its scalability is limited by the need to evaluate a second-order derivative.

Triple Memory Networks: a Brain-Inspired Method for Continual Learning

1 code implementation6 Mar 2020 Liyuan Wang, Bo Lei, Qian Li, Hang Su, Jun Zhu, Yi Zhong

Continual acquisition of novel experience without interfering previously learned knowledge, i. e. continual learning, is critical for artificial neural networks, but limited by catastrophic forgetting.

Attribute Class Incremental Learning +2

On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithms

1 code implementation NeurIPS 2021 Shuyu Cheng, Guoqiang Wu, Jun Zhu

Finally, our theoretical results are confirmed by experiments on several numerical benchmarks as well as adversarial attacks.

Efficient Backpropagation with Variance-Controlled Adaptive Sampling

1 code implementation27 Feb 2024 Ziteng Wang, Jianfei Chen, Jun Zhu

On all the tasks, VCAS can preserve the original training loss trajectory and validation accuracy with an up to 73. 87% FLOPs reduction of BP and 49. 58% FLOPs reduction of the whole training process.

Stability and Generalization of Bilevel Programming in Hyperparameter Optimization

1 code implementation NeurIPS 2021 Fan Bao, Guoqiang Wu, Chongxuan Li, Jun Zhu, Bo Zhang

Our results can explain some mysterious behaviours of the bilevel programming in practice, for instance, overfitting to the validation set.

Hyperparameter Optimization

DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimization

1 code implementation12 Jul 2022 Wentse Chen, Shiyu Huang, Yuan Chiang, Tim Pearce, Wei-Wei Tu, Ting Chen, Jun Zhu

We propose Diversity-Guided Policy Optimization (DGPO), an on-policy algorithm that discovers multiple strategies for solving a given task.

reinforcement-learning Reinforcement Learning (RL)

On the Reuse Bias in Off-Policy Reinforcement Learning

1 code implementation15 Sep 2022 Chengyang Ying, Zhongkai Hao, Xinning Zhou, Hang Su, Dong Yan, Jun Zhu

In this paper, we reveal that the instability is also related to a new notion of Reuse Bias of IS -- the bias in off-policy evaluation caused by the reuse of the replay buffer for evaluation and optimization.

Continuous Control Off-policy evaluation +1

Boundary-semantic collaborative guidance network with dual-stream feedback mechanism for salient object detection in optical remote sensing imagery

1 code implementation6 Mar 2023 Dejun Feng, Hongyu Chen, Suning Liu, Ziyang Liao, Xingyu Shen, Yakun Xie, Jun Zhu

Finally, to obtain more complete saliency maps, we consider the uniqueness of the last layer of the decoder for the first time and propose the adaptive feedback refinement (AFR) module, which further refines feature representation and eliminates differences between features through a unique feedback mechanism.

object-detection Object Detection +1

Scaling up Dynamic Topic Models

1 code implementation19 Feb 2016 Arnab Bhadury, Jianfei Chen, Jun Zhu, Shixia Liu

Dynamic topic models (DTMs) are very effective in discovering topics and capturing their evolution trends in time series data.

Time Series Time Series Analysis +1

Understanding and Exploring the Network with Stochastic Architectures

1 code implementation NeurIPS 2020 Zhijie Deng, Yinpeng Dong, Shifeng Zhang, Jun Zhu

In this work, we decouple the training of a network with stochastic architectures (NSA) from NAS and provide a first systematical investigation on it as a stand-alone problem.

Neural Architecture Search

Quasi-Bayesian Dual Instrumental Variable Regression

1 code implementation NeurIPS 2021 Ziyu Wang, Yuhao Zhou, Tongzheng Ren, Jun Zhu

Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking.

Bayesian Inference regression +1

Learning to Write Stylized Chinese Characters by Reading a Handful of Examples

no code implementations6 Dec 2017 Danyang Sun, Tongzheng Ren, Chongxun Li, Hang Su, Jun Zhu

Automatically writing stylized Chinese characters is an attractive yet challenging task due to its wide applicabilities.

Message Passing Stein Variational Gradient Descent

no code implementations ICML 2018 Jingwei Zhuo, Chang Liu, Jiaxin Shi, Jun Zhu, Ning Chen, Bo Zhang

Stein variational gradient descent (SVGD) is a recently proposed particle-based Bayesian inference method, which has attracted a lot of interest due to its remarkable approximation ability and particle efficiency compared to traditional variational inference and Markov Chain Monte Carlo methods.

Bayesian Inference Variational Inference

Towards Training Probabilistic Topic Models on Neuromorphic Multi-chip Systems

no code implementations10 Apr 2018 Zihao Xiao, Jianfei Chen, Jun Zhu

We also propose an extension to train pLSI and a method to prune the network to obey the limited fan-in of some NMSs.

Stochastic Optimization Topic Models

Stochastic Gradient Hamiltonian Monte Carlo with Variance Reduction for Bayesian Inference

no code implementations29 Mar 2018 Zhize Li, Tianyi Zhang, Shuyu Cheng, Jun Zhu, Jian Li

In this paper, we apply the variance reduction tricks on Hamiltonian Monte Carlo and achieve better theoretical convergence results compared with the variance-reduced Langevin dynamics.

Bayesian Inference

Kernel Implicit Variational Inference

no code implementations ICLR 2018 Jiaxin Shi, Shengyang Sun, Jun Zhu

Recent progress in variational inference has paid much attention to the flexibility of variational posteriors.

General Classification regression +1

Understanding Human Behaviors in Crowds by Imitating the Decision-Making Process

no code implementations25 Jan 2018 Haosheng Zou, Hang Su, Shihong Song, Jun Zhu

Crowd behavior understanding is crucial yet challenging across a wide range of applications, since crowd behavior is inherently determined by a sequential decision-making process based on various factors, such as the pedestrians' own destinations, interaction with nearby pedestrians and anticipation of upcoming events.

Collision Avoidance Imitation Learning

Learning Random Fourier Features by Hybrid Constrained Optimization

no code implementations7 Dec 2017 Jianqiao Wangni, Jingwei Zhuo, Jun Zhu

Since the algorithm consumes a major computation cost in the testing phase, we propose a novel teacher-learner framework of learning computation-efficient kernel embeddings from specific data.

Diversity-Promoting Bayesian Learning of Latent Variable Models

no code implementations23 Nov 2017 Pengtao Xie, Jun Zhu, Eric P. Xing

We also extend our approach to "diversify" Bayesian nonparametric models where the number of components is infinite.

Variational Inference

Visual Concepts and Compositional Voting

no code implementations13 Nov 2017 Jianyu Wang, Zhishuai Zhang, Cihang Xie, Yuyin Zhou, Vittal Premachandran, Jun Zhu, Lingxi Xie, Alan Yuille

We use clustering algorithms to study the population activities of the features and extract a set of visual concepts which we show are visually tight and correspond to semantic parts of vehicles.

Clustering Semantic Part Detection

Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples

no code implementations18 Aug 2017 Yinpeng Dong, Hang Su, Jun Zhu, Fan Bao

We find that: (1) the neurons in DNNs do not truly detect semantic objects/parts, but respond to objects/parts only as recurrent discriminative patches; (2) deep visual representations are not robust distributed codes of visual concepts because the representations of adversarial images are largely not consistent with those of real images, although they have similar visual appearance, both of which are different from previous findings.

Improving Interpretability of Deep Neural Networks with Semantic Information

no code implementations CVPR 2017 Yinpeng Dong, Hang Su, Jun Zhu, Bo Zhang

Interpretability of deep neural networks (DNNs) is essential since it enables users to understand the overall strengths and weaknesses of the models, conveys an understanding of how the models will behave in the future, and how to diagnose and correct potential problems.

Action Recognition Temporal Action Localization +1

Big Learning with Bayesian Methods

no code implementations24 Nov 2014 Jun Zhu, Jianfei Chen, Wen-Bo Hu, Bo Zhang

Explosive growth in data and availability of cheap computing resources have sparked increasing interest in Big learning, an emerging subfield that studies scalable machine learning algorithms, systems, and applications with Big Data.

Bayesian Inference BIG-bench Machine Learning +1

Scalable Inference for Nested Chinese Restaurant Process Topic Models

no code implementations23 Feb 2017 Jianfei Chen, Jun Zhu, Jie Lu, Shixia Liu

Finally, we propose an efficient distributed implementation of PCGS through vectorization, pre-processing, and a careful design of the concurrent data structures and communication strategy.

Topic Models Variational Inference

Towards Better Analysis of Machine Learning Models: A Visual Analytics Perspective

no code implementations4 Feb 2017 Shixia Liu, Xiting Wang, Mengchen Liu, Jun Zhu

Interactive model analysis, the process of understanding, diagnosing, and refining a machine learning model with the help of interactive visualization, is very important for users to efficiently solve real-world artificial intelligence and data mining problems.

BIG-bench Machine Learning

A Communication-Efficient Parallel Method for Group-Lasso

no code implementations7 Dec 2016 Binghong Chen, Jun Zhu

Group-Lasso (gLasso) identifies important explanatory factors in predicting the response variable by considering the grouping structure over input variables.

regression

SeDMiD for Confusion Detection: Uncovering Mind State from Time Series Brain Wave Data

no code implementations29 Nov 2016 Jingkang Yang, Haohan Wang, Jun Zhu, Eric P. Xing

In this paper, we propose an extension of State Space Model to work with different sources of information together with its learning and inference algorithms.

Time Series Time Series Analysis

Kernel Bayesian Inference with Posterior Regularization

no code implementations NeurIPS 2016 Yang Song, Jun Zhu, Yong Ren

We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution.

Bayesian Inference regression

Fast Sampling for Bayesian Max-Margin Models

no code implementations27 Apr 2015 Wenbo Hu, Jun Zhu, Bo Zhang

Bayesian max-margin models have shown superiority in various practical applications, such as text categorization, collaborative prediction, social network link prediction and crowdsourcing, and they conjoin the flexibility of Bayesian modeling and predictive strengths of max-margin learning.

Link Prediction Text Categorization

SaberLDA: Sparsity-Aware Learning of Topic Models on GPUs

no code implementations8 Oct 2016 Kaiwei Li, Jianfei Chen, WenGuang Chen, Jun Zhu

Latent Dirichlet Allocation (LDA) is a popular tool for analyzing discrete count data such as text and images.

Topic Models

Conditional Generative Moment-Matching Networks

no code implementations NeurIPS 2016 Yong Ren, Jialian Li, Yucen Luo, Jun Zhu

Maximum mean discrepancy (MMD) has been successfully applied to learn deep generative models for characterizing a joint distribution of variables via kernel mean embedding.

PSDVec: a Toolbox for Incremental and Scalable Word Embedding

no code implementations10 Jun 2016 Shaohua Li, Jun Zhu, Chunyan Miao

PSDVec is a Python/Perl toolbox that learns word embeddings, i. e. the mapping of words in a natural language to continuous vectors which encode the semantic/syntactic regularities between the words.

Word Embeddings Word Similarity

Towards Better Analysis of Deep Convolutional Neural Networks

no code implementations24 Apr 2016 Mengchen Liu, Jiaxin Shi, Zhen Li, Chongxuan Li, Jun Zhu, Shixia Liu

Deep convolutional neural networks (CNNs) have achieved breakthrough performance in many pattern recognition tasks such as image classification.

Image Classification

Learning Deep Generative Models with Doubly Stochastic MCMC

no code implementations15 Jun 2015 Chao Du, Jun Zhu, Bo Zhang

We present doubly stochastic gradient MCMC, a simple and generic method for (approximate) Bayesian inference of deep generative models (DGMs) in a collapsed continuous parameter space.

Bayesian Inference Density Estimation +1

WarpLDA: a Cache Efficient O(1) Algorithm for Latent Dirichlet Allocation

no code implementations29 Oct 2015 Jianfei Chen, Kaiwei Li, Jun Zhu, WenGuang Chen

We then develop WarpLDA, an LDA sampler which achieves both the best O(1) time complexity per token and the best O(K) scope of random access.

Max-Margin Nonparametric Latent Feature Models for Link Prediction

no code implementations24 Feb 2016 Jun Zhu, Jiaming Song, Bei Chen

Our approach attempts to unite the ideas of max-margin learning and Bayesian nonparametrics to discover discriminative latent features for link prediction.

Link Prediction Variational Inference

Spectral Learning for Supervised Topic Models

no code implementations19 Feb 2016 Yong Ren, Yining Wang, Jun Zhu

Spectral methods have been applied to learn unsupervised topic models, such as latent Dirichlet allocation (LDA), with provable guarantees.

regression Topic Models

DeePM: A Deep Part-Based Model for Object Detection and Semantic Part Localization

no code implementations23 Nov 2015 Jun Zhu, Xianjie Chen, Alan L. Yuille

In this paper, we propose a deep part-based model (DeePM) for symbiotic object detection and semantic part localization.

Object object-detection +1

Streaming Gibbs Sampling for LDA Model

no code implementations6 Jan 2016 Yang Gao, Jianfei Chen, Jun Zhu

Streaming variational Bayes (SVB) is successful in learning LDA models in an online manner.

Fast Parallel SVM using Data Augmentation

no code implementations24 Dec 2015 Hugh Perkins, Minjie Xu, Jun Zhu, Bo Zhang

As one of the most popular classifiers, linear SVMs still have challenges in dealing with very large-scale problems, even though linear or sub-linear algorithms have been developed recently on single machines.

Bayesian Inference Data Augmentation

Discriminative Nonparametric Latent Feature Relational Models with Data Augmentation

no code implementations7 Dec 2015 Bei Chen, Ning Chen, Jun Zhu, Jiaming Song, Bo Zhang

We present a discriminative nonparametric latent feature relational model (LFRM) for link prediction to automatically infer the dimensionality of latent features.

Bayesian Inference Data Augmentation +1

Jointly Modeling Topics and Intents with Global Order Structure

no code implementations7 Dec 2015 Bei Chen, Jun Zhu, Nan Yang, Tian Tian, Ming Zhou, Bo Zhang

Modeling document structure is of great importance for discourse analysis and related applications.

Building Memory with Concept Learning Capabilities from Large-scale Knowledge Base

no code implementations3 Dec 2015 Jiaxin Shi, Jun Zhu

We present a new perspective on neural knowledge base (KB) embeddings, from which we build a framework that can model symbolic knowledge in the KB together with its learning process.

Pose-Guided Human Parsing with Deep Learned Features

no code implementations17 Aug 2015 Fangting Xia, Jun Zhu, Peng Wang, Alan Yuille

Parsing human body into semantic regions is crucial to human-centric analysis.

Human Parsing

Dropout Training for SVMs with Data Augmentation

no code implementations10 Aug 2015 Ning Chen, Jun Zhu, Jianfei Chen, Ting Chen

Empirical results on several real datasets demonstrate the effectiveness of dropout training on significantly boosting the classification accuracy of both linear and nonlinear SVMs.

Data Augmentation Representation Learning

Bounded-Distortion Metric Learning

no code implementations10 May 2015 Renjie Liao, Jianping Shi, Ziyang Ma, Jun Zhu, Jiaya Jia

Metric learning aims to embed one metric space into another to benefit tasks like classification and clustering.

Clustering General Classification +1

Contrastive Feature Induction for Efficient Structure Learning of Conditional Random Fields

no code implementations28 Jun 2014 Ni Lao, Jun Zhu

We prove that the gradient of candidate features can be represented solely as a function of signals and errors, and that CFI is an efficient approximation of gradient-based evaluation methods.

feature selection Relational Reasoning

Dropout Training for Support Vector Machines

no code implementations16 Apr 2014 Ning Chen, Jun Zhu, Jianfei Chen, Bo Zhang

To deal with the intractable expectation of the non-smooth hinge loss under corrupting distributions, we develop an iteratively re-weighted least square (IRLS) algorithm by exploring data augmentation techniques.

Data Augmentation

Bayesian Inference with Posterior Regularization and applications to Infinite Latent SVMs

no code implementations5 Oct 2012 Jun Zhu, Ning Chen, Eric P. Xing

When the regularization is induced from a linear operator on the posterior distributions, such as the expectation operator, we present a general convex-analysis theorem to characterize the solution of RegBayes.

Bayesian Inference Multi-Task Learning

Online Bayesian Passive-Aggressive Learning

no code implementations12 Dec 2013 Tianlin Shi, Jun Zhu

Online Passive-Aggressive (PA) learning is an effective framework for performing max-margin online learning.

Bayesian Inference Descriptive +1

Gibbs Max-margin Topic Models with Data Augmentation

no code implementations10 Oct 2013 Jun Zhu, Ning Chen, Hugh Perkins, Bo Zhang

Gibbs max-margin supervised topic models minimize an expected margin loss, which is an upper bound of the existing margin loss derived from an expected prediction rule.

Data Augmentation General Classification +3

Improved Bayesian Logistic Supervised Topic Models with Data Augmentation

no code implementations ACL 2013 Jun Zhu, Xun Zheng, Bo Zhang

Supervised topic models with a logistic likelihood have two issues that potentially limit their practical use: 1) response variables are usually over-weighted by document word counts; and 2) existing variational inference methods make strict mean-field assumptions.

Bayesian Inference Data Augmentation +2

Deep Structured Generative Models

no code implementations10 Jul 2018 Kun Xu, Haoyu Liang, Jun Zhu, Hang Su, Bo Zhang

Deep generative models have shown promising results in generating realistic images, but it is still non-trivial to generate images with complicated structures.

Direct Training for Spiking Neural Networks: Faster, Larger, Better

no code implementations16 Sep 2018 Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Luping Shi

Spiking neural networks (SNNs) that enables energy efficient implementation on emerging neuromorphic hardware are gaining more attention.

Cannot find the paper you are looking for? You can Submit a new open access paper.