Search Results for author: Jun Zhu

Found 323 papers, 158 papers with code

Understanding and Stabilizing GANs' Training Dynamics Using Control Theory

no code implementations ICML 2020 Kun Xu, Chongxuan Li, Jun Zhu, Bo Zhang

There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods.

Variance Reduction and Quasi-Newton for Particle-Based Variational Inference

no code implementations ICML 2020 Michael Zhu, Chang Liu, Jun Zhu

Particle-based Variational Inference methods (ParVIs), like Stein Variational Gradient Descent, are nonparametric variational inference methods that optimize a set of particles to best approximate a target distribution.

Bayesian Inference Riemannian optimization +1

Defense Against Adversarial Attacks via Controlling Gradient Leaking on Embedded Manifolds

no code implementations ECCV 2020 Yueru Li, Shuyu Cheng, Hang Su, Jun Zhu

Based on our investigation, we further present a new robust learning algorithm which encourages a larger gradient component in the tangent space of data manifold, suppressing the gradient leaking phenomenon consequently.

CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model

no code implementations8 Mar 2024 Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, Jun Zhu

In this work, we present the Convolutional Reconstruction Model (CRM), a high-fidelity feed-forward single image-to-3D generative model.

Image to 3D

DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training

1 code implementation6 Mar 2024 Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandkumar, Jian Song, Jun Zhu

Pre-training has been investigated to improve the efficiency and performance of training neural operators in data-scarce settings.

Denoising

Efficient Backpropagation with Variance-Controlled Adaptive Sampling

1 code implementation27 Feb 2024 Ziteng Wang, Jianfei Chen, Jun Zhu

On all the tasks, VCAS can preserve the original training loss trajectory and validation accuracy with an up to 73. 87% FLOPs reduction of BP and 49. 58% FLOPs reduction of the whole training process.

CodeS: Towards Building Open-source Language Models for Text-to-SQL

1 code implementation26 Feb 2024 Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang, Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, Hong Chen

To address the limitations, we introduce CodeS, a series of pre-trained language models with parameters ranging from 1B to 15B, specifically designed for the text-to-SQL task.

Data Augmentation Domain Adaptation +2

BSPA: Exploring Black-box Stealthy Prompt Attacks against Image Generators

no code implementations23 Feb 2024 Yu Tian, Xiao Yang, Yinpeng Dong, Heming Yang, Hang Su, Jun Zhu

It allows users to design specific prompts to generate realistic images through some black-box APIs.

Noise Contrastive Alignment of Language Models with Explicit Rewards

1 code implementation8 Feb 2024 Huayu Chen, Guande He, Hang Su, Jun Zhu

Existing alignment methods, such as Direct Preference Optimization (DPO), are mainly tailored for pairwise preference data where rewards are implicitly defined rather than explicitly given.

Language Modelling

Towards Efficient and Exact Optimization of Language Model Alignment

1 code implementation1 Feb 2024 Haozhe Ji, Cheng Lu, Yilin Niu, Pei Ke, Hongning Wang, Jun Zhu, Jie Tang, Minlie Huang

We prove that EXO is guaranteed to optimize in the same direction as the RL algorithms asymptotically for arbitary parametrization of the policy, while enables efficient optimization by circumventing the complexities associated with RL algorithms.

Language Modelling Reinforcement Learning (RL)

Preconditioning for Physics-Informed Neural Networks

no code implementations1 Feb 2024 Songming Liu, Chang Su, Jiachen Yao, Zhongkai Hao, Hang Su, Youjia Wu, Jun Zhu

Physics-informed neural networks (PINNs) have shown promise in solving various partial differential equations (PDEs).

DualTeacher: Bridging Coexistence of Unlabelled Classes for Semi-supervised Incremental Object Detection

1 code implementation13 Dec 2023 Ziqi Yuan, Liyuan Wang, Wenbo Ding, Xingxing Zhang, Jiachen Zhong, Jianyong Ai, Jianmin Li, Jun Zhu

A commonly-used strategy for supervised IOD is to encourage the current model (as a student) to mimic the behavior of the old model (as a teacher), but it generally fails in SSIOD because a dominant number of object instances from old and new classes are coexisting and unlabelled, with the teacher only recognizing a fraction of them.

Object object-detection +1

Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesis

no code implementations6 Dec 2023 Zehua Chen, Guande He, Kaiwen Zheng, Xu Tan, Jun Zhu

Specifically, we leverage the latent representation obtained from text input as our prior, and build a fully tractable Schrodinger bridge between it and the ground-truth mel-spectrogram, leading to a data-to-data process.

Speech Synthesis Text-To-Speech Synthesis

Spatiotemporal Transformer for Imputing Sparse Data: A Deep Learning Approach

no code implementations1 Dec 2023 Kehui Yao, Jingyi Huang, Jun Zhu

Effective management of environmental resources and agricultural sustainability heavily depends on accurate soil moisture data.

Imputation Management

InstructPix2NeRF: Instructed 3D Portrait Editing from a Single Image

1 code implementation6 Nov 2023 Jianhui Li, Shilong Liu, Zidong Liu, Yikai Wang, Kaiwen Zheng, Jinghui Xu, Jianmin Li, Jun Zhu

With the success of Neural Radiance Field (NeRF) in 3D-aware portrait editing, a variety of works have achieved promising results regarding both quality and 3D consistency.

Towards a General Framework for Continual Learning with Pre-training

1 code implementation21 Oct 2023 Liyuan Wang, Jingyi Xie, Xingxing Zhang, Hang Su, Jun Zhu

In this work, we present a general framework for continual learning of sequentially arrived tasks with the use of pre-training, which has emerged as a promising direction for artificial intelligence systems to accommodate real-world dynamics.

Continual Learning

DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics

1 code implementation NeurIPS 2023 Kaiwen Zheng, Cheng Lu, Jianfei Chen, Jun Zhu

In this work, we propose a novel formulation towards the optimal parameterization during sampling that minimizes the first-order discretization error of the ODE solution.

Image Generation

Investigating Uncertainty Calibration of Aligned Language Models under the Multiple-Choice Setting

no code implementations18 Oct 2023 Guande He, Peng Cui, Jianfei Chen, WenBo Hu, Jun Zhu

Despite the significant progress made in practical applications of aligned language models (LMs), they tend to be overconfident in output answers compared to the corresponding pre-trained LMs.

Multiple-choice

Score Regularized Policy Optimization through Diffusion Behavior

1 code implementation11 Oct 2023 Huayu Chen, Cheng Lu, Zhengyi Wang, Hang Su, Jun Zhu

Recent developments in offline reinforcement learning have uncovered the immense potential of diffusion modeling, which excels at representing heterogeneous behavior policies.

D4RL

Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality

1 code implementation NeurIPS 2023 Liyuan Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, Jun Zhu

Following these empirical and theoretical insights, we propose Hierarchical Decomposition (HiDe-)Prompt, an innovative approach that explicitly optimizes the hierarchical components with an ensemble of task-specific prompts and statistics of both uninstructed and instructed representations, further with the coordination of a contrastive regularization strategy.

Continual Learning

How Robust is Google's Bard to Adversarial Image Attacks?

1 code implementation21 Sep 2023 Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu

By attacking white-box surrogate vision encoders or MLLMs, the generated adversarial examples can mislead Bard to output wrong image descriptions with a 22% success rate based solely on the transferability.

Adversarial Robustness Chatbot +1

Incorporating Neuro-Inspired Adaptability for Continual Learning in Artificial Intelligence

1 code implementation29 Aug 2023 Liyuan Wang, Xingxing Zhang, Qian Li, Mingtian Zhang, Hang Su, Jun Zhu, Yi Zhong

Continual learning aims to empower artificial intelligence (AI) with strong adaptability to the real world.

Continual Learning

Heterogeneous Multi-Task Gaussian Cox Processes

1 code implementation29 Aug 2023 Feng Zhou, Quyu Kong, Zhijie Deng, Fengxiang He, Peng Cui, Jun Zhu

This paper presents a novel extension of multi-task Gaussian Cox processes for modeling multiple heterogeneous correlated tasks jointly, e. g., classification and regression, via multi-output Gaussian processes (MOGP).

Bayesian Inference Data Augmentation +2

6G Network Business Support System

no code implementations19 Jul 2023 Ye Ouyang, Yaqin Zhang, Peng Wang, Yunxin Liu, Wen Qiao, Jun Zhu, Yang Liu, Feng Zhang, Shuling Wang, Xidong Wang

6G is the next-generation intelligent and integrated digital information infrastructure, characterized by ubiquitous interconnection, native intelligence, multi-dimensional perception, global coverage, green and low-carbon, native network security, etc.

Multi-task multi-station earthquake monitoring: An all-in-one seismic Phase picking, Location, and Association Network (PLAN)

no code implementations24 Jun 2023 Xu Si, Xinming Wu, Zefeng Li, Shenghou Wang, Jun Zhu

Overall, our study provides for the first time a prototype self-consistent all-in-one system of simultaneous seismic phase picking, association, and location, which has the potential for next-generation autonomous earthquake monitoring.

Training Transformers with 4-bit Integers

1 code implementation NeurIPS 2023 Haocheng Xi, Changhao Li, Jianfei Chen, Jun Zhu

To achieve this, we carefully analyze the specific structures of activation and gradients in transformers to propose dedicated quantizers for them.

Image Classification Machine Translation +1

Stabilizing GANs' Training with Brownian Motion Controller

no code implementations18 Jun 2023 Tianjiao Luo, Ziyu Zhu, Jianfei Chen, Jun Zhu

We theoretically prove that the training process of DiracGANs-BMC is globally exponential stable and derive bounds on the rate of convergence.

PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs

1 code implementation15 Jun 2023 Zhongkai Hao, Jiachen Yao, Chang Su, Hang Su, Ziao Wang, Fanzhi Lu, Zeyu Xia, Yichi Zhang, Songming Liu, Lu Lu, Jun Zhu

In addition to providing a standardized means of assessing performance, PINNacle also offers an in-depth analysis to guide future research, particularly in areas such as domain decomposition methods and loss reweighting for handling multi-scale problems and complex geometry.

Benchmarking

MultiAdam: Parameter-wise Scale-invariant Optimizer for Multiscale Training of Physics-informed Neural Networks

no code implementations5 Jun 2023 Jiachen Yao, Chang Su, Zhongkai Hao, Songming Liu, Hang Su, Jun Zhu

Physics-informed Neural Networks (PINNs) have recently achieved remarkable progress in solving Partial Differential Equations (PDEs) in various fields by minimizing a weighted sum of PDE loss and boundary loss.

NUNO: A General Framework for Learning Parametric PDEs with Non-Uniform Data

1 code implementation30 May 2023 Songming Liu, Zhongkai Hao, Chengyang Ying, Hang Su, Ze Cheng, Jun Zhu

The neural operator has emerged as a powerful tool in learning mappings between function spaces in PDEs.

Operator learning

Preserving Pre-trained Features Helps Calibrate Fine-tuned Language Models

1 code implementation30 May 2023 Guande He, Jianfei Chen, Jun Zhu

In light of these observations, we evaluate the calibration of several methods that preserve pre-trained features and show that preserving pre-trained features can improve the calibration of fine-tuned language models.

Language Modelling Masked Language Modeling +1

Amplification trojan network: Attack deep neural networks by amplifying their inherent weakness

1 code implementation28 May 2023 Zhanhao Hu, Jun Zhu, Bo Zhang, Xiaolin Hu

Recent works found that deep neural networks (DNNs) can be fooled by adversarial examples, which are crafted by adding adversarial noise on clean inputs.

ControlVideo: Conditional Control for One-shot Text-driven Video Editing and Beyond

1 code implementation26 May 2023 Min Zhao, Rongzhen Wang, Fan Bao, Chongxuan Li, Jun Zhu

This paper presents \emph{ControlVideo} for text-driven video editing -- generating a video that aligns with a given text while preserving the structure of the source video.

Text-to-Video Editing Video Editing

ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation

2 code implementations NeurIPS 2023 Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, Jun Zhu

In comparison, VSD works well with various CFG weights as ancestral sampling from diffusion models and simultaneously improves the diversity and sample quality with a common CFG weight (i. e., $7. 5$).

Text to 3D

Robust Classification via a Single Diffusion Model

2 code implementations24 May 2023 Huanran Chen, Yinpeng Dong, Zhengyi Wang, Xiao Yang, Chengqi Duan, Hang Su, Jun Zhu

Since our method does not require training on particular adversarial attacks, we demonstrate that it is more generalizable to defend against multiple unseen threats.

Adversarial Defense Adversarial Robustness +2

Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs

1 code implementation6 May 2023 Kaiwen Zheng, Cheng Lu, Jianfei Chen, Jun Zhu

The probability flow ordinary differential equation (ODE) of diffusion models (i. e., diffusion ODEs) is a particular case of continuous normalizing flows (CNFs), which enables deterministic inference and exact likelihood evaluation.

 Ranked #1 on Image Generation on ImageNet 32x32 (bpd metric)

Image Generation

Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning

3 code implementations25 Apr 2023 Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, Jun Zhu

The main challenge for this setting is that the intermediate guidance during the diffusion sampling procedure, which is jointly defined by the sampling distribution and the energy function, is unknown and is hard to estimate.

D4RL Image Generation +1

Learning CLIP Guided Visual-Text Fusion Transformer for Video-based Pedestrian Attribute Recognition

1 code implementation20 Apr 2023 Jun Zhu, Jiandong Jin, Zihan Yang, Xiaohao Wu, Xiao Wang

The averaged visual tokens and text tokens are concatenated and fed into a fusion Transformer for multi-modal interactive learning.

Attribute Pedestrian Attribute Recognition +1

PREIM3D: 3D Consistent Precise Image Attribute Editing from a Single Image

1 code implementation CVPR 2023 Jianhui Li, Jianmin Li, Haoji Zhang, Shilong Liu, Zhengyi Wang, Zihao Xiao, Kaiwen Zheng, Jun Zhu

As for imprecise image editing, we attribute the problem to the gap between the latent space of real images and that of generated images.

Attribute

Detection Transformer with Stable Matching

1 code implementation ICCV 2023 Shilong Liu, Tianhe Ren, Jiayu Chen, Zhaoyang Zeng, Hao Zhang, Feng Li, Hongyang Li, Jun Huang, Hang Su, Jun Zhu, Lei Zhang

We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR.

Position

A Closer Look at Parameter-Efficient Tuning in Diffusion Models

1 code implementation31 Mar 2023 Chendong Xiang, Fan Bao, Chongxuan Li, Hang Su, Jun Zhu

Large-scale diffusion models like Stable Diffusion are powerful and find various real-world applications while customizing such models by fine-tuning is both memory and time inefficient.

Efficient Diffusion Personalization Position

Towards Effective Adversarial Textured 3D Meshes on Physical Face Recognition

1 code implementation CVPR 2023 Xiao Yang, Chang Liu, Longlong Xu, Yikai Wang, Yinpeng Dong, Ning Chen, Hang Su, Jun Zhu

The goal of this work is to develop a more reliable technique that can carry out an end-to-end evaluation of adversarial robustness for commercial systems.

Adversarial Robustness Face Recognition

One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale

3 code implementations12 Mar 2023 Fan Bao, Shen Nie, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yue, Yue Cao, Hang Su, Jun Zhu

Inspired by the unified view, UniDiffuser learns all distributions simultaneously with a minimal modification to the original diffusion model -- perturbs data in all modalities instead of a single modality, inputs individual timesteps in different modalities, and predicts the noise of all modalities instead of a single modality.

Text-to-Image Generation

Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

5 code implementations9 Mar 2023 Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang

To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion.

Referring Expression Referring Expression Comprehension +2

Task Aware Dreamer for Task Generalization in Reinforcement Learning

no code implementations9 Mar 2023 Chengyang Ying, Zhongkai Hao, Xinning Zhou, Hang Su, Songming Liu, Dong Yan, Jun Zhu

Extensive experiments in both image-based and state-based tasks show that TAD can significantly improve the performance of handling different tasks simultaneously, especially for those with high TDR, and display a strong generalization ability to unseen tasks.

reinforcement-learning Reinforcement Learning (RL)

Boundary-semantic collaborative guidance network with dual-stream feedback mechanism for salient object detection in optical remote sensing imagery

1 code implementation6 Mar 2023 Dejun Feng, Hongyu Chen, Suning Liu, Ziyang Liao, Xingyu Shen, Yakun Xie, Jun Zhu

Finally, to obtain more complete saliency maps, we consider the uniqueness of the last layer of the decoder for the first time and propose the adaptive feedback refinement (AFR) module, which further refines feature representation and eliminates differences between features through a unique feedback mechanism.

object-detection Object Detection +1

Model-based Constrained MDP for Budget Allocation in Sequential Incentive Marketing

no code implementations2 Mar 2023 Shuai Xiao, Le Guo, Zaifan Jiang, Lei Lv, Yuanbo Chen, Jun Zhu, Shuang Yang

Furthermore, we show that the dual problem can be solved by policy learning, with the optimal dual variable being found efficiently via bisection search (i. e., by taking advantage of the monotonicity).

counterfactual Marketing

To Make Yourself Invisible with Adversarial Semantic Contours

no code implementations1 Mar 2023 Yichi Zhang, Zijian Zhu, Hang Su, Jun Zhu, Shibao Zheng, Yuan He, Hui Xue

In this paper, we propose Adversarial Semantic Contour (ASC), an MAP estimate of a Bayesian formulation of sparse attack with a deceived prior of object contour.

Autonomous Driving Object +2

GNOT: A General Neural Operator Transformer for Operator Learning

2 code implementations28 Feb 2023 Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, Jun Zhu

However, there are several challenges for learning operators in practical applications like the irregular mesh, multiple input functions, and complexity of the PDEs' solution.

Operator learning

A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking

no code implementations28 Feb 2023 Chang Liu, Yinpeng Dong, Wenzhao Xiang, Xiao Yang, Hang Su, Jun Zhu, Yuefeng Chen, Yuan He, Hui Xue, Shibao Zheng

In our benchmark, we evaluate the robustness of 55 typical deep learning models on ImageNet with diverse architectures (e. g., CNNs, Transformers) and learning algorithms (e. g., normal supervised training, pre-training, adversarial training) under numerous adversarial attacks and out-of-distribution (OOD) datasets.

Adversarial Robustness Benchmarking +2

Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels

2 code implementations NeurIPS 2023 Zebin You, Yong Zhong, Fan Bao, Jiacheng Sun, Chongxuan Li, Jun Zhu

In an effort to further advance semi-supervised generative and classification tasks, we propose a simple yet effective training strategy called dual pseudo training (DPT), built upon strong semi-supervised learners and diffusion models.

Classification

Confidence-based Reliable Learning under Dual Noises

no code implementations10 Feb 2023 Peng Cui, Yang Yue, Zhijie Deng, Jun Zhu

Deep neural networks (DNNs) have achieved remarkable success in a variety of computer vision tasks, where massive labeled images are routinely required for model optimization.

Model Optimization

Revisiting Discriminative vs. Generative Classifiers: Theory and Implications

1 code implementation5 Feb 2023 Chenyu Zheng, Guoqiang Wu, Fan Bao, Yue Cao, Chongxuan Li, Jun Zhu

Theoretically, the paper considers the surrogate loss instead of the zero-one loss in analyses and generalizes the classical results from binary cases to multiclass ones.

Few-Shot Learning Image Classification +1

A Comprehensive Survey of Continual Learning: Theory, Method and Application

1 code implementation31 Jan 2023 Liyuan Wang, Xingxing Zhang, Hang Su, Jun Zhu

To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime.

Continual Learning Learning Theory

Semiparametric Regression for Spatial Data via Deep Learning

no code implementations10 Jan 2023 Kexuan Li, Jun Zhu, Anthony R. Ives, Volker C. Radeloff, Fangfang Wang

To be specific, we use a sparsely connected deep neural network with rectified linear unit (ReLU) activation function to estimate the unknown regression function that describes the relationship between response and covariates in the presence of spatial dependence.

regression

Why Are Conditional Generative Models Better Than Unconditional Ones?

no code implementations1 Dec 2022 Fan Bao, Chongxuan Li, Jiacheng Sun, Jun Zhu

Extensive empirical evidence demonstrates that conditional generative models are easier to train and perform better than unconditional ones by exploiting the labels of data.

DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding

1 code implementation28 Nov 2022 Shilong Liu, Yaoyuan Liang, Feng Li, Shijia Huang, Hao Zhang, Hang Su, Jun Zhu, Lei Zhang

As phrase extraction can be regarded as a $1$D text segmentation problem, we formulate PEG as a dual detection problem and propose a novel DQ-DETR model, which introduces dual queries to probe different features from image and text for object prediction and phrase mask prediction.

object-detection Object Detection +4

Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications

1 code implementation15 Nov 2022 Zhongkai Hao, Songming Liu, Yichi Zhang, Chengyang Ying, Yao Feng, Hang Su, Jun Zhu

Recent work shows that it provides potential benefits for machine learning models by incorporating the physical prior and collected data, which makes the intersection of machine learning and physics become a prevailing paradigm.

Physics-informed machine learning

DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models

1 code implementation2 Nov 2022 Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu

The commonly-used fast sampler for guided sampling is DDIM, a first-order diffusion ODE solver that generally needs 100 to 250 steps for high-quality samples.

Text-to-Image Generation

Model-based Reinforcement Learning with a Hamiltonian Canonical ODE Network

no code implementations2 Nov 2022 Yao Feng, Yuhong Jiang, Hang Su, Dong Yan, Jun Zhu

Model-based reinforcement learning usually suffers from a high sample complexity in training the world model, especially for the environments with complex dynamics.

Model-based Reinforcement Learning reinforcement-learning +1

Improving transferability of 3D adversarial attacks with scale and shear transformations

1 code implementation2 Nov 2022 Jinali Zhang, Yinpeng Dong, Jun Zhu, Jihong Zhu, Minchi Kuang, Xiaming Yuan

Extensive experiments show that the SS attack proposed in this paper can be seamlessly combined with the existing state-of-the-art (SOTA) 3D point cloud attack methods to form more powerful attack methods, and the SS attack improves the transferability over 3. 6 times compare to the baseline.

Spectral Representation Learning for Conditional Moment Models

no code implementations29 Oct 2022 Ziyu Wang, Yucen Luo, Yueru Li, Jun Zhu, Bernhard Schölkopf

For nonparametric conditional moment models, efficient estimation often relies on preimposed conditions on various measures of ill-posedness of the hypothesis space, which are hard to validate when flexible models are used.

Causal Inference Representation Learning

Isometric 3D Adversarial Examples in the Physical World

no code implementations27 Oct 2022 Yibo Miao, Yinpeng Dong, Jun Zhu, Xiao-Shan Gao

For naturalness, we constrain the adversarial example to be $\epsilon$-isometric to the original one by adopting the Gaussian curvature as a surrogate metric guaranteed by a theoretical analysis.

Accelerated Linearized Laplace Approximation for Bayesian Deep Learning

1 code implementation23 Oct 2022 Zhijie Deng, Feng Zhou, Jun Zhu

Laplace approximation (LA) and its linearized variant (LLA) enable effortless adaptation of pretrained deep neural networks to Bayesian neural networks.

Neural Eigenfunctions Are Structured Representation Learners

1 code implementation23 Oct 2022 Zhijie Deng, Jiaxin Shi, Hao Zhang, Peng Cui, Cewu Lu, Jun Zhu

Unlike prior spectral methods such as Laplacian Eigenmap that operate in a nonparametric manner, Neural Eigenmap leverages NeuralEF to parametrically model eigenfunctions using a neural network.

Contrastive Learning Data Augmentation +7

A Unified Hard-Constraint Framework for Solving Geometrically Complex PDEs

1 code implementation6 Oct 2022 Songming Liu, Zhongkai Hao, Chengyang Ying, Hang Su, Jun Zhu, Ze Cheng

We present a unified hard-constraint framework for solving geometrically complex PDEs with neural networks, where the most commonly used Dirichlet, Neumann, and Robin boundary conditions (BCs) are considered.

Equivariant Energy-Guided SDE for Inverse Molecular Design

2 code implementations30 Sep 2022 Fan Bao, Min Zhao, Zhongkai Hao, Peiyao Li, Chongxuan Li, Jun Zhu

Inverse molecular design is critical in material science and drug discovery, where the generated molecules should satisfy certain desirable properties.

3D Molecule Generation Drug Discovery

INT: Towards Infinite-frames 3D Detection with An Efficient Framework

1 code implementation30 Sep 2022 Jianyun Xu, Zhenwei Miao, Da Zhang, Hongyu Pan, Kaixuan Liu, Peihan Hao, Jun Zhu, Zhengyang Sun, Hongmin Li, Xin Zhan

By employing INT on CenterPoint, we can get around 7% (Waymo) and 15% (nuScenes) performance boost with only 2~4ms latency overhead, and currently SOTA on the Waymo 3D Detection leaderboard.

Offline Reinforcement Learning via High-Fidelity Generative Behavior Modeling

1 code implementation29 Sep 2022 Huayu Chen, Cheng Lu, Chengyang Ying, Hang Su, Jun Zhu

To address this problem, we adopt a generative approach by decoupling the learned policy into two parts: an expressive generative behavior model and an action evaluation model.

Computational Efficiency D4RL +4

All are Worth Words: A ViT Backbone for Diffusion Models

3 code implementations CVPR 2023 Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, Jun Zhu

We evaluate U-ViT in unconditional and class-conditional image generation, as well as text-to-image generation tasks, where U-ViT is comparable if not superior to a CNN-based U-Net of a similar size.

Conditional Image Generation Text-to-Image Generation

On the Reuse Bias in Off-Policy Reinforcement Learning

1 code implementation15 Sep 2022 Chengyang Ying, Zhongkai Hao, Xinning Zhou, Hang Su, Dong Yan, Jun Zhu

In this paper, we reveal that the instability is also related to a new notion of Reuse Bias of IS -- the bias in off-policy evaluation caused by the reuse of the replay buffer for evaluation and optimization.

Continuous Control Off-policy evaluation +1

Bi-level Physics-Informed Neural Networks for PDE Constrained Optimization using Broyden's Hypergradients

no code implementations15 Sep 2022 Zhongkai Hao, Chengyang Ying, Hang Su, Jun Zhu, Jian Song, Ze Cheng

In this paper, we present a novel bi-level optimization framework to resolve the challenge by decoupling the optimization of the targets and constraints.

Regret Analysis for Hierarchical Experts Bandit Problem

no code implementations11 Aug 2022 Qihan Guo, Siwei Wang, Jun Zhu

We study an extension of standard bandit problem in which there are R layers of experts.

Robust Learning of Deep Time Series Anomaly Detection Models with Contaminated Training Data

no code implementations3 Aug 2022 Wenkai Li, Cheng Feng, Ting Chen, Jun Zhu

In this work, to tackle this important challenge, we firstly investigate the robustness of commonly used deep TSAD methods with contaminated training data which provides a guideline for applying these methods when the provided training data are not guaranteed to be anomaly-free.

Anomaly Detection Time Series +1

EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations

1 code implementation14 Jul 2022 Min Zhao, Fan Bao, Chongxuan Li, Jun Zhu

Further, we provide an alternative explanation of the EGSDE as a product of experts, where each of the three experts (corresponding to the SDE and two feature extractors) solely contributes to faithfulness or realism.

Image-to-Image Translation Translation

DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimization

1 code implementation12 Jul 2022 Wentse Chen, Shiyu Huang, Yuan Chiang, Tim Pearce, Wei-Wei Tu, Ting Chen, Jun Zhu

We propose Diversity-Guided Policy Optimization (DGPO), an on-policy algorithm that discovers multiple strategies for solving a given task.

reinforcement-learning Reinforcement Learning (RL)

Thompson Sampling for (Combinatorial) Pure Exploration

no code implementations18 Jun 2022 Siwei Wang, Jun Zhu

To make the algorithm efficient, they usually use the sum of upper confidence bounds within arm set $S$ to represent the upper confidence bound of $S$, which can be much larger than the tight upper confidence bound of $S$ and leads to a much higher complexity than necessary, since the empirical means of different arms in $S$ are independent.

Thompson Sampling

Fast Lossless Neural Compression with Integer-Only Discrete Flows

1 code implementation17 Jun 2022 Siyu Wang, Jianfei Chen, Chongxuan Li, Jun Zhu, Bo Zhang

In this work, we propose Integer-only Discrete Flows (IODF), an efficient neural compressor with integer-only arithmetic.

Quantization

Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matching

1 code implementation16 Jun 2022 Cheng Lu, Kaiwen Zheng, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu

To fill up this gap, we show that the negative likelihood of the ODE can be bounded by controlling the first, second, and third-order score matching errors; and we further present a novel high-order denoising score matching method to enable maximum likelihood training of score-based diffusion ODEs.

Denoising

Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models

1 code implementation15 Jun 2022 Fan Bao, Chongxuan Li, Jiacheng Sun, Jun Zhu, Bo Zhang

Thus, the generation performance on a subset of timesteps is crucial, which is greatly influenced by the covariance design in DPMs.

Computational Efficiency

Diagnosing Ensemble Few-Shot Classifiers

no code implementations9 Jun 2022 Weikai Yang, Xi Ye, Xingxing Zhang, Lanxi Xiao, Jiazhi Xia, Zhongyuan Wang, Jun Zhu, Hanspeter Pfister, Shixia Liu

The base learners and labeled samples (shots) in an ensemble few-shot classifier greatly affect the model performance.

GSmooth: Certified Robustness against Semantic Transformations via Generalized Randomized Smoothing

no code implementations9 Jun 2022 Zhongkai Hao, Chengyang Ying, Yinpeng Dong, Hang Su, Jun Zhu, Jian Song

Under the GSmooth framework, we present a scalable algorithm that uses a surrogate image-to-image network to approximate the complex transformation.

Towards Safe Reinforcement Learning via Constraining Conditional Value-at-Risk

1 code implementation9 Jun 2022 Chengyang Ying, Xinning Zhou, Hang Su, Dong Yan, Ning Chen, Jun Zhu

Though deep reinforcement learning (DRL) has obtained substantial success, it may encounter catastrophic failures due to the intrinsic uncertainty of both transition and observation.

Continuous Control reinforcement-learning +2

BadDet: Backdoor Attacks on Object Detection

no code implementations28 May 2022 Shih-Han Chan, Yinpeng Dong, Jun Zhu, Xiaolu Zhang, Jun Zhou

We propose four kinds of backdoor attacks for object detection task: 1) Object Generation Attack: a trigger can falsely generate an object of the target class; 2) Regional Misclassification Attack: a trigger can change the prediction of a surrounding object to the target class; 3) Global Misclassification Attack: a single trigger can change the predictions of all objects in an image to the target class; and 4) Object Disappearance Attack: a trigger can make the detector fail to detect the object of the target class.

Autonomous Driving Backdoor Attack +4

Censored Quantile Regression Neural Networks for Distribution-Free Survival Analysis

1 code implementation26 May 2022 Tim Pearce, Jong-Hyeon Jeong, Yichen Jia, Jun Zhu

To offer theoretical insight into our algorithm, we show firstly that it can be interpreted as a form of expectation-maximisation, and secondly that it exhibits a desirable `self-correcting' property.

regression Survival Analysis

Fast Instrument Learning with Faster Rates

1 code implementation22 May 2022 Ziyu Wang, Yuhao Zhou, Jun Zhu

We investigate nonlinear instrumental variable (IV) regression given high-dimensional instruments.

Model Selection regression +1

Towards Job-Transition-Tag Graph for a Better Job Title Representation Learning

1 code implementation Findings (NAACL) 2022 Jun Zhu, Céline Hudelot

Works on learning job title representation are mainly based on \textit{Job-Transition Graph}, built from the working history of talents.

Representation Learning TAG

NeuralEF: Deconstructing Kernels by Deep Neural Networks

2 code implementations30 Apr 2022 Zhijie Deng, Jiaxin Shi, Jun Zhu

Learning the principal eigenfunctions of an integral operator defined by a kernel and a data distribution is at the core of many machine learning problems.

Image Classification

Policy Learning for Robust Markov Decision Process with a Mismatched Generative Model

no code implementations13 Mar 2022 Jialian Li, Tongzheng Ren, Dong Yan, Hang Su, Jun Zhu

Our goal is to identify a near-optimal robust policy for the perturbed testing environment, which introduces additional technical difficulties as we need to simultaneously estimate the training environment uncertainty from samples and find the worst-case perturbation for testing.

Query-Efficient Black-box Adversarial Attacks Guided by a Transfer-based Prior

1 code implementation13 Mar 2022 Yinpeng Dong, Shuyu Cheng, Tianyu Pang, Hang Su, Jun Zhu

However, the existing methods inevitably suffer from low attack success rates or poor query efficiency since it is difficult to estimate the gradient in a high-dimensional input space with limited information.

Controllable Evaluation and Generation of Physical Adversarial Patch on Face Recognition

no code implementations9 Mar 2022 Xiao Yang, Yinpeng Dong, Tianyu Pang, Zihao Xiao, Hang Su, Jun Zhu

It is therefore imperative to develop a framework that can enable a comprehensive evaluation of the vulnerability of face recognition in the physical world.

3D Face Modelling Face Recognition

DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection

14 code implementations7 Mar 2022 Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M. Ni, Heung-Yeung Shum

Compared to other models on the leaderboard, DINO significantly reduces its model size and pre-training data size while achieving better results.

Real-Time Object Detection

Robustness and Accuracy Could Be Reconcilable by (Proper) Definition

1 code implementation21 Feb 2022 Tianyu Pang, Min Lin, Xiao Yang, Jun Zhu, Shuicheng Yan

The trade-off between robustness and accuracy has been widely studied in the adversarial literature.

Inductive Bias

Memory Replay with Data Compression for Continual Learning

1 code implementation ICLR 2022 Liyuan Wang, Xingxing Zhang, Kuo Yang, Longhui Yu, Chongxuan Li, Lanqing Hong, Shifeng Zhang, Zhenguo Li, Yi Zhong, Jun Zhu

In this work, we propose memory replay with data compression (MRDC) to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer.

Autonomous Driving Class Incremental Learning +5

DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR

7 code implementations ICLR 2022 Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, Lei Zhang

We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR.

Object Detection

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

2 code implementations ICLR 2022 Fan Bao, Chongxuan Li, Jun Zhu, Bo Zhang

In this work, we present a surprising result that both the optimal reverse variance and the corresponding optimal KL divergence of a DPM have analytic forms w. r. t.

AutoLoss-GMS: Searching Generalized Margin-Based Softmax Loss Function for Person Re-Identification

no code implementations CVPR 2022 Hongyang Gu, Jianmin Li, Guangyuan Fu, Chifong Wong, Xinghao Chen, Jun Zhu

In this paper, we propose a novel method, AutoLoss-GMS, to search the better loss function in the space of generalized margin-based softmax loss function for person re-identification automatically.

Person Re-Identification

BE-STI: Spatial-Temporal Integrated Network for Class-Agnostic Motion Prediction With Bidirectional Enhancement

no code implementations CVPR 2022 Yunlong Wang, Hongyu Pan, Jun Zhu, Yu-Huan Wu, Xin Zhan, Kun Jiang, Diange Yang

In this paper, we propose a novel Spatial-Temporal Integrated network with Bidirectional Enhancement, BE-STI, to improve the temporal motion prediction performance by spatial semantic features, which points out an efficient way to combine semantic segmentation and motion prediction.

Autonomous Driving motion prediction +1

AFEC: Active Forgetting of Negative Transfer in Continual Learning

1 code implementation NeurIPS 2021 Liyuan Wang, Mingtian Zhang, Zhongfan Jia, Qian Li, Chenglong Bao, Kaisheng Ma, Jun Zhu, Yi Zhong

Without accessing to the old training samples, knowledge transfer from the old tasks to each new task is difficult to determine, which might be either positive or negative.

Continual Learning Transfer Learning

Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness

no code implementations13 Oct 2021 Xiao Yang, Yinpeng Dong, Wenzhao Xiang, Tianyu Pang, Hang Su, Jun Zhu

The vulnerability of deep neural networks to adversarial examples has motivated an increasing number of defense strategies for promoting model robustness.

Adversarial Robustness

TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrations

1 code implementation9 Oct 2021 Shiyu Huang, Wenze Chen, Longfei Zhang, Shizhen Xu, Ziyang Li, Fengming Zhu, Deheng Ye, Ting Chen, Jun Zhu

To the best of our knowledge, Tikick is the first learning-based AI system that can take over the multi-agent Google Research Football full game, while previous work could either control a single agent or experiment on toy academic scenarios.

Starcraft Starcraft II

Ranking Cost: Building An Efficient and Scalable Circuit Routing Planner with Evolution-Based Optimization

1 code implementation8 Oct 2021 Shiyu Huang, Bin Wang, Dong Li, Jianye Hao, Ting Chen, Jun Zhu

In this work, we propose a new algorithm for circuit routing, named Ranking Cost, which innovatively combines search-based methods (i. e., A* algorithm) and learning-based methods (i. e., Evolution Strategies) to form an efficient and trainable router.

Adversarial Semantic Contour for Object Detection

no code implementations ICML Workshop AML 2021 Yichi Zhang, Zijian Zhu, Xiao Yang, Jun Zhu

To address this issue, we propose a novel method of Adversarial Semantic Contour (ASC) guided by object contour as prior.

Object object-detection +1

Regularized-OFU: an efficient algorithm for general contextual bandit with optimization oracles

no code implementations29 Sep 2021 Yichi Zhou, Shihong Song, Huishuai Zhang, Jun Zhu, Wei Chen, Tie-Yan Liu

In contextual bandit, one major challenge is to develop theoretically solid and empirically efficient algorithms for general function classes.

Multi-Armed Bandits Thompson Sampling

Deep Ensemble as a Gaussian Process Posterior

no code implementations29 Sep 2021 Zhijie Deng, Feng Zhou, Jianfei Chen, Guoqiang Wu, Jun Zhu

Deep Ensemble (DE) is a flexible, feasible, and effective alternative to Bayesian neural networks (BNNs) for uncertainty estimation in deep learning.

Variational Inference

Cluster Attack: Query-based Adversarial Attacks on Graphs with Graph-Dependent Priors

1 code implementation ICML Workshop AML 2021 Zhengyi Wang, Zhongkai Hao, Ziqiao Wang, Hang Su, Jun Zhu

In this work, we propose Cluster Attack -- a Graph Injection Attack (GIA) on node classification, which injects fake nodes into the original graph to degenerate the performance of graph neural networks (GNNs) on certain victim nodes while affecting the other nodes as little as possible.

Adversarial Attack Clustering +3

Tianshou: a Highly Modularized Deep Reinforcement Learning Library

1 code implementation29 Jul 2021 Jiayi Weng, Huayu Chen, Dong Yan, Kaichao You, Alexis Duburcq, Minghao Zhang, Yi Su, Hang Su, Jun Zhu

In this paper, we present Tianshou, a highly modularized Python library for deep reinforcement learning (DRL) that uses PyTorch as its backend.

reinforcement-learning Reinforcement Learning (RL)

Query2Label: A Simple Transformer Way to Multi-Label Classification

2 code implementations22 Jul 2021 Shilong Liu, Lei Zhang, Xiao Yang, Hang Su, Jun Zhu

The use of Transformer is rooted in the need of extracting local discriminative features adaptively for different labels, which is a strongly desired property due to the existence of multiple objects in one image.

Classification Multi-Label Classification

On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithms

1 code implementation NeurIPS 2021 Shuyu Cheng, Guoqiang Wu, Jun Zhu

Finally, our theoretical results are confirmed by experiments on several numerical benchmarks as well as adversarial attacks.

Understanding Adversarial Attacks on Observations in Deep Reinforcement Learning

no code implementations30 Jun 2021 You Qiaoben, Chengyang Ying, Xinning Zhou, Hang Su, Jun Zhu, Bo Zhang

In this paper, we provide a framework to better understand the existing methods by reformulating the problem of adversarial attacks on reinforcement learning in the function space.

reinforcement-learning Reinforcement Learning (RL)

Improving Transferability of Adversarial Patches on Face Recognition with Generative Models

no code implementations CVPR 2021 Zihao Xiao, Xianfeng Gao, Chilin Fu, Yinpeng Dong, Wei Gao, Xiaolu Zhang, Jun Zhou, Jun Zhu

However, deep CNNs are vulnerable to adversarial patches, which are physically realizable and stealthy, raising new security concerns on the real-world applications of these models.

Face Recognition

Regularized OFU: an Efficient UCB Estimator forNon-linear Contextual Bandit

no code implementations29 Jun 2021 Yichi Zhou, Shihong Song, Huishuai Zhang, Jun Zhu, Wei Chen, Tie-Yan Liu

However, it is in general unknown how to deriveefficient and effective EE trade-off methods for non-linearcomplex tasks, suchas contextual bandit with deep neural network as the reward function.

Multi-Armed Bandits

Towards Safe Reinforcement Learning via Constraining Conditional Value at Risk

no code implementations ICML Workshop AML 2021 Chengyang Ying, Xinning Zhou, Dong Yan, Jun Zhu

Though deep reinforcement learning (DRL) has obtained substantial success, it may encounter catastrophic failures due to the intrinsic uncertainty caused by stochastic policies and environment variability.

Continuous Control reinforcement-learning +2

Strategically-timed State-Observation Attacks on Deep Reinforcement Learning Agents

no code implementations ICML Workshop AML 2021 You Qiaoben, Xinning Zhou, Chengyang Ying, Jun Zhu

Deep reinforcement learning (DRL) policies are vulnerable to the adversarial attack on their observations, which may mislead real-world RL agents to catastrophic failures.

Adversarial Attack Continuous Control +2

Accumulative Poisoning Attacks on Real-time Data

1 code implementation NeurIPS 2021 Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, Jun Zhu

Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy.

Federated Learning

Quasi-Bayesian Dual Instrumental Variable Regression

1 code implementation NeurIPS 2021 Ziyu Wang, Yuhao Zhou, Tongzheng Ren, Jun Zhu

Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking.

Bayesian Inference regression +1

Understanding Softmax Confidence and Uncertainty

no code implementations9 Jun 2021 Tim Pearce, Alexandra Brintrup, Jun Zhu

It is often remarked that neural networks fail to increase their uncertainty when predicting on data far from the training distribution.

Out of Distribution (OOD) Detection

Nonlinear Hawkes Processes in Time-Varying System

no code implementations9 Jun 2021 Feng Zhou, Quyu Kong, Yixuan Zhang, Cheng Feng, Jun Zhu

Hawkes processes are a class of point processes that have the ability to model the self- and mutual-exciting phenomena.

Bayesian Inference Point Processes +1

Stability and Generalization of Bilevel Programming in Hyperparameter Optimization

1 code implementation NeurIPS 2021 Fan Bao, Guoqiang Wu, Chongxuan Li, Jun Zhu, Bo Zhang

Our results can explain some mysterious behaviours of the bilevel programming in practice, for instance, overfitting to the validation set.

Hyperparameter Optimization

Exploring Memorization in Adversarial Training

1 code implementation ICLR 2022 Yinpeng Dong, Ke Xu, Xiao Yang, Tianyu Pang, Zhijie Deng, Hang Su, Jun Zhu

In this paper, we explore the memorization effect in adversarial training (AT) for promoting a deeper understanding of model capacity, convergence, generalization, and especially robust overfitting of the adversarially trained models.

Memorization

Physics-Guided Discovery of Highly Nonlinear Parametric Partial Differential Equations

no code implementations2 Jun 2021 Yingtao Luo, Qiang Liu, Yuntian Chen, WenBo Hu, Tian Tian, Jun Zhu

Especially, the discovery of PDEs with highly nonlinear coefficients from low-quality data remains largely under-addressed.

Density Estimation Model Optimization

Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart

1 code implementation CVPR 2022 Tianyu Pang, Huishuai Zhang, Di He, Yinpeng Dong, Hang Su, Wei Chen, Jun Zhu, Tie-Yan Liu

Along with this routine, we find that confidence and a rectified confidence (R-Con) can form two coupled rejection metrics, which could provably distinguish wrongly classified inputs from correctly classified ones.

Vocal Bursts Valence Prediction

Unsupervised Part Segmentation through Disentangling Appearance and Shape

no code implementations CVPR 2021 Shilong Liu, Lei Zhang, Xiao Yang, Hang Su, Jun Zhu

We study the problem of unsupervised discovery and segmentation of object parts, which, as an intermediate local representation, are capable of finding intrinsic object structure and providing more explainable recognition results.

Disentanglement Object +3

Scalable Quasi-Bayesian Inference for Instrumental Variable Regression

no code implementations NeurIPS 2021 Ziyu Wang, Yuhao Zhou, Tongzheng Ren, Jun Zhu

Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking.

Bayesian Inference regression +1

Rethinking and Reweighting the Univariate Losses for Multi-Label Ranking: Consistency and Generalization

no code implementations NeurIPS 2021 Guoqiang Wu, Chongxuan Li, Kun Xu, Jun Zhu

Our results show that learning algorithms with the consistent univariate loss have an error bound of $O(c)$ ($c$ is the number of labels), while algorithms with the inconsistent pairwise loss depend on $O(\sqrt{c})$ as shown in prior work.

Computational Efficiency Multi-Label Classification

Automated Decision-based Adversarial Attacks

no code implementations9 May 2021 Qi-An Fu, Yinpeng Dong, Hang Su, Jun Zhu

Deep learning models are vulnerable to adversarial examples, which can fool a target classifier by imposing imperceptible perturbations onto natural examples.

Adversarial Attack Program Synthesis

MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering

1 code implementation ICLR 2021 Tsung Wei Tsai, Chongxuan Li, Jun Zhu

We present Mixture of Contrastive Experts (MiCE), a unified probabilistic clustering framework that simultaneously exploits the discriminative representations learned by contrastive learning and the semantic structures captured by a latent mixture model.

Clustering Contrastive Learning +1

Few-shot Continual Learning: a Brain-inspired Approach

no code implementations19 Apr 2021 Liyuan Wang, Qian Li, Yi Zhong, Jun Zhu

Our solution is based on the observation that continual learning of a task sequence inevitably interferes few-shot generalization, which makes it highly nontrivial to extend few-shot learning strategies to continual learning scenarios.

Continual Learning Few-Shot Learning

Counter-Strike Deathmatch with Large-Scale Behavioural Cloning

2 code implementations9 Apr 2021 Tim Pearce, Jun Zhu

This paper describes an AI agent that plays the popular first-person-shooter (FPS) video game `Counter-Strike; Global Offensive' (CSGO) from pixel input.

Behavioural cloning FPS Games

Accurate and Reliable Forecasting using Stochastic Differential Equations

no code implementations28 Mar 2021 Peng Cui, Zhijie Deng, WenBo Hu, Jun Zhu

It is critical yet challenging for deep learning models to properly characterize uncertainty that is pervasive in real-world environments.

Prediction Intervals Uncertainty Quantification

Black-box Detection of Backdoor Attacks with Limited Information and Data

no code implementations ICCV 2021 Yinpeng Dong, Xiao Yang, Zhijie Deng, Tianyu Pang, Zihao Xiao, Hang Su, Jun Zhu

Although deep neural networks (DNNs) have made rapid progress in recent years, they are vulnerable in adversarial environments.

Implicit Normalizing Flows

1 code implementation ICLR 2021 Cheng Lu, Jianfei Chen, Chongxuan Li, Qiuhao Wang, Jun Zhu

Through theoretical analysis, we show that the function space of ImpFlow is strictly richer than that of ResFlows.

DNN2LR: Automatic Feature Crossing for Credit Scoring

no code implementations24 Feb 2021 Qiang Liu, Zhaocheng Liu, Haoli Zhang, Yuntian Chen, Jun Zhu

Accordingly, we can design an automatic feature crossing method to find feature interactions in DNN, and use them as cross features in LR.

Feature Engineering

Rethinking Natural Adversarial Examples for Classification Models

1 code implementation23 Feb 2021 Xiao Li, Jianmin Li, Ting Dai, Jie Shi, Jun Zhu, Xiaolin Hu

A detection model based on the classification model EfficientNet-B7 achieved a top-1 accuracy of 53. 95%, surpassing previous state-of-the-art classification models trained on ImageNet, suggesting that accurate localization information can significantly boost the performance of classification models on ImageNet-A.

Classification General Classification +2

High-fidelity Prediction of Megapixel Longitudinal Phase-space Images of Electron Beams using Encoder-Decoder Neural Networks

no code implementations25 Jan 2021 Jun Zhu, Ye Chen, Frank Brinker, Winfried Decking, Sergey Tomin, Holger Schlarb

We also show the scalability and interpretability of the model by sharing the same decoder with more than one encoder used for different setups of the photoinjector, and propose a pragmatic way to model a facility with various diagnostics and working points.

Cognitive Visual Inspection Service for LCD Manufacturing Industry

no code implementations11 Jan 2021 Yuanyuan Ding, Junchi Yan, Guoqiang Hu, Jun Zhu

This paper discloses a novel visual inspection system for liquid crystal display (LCD), which is currently a dominant type in the FPD industry.

Defect Detection

Relaxed Conditional Image Transfer for Semi-supervised Domain Adaptation

no code implementations5 Jan 2021 Qijun Luo, Zhili Liu, Lanqing Hong, Chongxuan Li, Kuo Yang, Liyuan Wang, Fengwei Zhou, Guilin Li, Zhenguo Li, Jun Zhu

Semi-supervised domain adaptation (SSDA), which aims to learn models in a partially labeled target domain with the assistance of the fully labeled source domain, attracts increasing attention in recent years.

Domain Adaptation Semi-supervised Domain Adaptation

Ranking Cost: One-Stage Circuit Routing by Directly Optimizing Global Objective Function

no code implementations1 Jan 2021 Shiyu Huang, Bin Wang, Dong Li, Jianye Hao, Jun Zhu, Ting Chen

In our method, we introduce a new set of variables called cost maps, which can help the A* router to find out proper paths to achieve the global object.

Adaptive N-step Bootstrapping with Off-policy Data

no code implementations1 Jan 2021 Guan Wang, Dong Yan, Hang Su, Jun Zhu

In this work, we point out that the optimal value of n actually differs on each data point, while the fixed value n is a rough average of them.

Atari Games

Series Saliency: Temporal Interpretation for Multivariate Time Series Forecasting

no code implementations16 Dec 2020 Qingyi Pan, WenBo Hu, Jun Zhu

Though deep learning methods have recently been developed to give superior forecasting results, it is crucial to improve the interpretability of time series models.

Data Augmentation Multivariate Time Series Forecasting +1

Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings

1 code implementation14 Dec 2020 Qipeng Guo, Zhijing Jin, Ziyu Wang, Xipeng Qiu, Weinan Zhang, Jun Zhu, Zheng Zhang, David Wipf

Cycle-consistent training is widely used for jointly learning a forward and inverse mapping between two domains of interest without the cumbersome requirement of collecting matched pairs within each domain.

Knowledge Graphs Text Generation

Understanding and Exploring the Network with Stochastic Architectures

1 code implementation NeurIPS 2020 Zhijie Deng, Yinpeng Dong, Shifeng Zhang, Jun Zhu

In this work, we decouple the training of a network with stochastic architectures (NSA) from NAS and provide a first systematical investigation on it as a stand-alone problem.

Neural Architecture Search

Multi-label classification: do Hamming loss and subset accuracy really conflict with each other?

1 code implementation NeurIPS 2020 Guoqiang Wu, Jun Zhu

On the other hand, when directly optimizing SA with its surrogate loss, it has learning guarantees that depend on $O(\sqrt{c})$ for both HL and SA measures.

General Classification Multi-Label Classification

Further Analysis of Outlier Detection with Deep Generative Models

1 code implementation NeurIPS 2020 Ziyu Wang, Bin Dai, David Wipf, Jun Zhu

The recent, counter-intuitive discovery that deep generative models (DGMs) can frequently assign a higher likelihood to outliers has implications for both outlier detection applications as well as our overall understanding of generative modeling.

Outlier Detection

Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models

1 code implementation NeurIPS Workshop ICBINB 2020 Fan Bao, Kun Xu, Chongxuan Li, Lanqing Hong, Jun Zhu, Bo Zhang

The learning and evaluation of energy-based latent variable models (EBLVMs) without any structural assumptions are highly challenging, because the true posteriors and the partition functions in such models are generally intractable.

Bi-level Score Matching for Learning Energy-based Latent Variable Models

1 code implementation NeurIPS 2020 Fan Bao, Chongxuan Li, Kun Xu, Hang Su, Jun Zhu, Bo Zhang

This paper presents a bi-level score matching (BiSM) method to learn EBLVMs with general structures by reformulating SM as a bi-level optimization problem.

Rolling Shutter Correction Stochastic Optimization

BayesAdapter: Being Bayesian, Inexpensively and Reliably, via Bayesian Fine-tuning

1 code implementation5 Oct 2020 Zhijie Deng, Jun Zhu

Despite their theoretical appealingness, Bayesian neural networks (BNNs) are left behind in real-world adoption, mainly due to persistent concerns on their scalability, accessibility, and reliability.

Variational Inference

Bag of Tricks for Adversarial Training

2 code implementations ICLR 2021 Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, Jun Zhu

Adversarial training (AT) is one of the most effective strategies for promoting model robustness.

Adversarial Robustness Benchmarking

BayesAdapter: Being Bayesian, Inexpensively and Robustly, via Bayesian Fine-tuning

no code implementations28 Sep 2020 Zhijie Deng, Xiao Yang, Hao Zhang, Yinpeng Dong, Jun Zhu

Despite their theoretical appealingness, Bayesian neural networks (BNNs) are falling far behind in terms of adoption in real-world applications compared with normal NNs, mainly due to their limited scalability in training, and low fidelity in their uncertainty estimates.

Uncertainty Quantification Variational Inference

Switching Transferable Gradient Directions for Query-Efficient Black-Box Adversarial Attacks

no code implementations15 Sep 2020 Chen Ma, Shuyu Cheng, Li Chen, Jun Zhu, Junhai Yong

In each iteration, SWITCH first tries to update the current sample along the direction of $\hat{\mathbf{g}}$, but considers switching to its opposite direction $-\hat{\mathbf{g}}$ if our algorithm detects that it does not increase the value of the attack objective function.

Adversarial Attack

Training Interpretable Convolutional Neural Networks by Differentiating Class-specific Filters

1 code implementation ECCV 2020 Haoyu Liang, Zhihao Ouyang, Yuyuan Zeng, Hang Su, Zihao He, Shu-Tao Xia, Jun Zhu, Bo Zhang

Most existing works attempt post-hoc interpretation on a pre-trained model, while neglecting to reduce the entanglement underlying the model.

Object Localization

RobFR: Benchmarking Adversarial Robustness on Face Recognition

2 code implementations8 Jul 2020 Xiao Yang, Dingcheng Yang, Yinpeng Dong, Hang Su, Wenjian Yu, Jun Zhu

Based on large-scale evaluations, the commercial FR API services fail to exhibit acceptable performance on robustness evaluation, and we also draw several important conclusions for understanding the adversarial robustness of FR models and providing insights for the design of robust FR models.

Adversarial Robustness Benchmarking +1

Efficient Learning of Generative Models via Finite-Difference Score Matching

1 code implementation NeurIPS 2020 Tianyu Pang, Kun Xu, Chongxuan Li, Yang song, Stefano Ermon, Jun Zhu

Several machine learning applications involve the optimization of higher-order derivatives (e. g., gradients of gradients) during training, which can be expensive in respect to memory and computation even with automatic differentiation.

Efficient Inference of Flexible Interaction in Spiking-neuron Networks

no code implementations ICLR 2021 Feng Zhou, Yixuan Zhang, Jun Zhu

Hawkes process provides an effective statistical framework for analyzing the time-dependent interaction of neuronal spiking activities.

Dynamic Window-level Granger Causality of Multi-channel Time Series

no code implementations14 Jun 2020 Zhiheng Zhang, Wen-Bo Hu, Tian Tian, Jun Zhu

In this paper, we present the dynamic window-level Granger causality method (DWGC) for multi-channel time series data.

Time Series Time Series Analysis

Brain-inspired global-local learning incorporated with neuromorphic computing

no code implementations5 Jun 2020 Yujie Wu, Rong Zhao, Jun Zhu, Feng Chen, Mingkun Xu, Guoqi Li, Sen Song, Lei Deng, Guanrui Wang, Hao Zheng, Jing Pei, Youhui Zhang, Mingguo Zhao, Luping Shi

We demonstrate the advantages of this model in multiple different tasks, including few-shot learning, continual learning, and fault-tolerance learning in neuromorphic vision sensors.

Continual Learning Few-Shot Learning

Nonparametric Score Estimators

1 code implementation ICML 2020 Yuhao Zhou, Jiaxin Shi, Jun Zhu

Estimating the score, i. e., the gradient of log density function, from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models that involve flexible yet intractable densities.

Lazy-CFR: fast and near-optimal regret minimization for extensive games with imperfect information

no code implementations ICLR 2020 Yichi Zhou, Tongzheng Ren, Jialian Li, Dong Yan, Jun Zhu

In this paper, we present Lazy-CFR, a CFR algorithm that adopts a lazy update strategy to avoid traversing the whole game tree in each round.

counterfactual

SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models

no code implementations ICLR 2020 Yucen Luo, Alex Beatson, Mohammad Norouzi, Jun Zhu, David Duvenaud, Ryan P. Adams, Ricky T. Q. Chen

Standard variational lower bounds used to train latent variable models produce biased estimates of most quantities of interest.

Towards Face Encryption by Generating Adversarial Identity Masks

1 code implementation ICCV 2021 Xiao Yang, Yinpeng Dong, Tianyu Pang, Hang Su, Jun Zhu, Yuefeng Chen, Hui Xue

As billions of personal data being shared through social media and network, the data privacy and security have drawn an increasing attention.

Face Recognition

Triple Memory Networks: a Brain-Inspired Method for Continual Learning

1 code implementation6 Mar 2020 Liyuan Wang, Bo Lei, Qian Li, Hang Su, Jun Zhu, Yi Zhong

Continual acquisition of novel experience without interfering previously learned knowledge, i. e. continual learning, is critical for artificial neural networks, but limited by catastrophic forgetting.

Attribute Class Incremental Learning +2

VFlow: More Expressive Generative Flows with Variational Data Augmentation

1 code implementation ICML 2020 Jianfei Chen, Cheng Lu, Biqi Chenli, Jun Zhu, Tian Tian

Generative flows are promising tractable models for density modeling that define probabilistic distributions with invertible transformations.

Ranked #30 on Image Generation on CIFAR-10 (bits/dimension metric)

Density Estimation Image Generation +2

Boosting Adversarial Training with Hypersphere Embedding

1 code implementation NeurIPS 2020 Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, Hang Su

Adversarial training (AT) is one of the most effective defenses against adversarial attacks for deep learning models.

Representation Learning

A Wasserstein Minimum Velocity Approach to Learning Unnormalized Models

1 code implementation pproximateinference AABI Symposium 2019 Ziyu Wang, Shuyu Cheng, Yueru Li, Jun Zhu, Bo Zhang

Score matching provides an effective approach to learning flexible unnormalized models, but its scalability is limited by the need to evaluate a second-order derivative.

Adversarial Distributional Training for Robust Deep Learning

1 code implementation NeurIPS 2020 Yinpeng Dong, Zhijie Deng, Tianyu Pang, Hang Su, Jun Zhu

Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples.

Analyzing the Noise Robustness of Deep Neural Networks

no code implementations26 Jan 2020 Kelei Cao, Mengchen Liu, Hang Su, Jing Wu, Jun Zhu, Shixia Liu

The key is to compare and analyze the datapaths of both the adversarial and normal examples.

Adversarial Attack

SVQN: Sequential Variational Soft Q-Learning Networks

no code implementations ICLR 2020 Shiyu Huang, Hang Su, Jun Zhu, Ting Chen

Partially Observable Markov Decision Processes (POMDPs) are popular and flexible models for real-world decision-making applications that demand the information from past observations to make optimal decisions.

Decision Making Q-Learning +2

Benchmarking Adversarial Robustness

no code implementations26 Dec 2019 Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Hang Su, Zihao Xiao, Jun Zhu

Deep neural networks are vulnerable to adversarial examples, which becomes one of the most important research problems in the development of deep learning.

Adversarial Attack Adversarial Robustness +2

Triple Generative Adversarial Networks

1 code implementation20 Dec 2019 Chongxuan Li, Kun Xu, Jiashuo Liu, Jun Zhu, Bo Zhang

It is formulated as a three-player minimax game consisting of a generator, a classifier and a discriminator, and therefore is referred to as Triple Generative Adversarial Network (Triple-GAN).

Classification Conditional Image Generation +4

The Search for Sparse, Robust Neural Networks

1 code implementation5 Dec 2019 Justin Cosentino, Federico Zaiter, Dan Pei, Jun Zhu

Recent work on deep neural network pruning has shown there exist sparse subnetworks that achieve equal or improved accuracy, training time, and loss using fewer network parameters when compared to their dense counterparts.

Network Pruning

Design and Interpretation of Universal Adversarial Patches in Face Detection

no code implementations ECCV 2020 Xiao Yang, Fangyun Wei, Hongyang Zhang, Jun Zhu

We consider universal adversarial patches for faces -- small visual elements whose addition to a face image reliably destroys the performance of face detectors.

Face Detection

Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structure

1 code implementation22 Nov 2019 Zhijie Deng, Yucen Luo, Jun Zhu, Bo Zhang

Bayesian neural networks (BNNs) augment deep networks with uncertainty quantification by Bayesian treatment of the network weights.

Bayesian Inference Neural Architecture Search +2

Generative Well-intentioned Networks

no code implementations NeurIPS 2019 Justin Cosentino, Jun Zhu

We propose Generative Well-intentioned Networks (GWINs), a novel framework for increasing the accuracy of certainty-based, closed-world classifiers.

Understanding and Stabilizing GANs' Training Dynamics with Control Theory

1 code implementation29 Sep 2019 Kun Xu, Chongxuan Li, Jun Zhu, Bo Zhang

There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods.

Ranked #37 on Image Generation on CIFAR-10 (Inception score metric)

Image Generation L2 Regularization

Training Interpretable Convolutional Neural Networks towards Class-specific Filters

no code implementations25 Sep 2019 Haoyu Liang, Zhihao Ouyang, Hang Su, Yuyuan Zeng, Zihao He, Shu-Tao Xia, Jun Zhu, Bo Zhang

Convolutional neural networks (CNNs) have often been treated as “black-box” and successfully used in a range of tasks.

Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks

1 code implementation ICLR 2020 Tianyu Pang, Kun Xu, Jun Zhu

Our experiments on CIFAR-10 and CIFAR-100 demonstrate that MI can further improve the adversarial robustness for the models trained by mixup and its variants.

Adversarial Robustness

Cannot find the paper you are looking for? You can Submit a new open access paper.