no code implementations • ECCV 2020 • Yueru Li, Shuyu Cheng, Hang Su, Jun Zhu
Based on our investigation, we further present a new robust learning algorithm which encourages a larger gradient component in the tangent space of data manifold, suppressing the gradient leaking phenomenon consequently.
no code implementations • ICML 2020 • Kun Xu, Chongxuan Li, Jun Zhu, Bo Zhang
There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods.
no code implementations • ICML 2020 • Michael Zhu, Chang Liu, Jun Zhu
Particle-based Variational Inference methods (ParVIs), like Stein Variational Gradient Descent, are nonparametric variational inference methods that optimize a set of particles to best approximate a target distribution.
no code implementations • 30 May 2025 • Yichi Zhang, Gongwei Chen, Jun Zhu, Jia Wan
This approach overcomes dataset size limitations and enriches both the quantity and diversity of referring expressions.
1 code implementation • 27 May 2025 • Jintao Zhang, Xiaoming Xu, Jia Wei, Haofeng Huang, Pengle Zhang, Chendong Xiang, Jun Zhu, Jianfei Chen
To further accelerate SageAttention2, we propose to utilize the faster instruction of FP8 Matmul accumulated in FP16.
1 code implementation • 21 May 2025 • Chenyu Zheng, Xinyu Zhang, Rongzhen Wang, Wei Huang, Zhi Tian, Weilin Huang, Jun Zhu, Chongxuan Li
Finally, we validate the effectiveness of $\mu$P on text-to-image generation by scaling PixArt-$\alpha$ from 0. 04B to 0. 61B and MMDiT from 0. 18B to 18B.
1 code implementation • 16 May 2025 • Jintao Zhang, Jia Wei, Pengle Zhang, Xiaoming Xu, Haofeng Huang, Haoxu Wang, Kai Jiang, Jun Zhu, Jianfei Chen
Second, we pioneer low-bit attention to training tasks.
1 code implementation • 5 Apr 2025 • Yikai Wang, Guangce Liu, Xinzhou Wang, Zilong Chen, Jiafang Li, Xin Liang, Fuchun Sun, Jun Zhu
With the ability of 4D and video generation, Video4DGen offers a powerful tool for applications in virtual reality, animation, and beyond.
no code implementations • 19 Mar 2025 • Ruowen Zhao, Junliang Ye, Zhengyi Wang, Guangce Liu, YiWen Chen, Yikai Wang, Jun Zhu
Triangle meshes play a crucial role in 3D applications for efficient manipulation and rendering.
no code implementations • 15 Mar 2025 • Shentong Mo, Zehua Chen, Fan Bao, Jun Zhu
Recent works in cross-modal understanding and generation, notably through models like CLAP (Contrastive Language-Audio Pretraining) and CAVP (Contrastive Audio-Visual Pretraining), have significantly enhanced the alignment of text, video, and audio embeddings via a single contrastive loss.
no code implementations • 11 Mar 2025 • Pengle Zhang, Jia Wei, Jintao Zhang, Jun Zhu, Jianfei Chen
Transformer models have achieved remarkable success across various AI applications but face significant training costs.
no code implementations • 4 Mar 2025 • Jia Wang, Xinfeng Zhang, Gai Zhang, Jun Zhu, Lv Tang, Li Zhang
Inspired by the success of traditional video compression frameworks, which process video frame by frame and can efficiently compress long videos, we adopt this modeling strategy for INRs to decrease memory consumption, while aiming to unify the frameworks from the perspective of timeline-based autoregressive modeling.
1 code implementation • 3 Mar 2025 • Kaiwen Zheng, Yongxin Chen, Huayu Chen, Guande He, Ming-Yu Liu, Jun Zhu, Qinsheng Zhang
While likelihood-based generative models, particularly diffusion and autoregressive models, have achieved remarkable fidelity in visual generation, the maximum likelihood estimation (MLE) objective, which minimizes the forward KL divergence, inherently suffers from a mode-covering tendency that limits the generation quality under limited model capacity.
Ranked #1 on
Image Generation
on ImageNet 512x512
1 code implementation • 28 Feb 2025 • Yuxiang Chen, Haocheng Xi, Jun Zhu, Jianfei Chen
However, training with MXFP4 data format still results in significant degradation and there is a lack of systematic research on the reason.
1 code implementation • 25 Feb 2025 • Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haocheng Xi, Jun Zhu, Jianfei Chen
A universal sparse attention that guarantees both the speedup and end-to-end performance of diverse models remains elusive.
no code implementations • 21 Feb 2025 • Min Zhao, Guande He, Yixiao Chen, Hongzhou Zhu, Chongxuan Li, Jun Zhu
Recent advancements in video generation have enabled models to synthesize high-quality, minute-long videos.
no code implementations • 14 Feb 2025 • Xinning Zhou, Chengyang Ying, Yao Feng, Hang Su, Jun Zhu
We further present a practical algorithm to optimize the objective by estimating the distribution of clean observations with a pre-trained world model.
no code implementations • 11 Feb 2025 • Chengyang Ying, Huayu Chen, Xinning Zhou, Zhongkai Hao, Hang Su, Jun Zhu
Specifically, ExDM can accurately estimate the distribution of collected data in the replay buffer with the diffusion model and introduces the score-based intrinsic reward, encouraging the agent to explore less-visited states.
no code implementations • 10 Feb 2025 • Lv Tang, Jun Zhu, Xinfeng Zhang, Li Zhang, Siwei Ma, Qingming Huang
Furthermore, to enhance the capture of dynamics between frames within a sequence, we implement a dynamic frame-level adjustment (DFA).
no code implementations • 9 Feb 2025 • Yijun Yang, Lichao Wang, Xiao Yang, Lanqing Hong, Jun Zhu
It comprises three complementary attack facets: Visual Attack that exploits the multimodal nature of VLLMs to inject toxic system prompts through images; Alignment Breaking Attack that manipulates the model's alignment mechanism to prioritize the generation of contrasting responses; and Adversarial Signature that deceives content moderators by strategically placing misleading information at the end of the response.
no code implementations • 7 Feb 2025 • Yuhao Zhou, Jintao Xu, Chenglong Bao, Chao Ding, Jun Zhu
We consider the problem of finding an $\epsilon$-stationary point of a nonconvex function with a Lipschitz continuous Hessian and propose a quadratic regularized Newton method incorporating a new class of regularizers constructed from the current and previous gradients.
no code implementations • 5 Feb 2025 • Kaiwen Zheng, Guande He, Jianfei Chen, Fan Bao, Jun Zhu
Consistency distillation is a prevalent way for accelerating diffusion models adopted in consistency (trajectory) models, in which a student model is trained to traverse backward on the probability flow (PF) ordinary differential equation (ODE) trajectory determined by the teacher model.
1 code implementation • 4 Feb 2025 • Yichi Zhang, Siyuan Zhang, Yao Huang, Zeyu Xia, Zhengwei Fang, Xiao Yang, Ranjie Duan, Dong Yan, Yinpeng Dong, Jun Zhu
Ensuring the safety and harmlessness of Large Language Models (LLMs) has become equally critical as their performance in applications.
no code implementations • 31 Jan 2025 • Huanran Chen, Yinpeng Dong, Zeming Wei, Hang Su, Jun Zhu
We propose a general tight lower bound for randomized smoothing using fractional knapsack solvers or 0-1 knapsack solvers, and using them to bound the worst-case robustness of all stochastic defenses.
no code implementations • 30 Jan 2025 • Haoyu Liang, Youran Sun, Yunfeng Cai, Jun Zhu, Bo Zhang
To eradicate this security risk, we also propose defense methods against such attacks, which can correct the bias of text embeddings and improve downstream performance in a train-free manner.
1 code implementation • 26 Jan 2025 • Huayu Chen, Kai Jiang, Kaiwen Zheng, Jianfei Chen, Hang Su, Jun Zhu
It retains the same maximum likelihood objective as CFG and differs mainly in the parameterization of conditional models.
1 code implementation • 22 Jan 2025 • Jiachen Lei, Julius Berner, Jiongxiao Wang, Zhongzhu Chen, Zhongjia Ba, Kui Ren, Jun Zhu, Anima Anandkumar
For example, our method outperforms the certified accuracy of diffusion-based methods on ImageNet across all perturbation radii by 5. 3% on average, with up to 11. 6% at larger radii, while reducing inference costs by 85$\times$ on average.
no code implementations • CVPR 2025 • Han Liu, Peng Cui, Bingning Wang, WeiPeng Chen, Yupeng Zhang, Jun Zhu, Xiaolin Hu
Deep Neural Networks (DNNs) have achieved remarkable success in a variety of tasks, particularly in terms of prediction accuracy.
1 code implementation • 19 Dec 2024 • Ziteng Wang, Jianfei Chen, Jun Zhu
Sparsely activated Mixture-of-Experts (MoE) models are widely adopted to scale up model capacity without increasing the computation budget.
1 code implementation • 4 Dec 2024 • Yingtao Luo, ZHIXUN LI, Qiang Liu, Jun Zhu
In this paper, we show that the correlation between gradients and groups can help identify and improve group fairness.
no code implementations • 26 Nov 2024 • Peng Cui, Guande He, Dan Zhang, Zhijie Deng, Yinpeng Dong, Jun Zhu
Datasets collected from the open world unavoidably suffer from various forms of randomness or noiseness, leading to the ubiquity of aleatoric (data) uncertainty.
no code implementations • 25 Nov 2024 • Chuan Liu, Huanran Chen, Yichi Zhang, Yinpeng Dong, Jun Zhu
However, since prior studies usually adopt few models in the ensemble, there remains an open question of whether scaling the number of models can further improve black-box attacks.
2 code implementations • 17 Nov 2024 • Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, Jianfei Chen
Second, we propose a method to smooth $Q$, enhancing the accuracy of INT4 $QK^\top$.
no code implementations • 14 Nov 2024 • Zhengyi Wang, Jonathan Lorraine, Yikai Wang, Hang Su, Jun Zhu, Sanja Fidler, Xiaohui Zeng
This work explores expanding the capabilities of large language models (LLMs) pretrained on text to generate 3D meshes within a unified model.
no code implementations • 6 Nov 2024 • Fengxiang Wang, Ranjie Duan, Peng Xiao, Xiaojun Jia, Shiji Zhao, Cheng Wei, Yuefeng Chen, Chongwen Wang, Jialing Tao, Hang Su, Jun Zhu, Hui Xue
Previous works mainly focus on jailbreak in single-round dialogue, overlooking the potential jailbreak risks in multi-round dialogues, which are a vital way humans interact with and extract information from LLMs.
no code implementations • 4 Nov 2024 • Hengkai Tan, Xuezhou Xu, Chengyang Ying, Xinyi Mao, Songming Liu, Xingxing Zhang, Hang Su, Jun Zhu
To overcome these challenges, we then focus on state-based policy generalization and present \textbf{ManiBox}, a novel bounding-box-guided manipulation method built on a simulation-based teacher-student framework.
no code implementations • 31 Oct 2024 • Yakun Xie, Suning Liu, Hongyu Chen, Shaohan Cao, Huixin Zhang, Dejun Feng, Qian Wan, Jun Zhu, Qing Zhu
Despite significant advancements in salient object detection(SOD) in optical remote sensing images(ORSI), challenges persist due to the intricate edge structures of ORSIs and the complexity of their contextual relationships.
2 code implementations • 30 Oct 2024 • Guande He, Kaiwen Zheng, Jianfei Chen, Fan Bao, Jun Zhu
Recently, diffusion denoising bridge models (DDBMs), a new formulation of generative modeling that builds stochastic processes between fixed data endpoints based on a reference diffusion process, have achieved empirical success across tasks with coupled data distribution, such as image-to-image translation.
no code implementations • 20 Oct 2024 • Yuji Wang, Zehua Chen, Xiaoyu Chen, Jun Zhu, Jianfei Chen
By formulating I2V synthesis as a frames-to-frames generation task and modelling it with a data-to-data process, we fully exploit the information in input image and facilitate the generative model to learn the image animation process.
2 code implementations • 12 Oct 2024 • Huayu Chen, Hang Su, Peize Sun, Jun Zhu
Motivated by language model alignment methods, we propose \textit{Condition Contrastive Alignment} (CCA) to facilitate guidance-free AR visual generation with high performance and analyze its theoretical connection with guided sampling methods.
1 code implementation • 10 Oct 2024 • Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, Jun Zhu
Bimanual manipulation is essential in robotics, yet developing foundation models is extremely challenging due to the inherent complexity of coordinating two robot arms (leading to multi-modal action distributions) and the scarcity of training data.
no code implementations • 7 Oct 2024 • Bingrui Li, Wei Huang, Andi Han, Zhanpeng Zhou, Taiji Suzuki, Jun Zhu, Jianfei Chen
We also show that Adam behaves similarly to SignGD in terms of both optimization and generalization in this setting.
2 code implementations • 3 Oct 2024 • Jintao Zhang, Jia Wei, Haofeng Huang, Pengle Zhang, Jun Zhu, Jianfei Chen
Although quantization has proven to be an effective method for accelerating model inference, existing quantization methods primarily focus on optimizing the linear layer.
2 code implementations • 13 Sep 2024 • Yuezhou Hu, Jun Zhu, Jianfei Chen
Training deep neural networks (DNNs) is costly.
no code implementations • 4 Sep 2024 • Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming-Yu Liu, Jun Zhu, Qinsheng Zhang
Masked diffusion models (MDMs) have emerged as a popular research topic for generative modeling of discrete data, thanks to their superior performance over other discrete diffusion models, and are rivaling the auto-regressive models (ARMs) for language modeling tasks.
1 code implementation • 5 Aug 2024 • YiWen Chen, Yikai Wang, Yihao Luo, Zhengyi Wang, Zilong Chen, Jun Zhu, Chi Zhang, Guosheng Lin
Meshes are the de facto 3D representation in the industry but are labor-intensive to produce.
1 code implementation • 30 Jul 2024 • Weiyu Huang, Yuezhou Hu, Guohao Jian, Jun Zhu, Jianfei Chen
However, these methods often suffer from considerable performance degradation on complex language understanding tasks, raising concerns about the feasibility of pruning in LLMs.
1 code implementation • 12 Jul 2024 • Huayu Chen, Kaiwen Zheng, Hang Su, Jun Zhu
Drawing upon recent advances in language model alignment, we formulate offline Reinforcement Learning as a two-stage optimization problem: First pretraining expressive generative policies on reward-free behavior datasets, then fine-tuning these policies to align with task-specific annotations like Q-values.
no code implementations • 8 Jul 2024 • Yibo Miao, Yifan Zhu, Yinpeng Dong, Lijia Yu, Jun Zhu, Xiao-Shan Gao
The recent development of Sora leads to a new era in text-to-video (T2V) generation.
1 code implementation • 7 Jul 2024 • Liyuan Wang, Jingyi Xie, Xingxing Zhang, Hang Su, Jun Zhu
The deployment of pre-trained models (PTMs) has greatly advanced the field of continual learning (CL), enabling positive knowledge transfer and resilience to catastrophic forgetting.
no code implementations • 22 Jun 2024 • Min Zhao, Hongzhou Zhu, Chendong Xiang, Kaiwen Zheng, Chongxuan Li, Jun Zhu
We attribute this to the issue called conditional image leakage, where the image-to-video diffusion models (I2V-DMs) tend to over-rely on the conditional image at large time steps.
no code implementations • 11 Jun 2024 • Yichi Zhang, Yao Huang, Yitong Sun, Chang Liu, Zhe Zhao, Zhengwei Fang, Yifan Wang, Huanran Chen, Xiao Yang, Xingxing Wei, Hang Su, Yinpeng Dong, Jun Zhu
Despite the superior capabilities of Multimodal Large Language Models (MLLMs) across diverse tasks, they still face significant trustworthiness challenges.
no code implementations • 2 Jun 2024 • Wenqiang Sun, Zhengyi Wang, Shuo Chen, Yikai Wang, Zilong Chen, Jun Zhu, Jun Zhang
We first analyze the role of triplanes in feed-forward methods and find that the inconsistent multi-view images introduce high-frequency artifacts on triplanes, leading to low-quality 3D meshes.
no code implementations • 30 May 2024 • Hengkai Tan, Songming Liu, Kai Ma, Chengyang Ying, Xingxing Zhang, Hang Su, Jun Zhu
In this paper, we propose to investigate the task from a new perspective of the frequency domain.
no code implementations • 30 May 2024 • Han Liu, Peng Cui, Bingning Wang, Jun Zhu, Xiaolin Hu
Deep Neural Networks (DNNs) have achieved remarkable success in a variety of tasks, especially when it comes to prediction accuracy.
1 code implementation • 29 May 2024 • Shuyu Cheng, Yibo Miao, Yinpeng Dong, Xiao Yang, Xiao-Shan Gao, Jun Zhu
In this paper, we propose a Prior-guided Bayesian Optimization (P-BO) algorithm that leverages the surrogate model as a global function prior in black-box adversarial attacks.
1 code implementation • 27 May 2024 • Yikai Wang, Xinzhou Wang, Zilong Chen, Zhengyi Wang, Fuchun Sun, Jun Zhu
Vidu4D also contains a novel initialization state that provides a proper start for the warping fields in DGS.
no code implementations • 27 May 2024 • Haohan Weng, Yikai Wang, Tong Zhang, C. L. Philip Chen, Jun Zhu
Generating compact and sharply detailed 3D meshes poses a significant challenge for current 3D generative models.
1 code implementation • 27 May 2024 • Chenyu Zheng, Wei Huang, Rongzhen Wang, Guoqiang Wu, Jun Zhu, Chongxuan Li
First, under a certain condition of data distribution, we prove that an autoregressively trained transformer learns $W$ by implementing one step of gradient descent to minimize an ordinary least squares (OLS) problem in-context.
1 code implementation • 24 May 2024 • Kaiwen Zheng, Guande He, Jianfei Chen, Fan Bao, Jun Zhu
In this work, we take the first step in fast sampling of DDBMs without extra training, motivated by the well-established recipes in diffusion models.
1 code implementation • 23 May 2024 • Chengyang Ying, Zhongkai Hao, Xinning Zhou, Xuezhou Xu, Hang Su, Xingxing Zhang, Jun Zhu
Designing generalizable agents capable of adapting to diverse embodiments has achieved significant attention in Reinforcement Learning (RL), which is critical for deploying RL agents in various real-world applications.
no code implementations • 7 May 2024 • Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, Jun Zhu
We introduce Vidu, a high-performance text-to-video generator that is capable of producing 1080p videos up to 16 seconds in a single generation.
1 code implementation • 30 Apr 2024 • Luxi Chen, Zhengyi Wang, Zihan Zhou, Tingting Gao, Hang Su, Jun Zhu, Chongxuan Li
Optimization-based approaches, such as score distillation sampling (SDS), show promise in zero-shot 3D generation but suffer from low efficiency, primarily due to the high number of function evaluations (NFEs) required for each sample and the limitation of optimization confined to latent space.
3 code implementations • 27 Apr 2024 • Xiao Wang, Qian Zhu, Jiandong Jin, Jun Zhu, Futian Wang, Bo Jiang, YaoWei Wang, Yonghong Tian
Specifically, we formulate the video-based PAR as a vision-language fusion problem and adopt a pre-trained foundation model CLIP to extract the visual features.
1 code implementation • CVPR 2024 • Yichi Zhang, Yinpeng Dong, Siyuan Zhang, Tianzan Min, Hang Su, Jun Zhu
To achieve this, we propose Transferable Visual Prompting (TVP), a simple and effective approach to generate visual prompts that can transfer to different models and improve their performance on downstream tasks after trained on only one model.
no code implementations • 16 Apr 2024 • Kafeng Wang, Jianfei Chen, He Li, Zhenpeng Mi, Jun Zhu
Diffusion models represent a powerful family of generative models widely used for image and video generation.
2 code implementations • 2 Apr 2024 • Yuezhou Hu, Kang Zhao, Weiyu Huang, Jianfei Chen, Jun Zhu
Utilizing this metric, we propose three techniques to preserve accuracy: to modify the sparse-refined straight-through estimator by applying the masked decay term on gradients, to determine a feasible decay factor in warm-up stage, and to enhance the model's quality by a dense fine-tuning procedure near the end of pre-training.
1 code implementation • 1 Apr 2024 • Ruowen Zhao, Zhengyi Wang, Yikai Wang, Zihan Zhou, Jun Zhu
However, since directly reconstructing triangle meshes from multi-view images is challenging, most methodologies opt to an implicit representation (such as NeRF) during the sparse-view reconstruction and acquire the target mesh by a post-processing extraction.
no code implementations • 31 Mar 2024 • Lingxuan Wu, Xiao Yang, Yinpeng Dong, Liuwei Xie, Hang Su, Jun Zhu
The vulnerability of deep neural networks to adversarial patches has motivated numerous defense strategies for boosting model robustness.
no code implementations • 21 Mar 2024 • Junliang Ye, Fangfu Liu, Qixiu Li, Zhengyi Wang, Yikai Wang, Xinzhou Wang, Yueqi Duan, Jun Zhu
Building upon the 3D reward model, we finally perform theoretical analysis and present the Reward3D Feedback Learning (DreamFL), a direct tuning algorithm to optimize the multi-view diffusion models with a redefined scorer.
1 code implementation • 19 Mar 2024 • Haocheng Xi, Yuxiang Chen, Kang Zhao, Kai Jun Teh, Jianfei Chen, Jun Zhu
Pretraining transformers are generally time-consuming.
no code implementations • 8 Mar 2024 • Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, Jun Zhu
In this work, we present the Convolutional Reconstruction Model (CRM), a high-fidelity feed-forward single image-to-3D generative model.
2 code implementations • 6 Mar 2024 • Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, Chengyang Ying, Hang Su, Anima Anandkumar, Jian Song, Jun Zhu
Pre-training has been investigated to improve the efficiency and performance of training neural operators in data-scarce settings.
1 code implementation • 27 Feb 2024 • Ziteng Wang, Jianfei Chen, Jun Zhu
On all the tasks, VCAS can preserve the original training loss trajectory and validation accuracy with an up to 73. 87% FLOPs reduction of BP and 49. 58% FLOPs reduction of the whole training process.
1 code implementation • 26 Feb 2024 • Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang, Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, Hong Chen
To address the limitations, we introduce CodeS, a series of pre-trained language models with parameters ranging from 1B to 15B, specifically designed for the text-to-SQL task.
no code implementations • 26 Feb 2024 • Tianjiao Luo, Tim Pearce, Huayu Chen, Jianfei Chen, Jun Zhu
Generative Adversarial Imitation Learning (GAIL) trains a generative policy to mimic a demonstrator.
no code implementations • 23 Feb 2024 • Yu Tian, Xiao Yang, Yinpeng Dong, Heming Yang, Hang Su, Jun Zhu
It allows users to design specific prompts to generate realistic images through some black-box APIs.
3 code implementations • 8 Feb 2024 • Huayu Chen, Guande He, Lifan Yuan, Ganqu Cui, Hang Su, Jun Zhu
We evaluate our methods in both reward and preference settings with Mistral-8*7B and 7B models.
1 code implementation • 4 Feb 2024 • Huanran Chen, Yinpeng Dong, Shitong Shao, Zhongkai Hao, Xiao Yang, Hang Su, Jun Zhu
Experimental results show the superior certified robustness of these Noised Diffusion Classifiers (NDCs).
no code implementations • 1 Feb 2024 • Songming Liu, Chang Su, Jiachen Yao, Zhongkai Hao, Hang Su, Youjia Wu, Jun Zhu
Physics-informed neural networks (PINNs) have shown promise in solving various partial differential equations (PDEs).
2 code implementations • 1 Feb 2024 • Haozhe Ji, Cheng Lu, Yilin Niu, Pei Ke, Hongning Wang, Jun Zhu, Jie Tang, Minlie Huang
This leads to the same mode-seeking solution, while enables efficient optimization by circumventing the complexities of RL.
1 code implementation • 13 Dec 2023 • Ziqi Yuan, Liyuan Wang, Wenbo Ding, Xingxing Zhang, Jiachen Zhong, Jianyong Ai, Jianmin Li, Jun Zhu
A commonly-used strategy for supervised IOD is to encourage the current model (as a student) to mimic the behavior of the old model (as a teacher), but it generally fails in SSIOD because a dominant number of object instances from old and new classes are coexisting and unlabelled, with the teacher only recognizing a fraction of them.
no code implementations • 6 Dec 2023 • Zehua Chen, Guande He, Kaiwen Zheng, Xu Tan, Jun Zhu
Specifically, we leverage the latent representation obtained from text input as our prior, and build a fully tractable Schrodinger bridge between it and the ground-truth mel-spectrogram, leading to a data-to-data process.
no code implementations • 1 Dec 2023 • Kehui Yao, Jingyi Huang, Jun Zhu
Effective management of environmental resources and agricultural sustainability heavily depends on accurate soil moisture data.
1 code implementation • 9 Nov 2023 • Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang, Jianfeng Gao, Chunyuan Li
LLaVA-Plus is a general-purpose multimodal assistant that expands the capabilities of large multimodal models.
Ranked #1 on
LMM real-life tasks
on Leaderboard
1 code implementation • 6 Nov 2023 • Jianhui Li, Shilong Liu, Zidong Liu, Yikai Wang, Kaiwen Zheng, Jinghui Xu, Jianmin Li, Jun Zhu
With the success of Neural Radiance Field (NeRF) in 3D-aware portrait editing, a variety of works have achieved promising results regarding both quality and 3D consistency.
1 code implementation • 21 Oct 2023 • Liyuan Wang, Jingyi Xie, Xingxing Zhang, Hang Su, Jun Zhu
In this work, we present a general framework for continual learning of sequentially arrived tasks with the use of pre-training, which has emerged as a promising direction for artificial intelligence systems to accommodate real-world dynamics.
2 code implementations • NeurIPS 2023 • Kaiwen Zheng, Cheng Lu, Jianfei Chen, Jun Zhu
In this work, we propose a novel formulation towards the optimal parameterization during sampling that minimizes the first-order discretization error of the ODE solution.
no code implementations • 18 Oct 2023 • Guande He, Peng Cui, Jianfei Chen, WenBo Hu, Jun Zhu
Despite the significant progress made in practical applications of aligned language models (LMs), they tend to be overconfident in output answers compared to the corresponding pre-trained LMs.
1 code implementation • NeurIPS 2023 • Yilin Lyu, Liyuan Wang, Xingxing Zhang, Zicheng Sun, Hang Su, Jun Zhu, Liping Jing
Continual learning entails learning a sequence of tasks and balancing their knowledge appropriately.
1 code implementation • NeurIPS 2023 • Liyuan Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, Jun Zhu
Following these empirical and theoretical insights, we propose Hierarchical Decomposition (HiDe-)Prompt, an innovative approach that explicitly optimizes the hierarchical components with an ensemble of task-specific prompts and statistics of both uninstructed and instructed representations, further with the coordination of a contrastive regularization strategy.
1 code implementation • 11 Oct 2023 • Huayu Chen, Cheng Lu, Zhengyi Wang, Hang Su, Jun Zhu
Recent developments in offline reinforcement learning have uncovered the immense potential of diffusion modeling, which excels at representing heterogeneous behavior policies.
1 code implementation • 21 Sep 2023 • Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, Jun Zhu
By attacking white-box surrogate vision encoders or MLLMs, the generated adversarial examples can mislead Bard to output wrong image descriptions with a 22% success rate based solely on the transferability.
1 code implementation • 5 Sep 2023 • Xu Si, Xinming Wu, Hanlin Sheng, Jun Zhu, Zefeng Li
Training specific deep learning models for particular tasks is common across various domains within seismology.
1 code implementation • 29 Aug 2023 • Liyuan Wang, Xingxing Zhang, Qian Li, Mingtian Zhang, Hang Su, Jun Zhu, Yi Zhong
Continual learning aims to empower artificial intelligence (AI) with strong adaptability to the real world.
1 code implementation • 29 Aug 2023 • Feng Zhou, Quyu Kong, Zhijie Deng, Fengxiang He, Peng Cui, Jun Zhu
This paper presents a novel extension of multi-task Gaussian Cox processes for modeling multiple heterogeneous correlated tasks jointly, e. g., classification and regression, via multi-output Gaussian processes (MOGP).
no code implementations • 19 Jul 2023 • Ye Ouyang, Yaqin Zhang, Peng Wang, Yunxin Liu, Wen Qiao, Jun Zhu, Yang Liu, Feng Zhang, Shuling Wang, Xidong Wang
6G is the next-generation intelligent and integrated digital information infrastructure, characterized by ubiquitous interconnection, native intelligence, multi-dimensional perception, global coverage, green and low-carbon, native network security, etc.
no code implementations • 24 Jun 2023 • Xu Si, Xinming Wu, Zefeng Li, Shenghou Wang, Jun Zhu
Overall, our study provides for the first time a prototype self-consistent all-in-one system of simultaneous seismic phase picking, association, and location, which has the potential for next-generation autonomous earthquake monitoring.
1 code implementation • NeurIPS 2023 • Haocheng Xi, Changhao Li, Jianfei Chen, Jun Zhu
To achieve this, we carefully analyze the specific structures of activation and gradients in transformers to propose dedicated quantizers for them.
no code implementations • 18 Jun 2023 • Tianjiao Luo, Ziyu Zhu, Jianfei Chen, Jun Zhu
We theoretically prove that the training process of DiracGANs-BMC is globally exponential stable and derive bounds on the rate of convergence.
2 code implementations • 15 Jun 2023 • Zhongkai Hao, Jiachen Yao, Chang Su, Hang Su, Ziao Wang, Fanzhi Lu, Zeyu Xia, Yichi Zhang, Songming Liu, Lu Lu, Jun Zhu
In addition to providing a standardized means of assessing performance, PINNacle also offers an in-depth analysis to guide future research, particularly in areas such as domain decomposition methods and loss reweighting for handling multi-scale problems and complex geometry.
no code implementations • 5 Jun 2023 • Jiachen Yao, Chang Su, Zhongkai Hao, Songming Liu, Hang Su, Jun Zhu
Physics-informed Neural Networks (PINNs) have recently achieved remarkable progress in solving Partial Differential Equations (PDEs) in various fields by minimizing a weighted sum of PDE loss and boundary loss.
1 code implementation • 30 May 2023 • Songming Liu, Zhongkai Hao, Chengyang Ying, Hang Su, Ze Cheng, Jun Zhu
The neural operator has emerged as a powerful tool in learning mappings between function spaces in PDEs.
1 code implementation • 30 May 2023 • Guande He, Jianfei Chen, Jun Zhu
In light of these observations, we evaluate the calibration of several methods that preserve pre-trained features and show that preserving pre-trained features can improve the calibration of fine-tuned language models.
1 code implementation • 28 May 2023 • Zhanhao Hu, Jun Zhu, Bo Zhang, Xiaolin Hu
Recent works found that deep neural networks (DNNs) can be fooled by adversarial examples, which are crafted by adding adversarial noise on clean inputs.
1 code implementation • 26 May 2023 • Min Zhao, Rongzhen Wang, Fan Bao, Chongxuan Li, Jun Zhu
This paper presents \emph{ControlVideo} for text-driven video editing -- generating a video that aligns with a given text while preserving the structure of the source video.
2 code implementations • NeurIPS 2023 • Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, Jun Zhu
In comparison, VSD works well with various CFG weights as ancestral sampling from diffusion models and simultaneously improves the diversity and sample quality with a common CFG weight (i. e., $7. 5$).
3 code implementations • 24 May 2023 • Huanran Chen, Yinpeng Dong, Zhengyi Wang, Xiao Yang, Chengqi Duan, Hang Su, Jun Zhu
As RDC does not require training on particular adversarial attacks, we demonstrate that it is more generalizable to defend against multiple unseen threats.
Ranked #2 on
Adversarial Defense
on CIFAR-10
1 code implementation • 6 May 2023 • Kaiwen Zheng, Cheng Lu, Jianfei Chen, Jun Zhu
The probability flow ordinary differential equation (ODE) of diffusion models (i. e., diffusion ODEs) is a particular case of continuous normalizing flows (CNFs), which enables deterministic inference and exact likelihood evaluation.
Ranked #1 on
Density Estimation
on CIFAR-10
3 code implementations • 25 Apr 2023 • Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, Jun Zhu
The main challenge for this setting is that the intermediate guidance during the diffusion sampling procedure, which is jointly defined by the sampling distribution and the energy function, is unknown and is hard to estimate.
1 code implementation • 20 Apr 2023 • Jun Zhu, Jiandong Jin, Zihan Yang, Xiaohao Wu, Xiao Wang
The averaged visual tokens and text tokens are concatenated and fed into a fusion Transformer for multi-modal interactive learning.
1 code implementation • CVPR 2023 • Jianhui Li, Jianmin Li, Haoji Zhang, Shilong Liu, Zhengyi Wang, Zihao Xiao, Kaiwen Zheng, Jun Zhu
As for imprecise image editing, we attribute the problem to the gap between the latent space of real images and that of generated images.
2 code implementations • ICCV 2023 • Shilong Liu, Tianhe Ren, Jiayu Chen, Zhaoyang Zeng, Hao Zhang, Feng Li, Hongyang Li, Jun Huang, Hang Su, Jun Zhu, Lei Zhang
We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR.
1 code implementation • 31 Mar 2023 • Chendong Xiang, Fan Bao, Chongxuan Li, Hang Su, Jun Zhu
Large-scale diffusion models like Stable Diffusion are powerful and find various real-world applications while customizing such models by fine-tuning is both memory and time inefficient.
1 code implementation • CVPR 2023 • Xiao Yang, Chang Liu, Longlong Xu, Yikai Wang, Yinpeng Dong, Ning Chen, Hang Su, Jun Zhu
The goal of this work is to develop a more reliable technique that can carry out an end-to-end evaluation of adversarial robustness for commercial systems.
no code implementations • 20 Mar 2023 • Yinpeng Dong, Caixin Kang, Jinlai Zhang, Zijian Zhu, Yikai Wang, Xiao Yang, Hang Su, Xingxing Wei, Jun Zhu
3D object detection is an important task in autonomous driving to perceive the surroundings.
2 code implementations • 16 Mar 2023 • Huanran Chen, Yichi Zhang, Yinpeng Dong, Xiao Yang, Hang Su, Jun Zhu
It is widely recognized that deep learning models lack robustness to adversarial examples.
3 code implementations • 12 Mar 2023 • Fan Bao, Shen Nie, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yue, Yue Cao, Hang Su, Jun Zhu
Inspired by the unified view, UniDiffuser learns all distributions simultaneously with a minimal modification to the original diffusion model -- perturbs data in all modalities instead of a single modality, inputs individual timesteps in different modalities, and predicts the noise of all modalities instead of a single modality.
10 code implementations • 9 Mar 2023 • Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang
To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion.
Ranked #2 on
Zero Shot Segmentation
on Segmentation in the Wild
no code implementations • 9 Mar 2023 • Chengyang Ying, Zhongkai Hao, Xinning Zhou, Hang Su, Songming Liu, Dong Yan, Jun Zhu
Extensive experiments in both image-based and state-based tasks show that TAD can significantly improve the performance of handling different tasks simultaneously, especially for those with high TDR, and display a strong generalization ability to unseen tasks.
1 code implementation • 6 Mar 2023 • Dejun Feng, Hongyu Chen, Suning Liu, Ziyang Liao, Xingyu Shen, Yakun Xie, Jun Zhu
Finally, to obtain more complete saliency maps, we consider the uniqueness of the last layer of the decoder for the first time and propose the adaptive feedback refinement (AFR) module, which further refines feature representation and eliminates differences between features through a unique feedback mechanism.
no code implementations • 2 Mar 2023 • Shuai Xiao, Le Guo, Zaifan Jiang, Lei Lv, Yuanbo Chen, Jun Zhu, Shuang Yang
Furthermore, we show that the dual problem can be solved by policy learning, with the optimal dual variable being found efficiently via bisection search (i. e., by taking advantage of the monotonicity).
no code implementations • 1 Mar 2023 • Yichi Zhang, Zijian Zhu, Hang Su, Jun Zhu, Shibao Zheng, Yuan He, Hui Xue
In this paper, we propose Adversarial Semantic Contour (ASC), an MAP estimate of a Bayesian formulation of sparse attack with a deceived prior of object contour.
2 code implementations • 28 Feb 2023 • Zhongkai Hao, Zhengyi Wang, Hang Su, Chengyang Ying, Yinpeng Dong, Songming Liu, Ze Cheng, Jian Song, Jun Zhu
However, there are several challenges for learning operators in practical applications like the irregular mesh, multiple input functions, and complexity of the PDEs' solution.
no code implementations • 28 Feb 2023 • Chang Liu, Yinpeng Dong, Wenzhao Xiang, Xiao Yang, Hang Su, Jun Zhu, Yuefeng Chen, Yuan He, Hui Xue, Shibao Zheng
In our benchmark, we evaluate the robustness of 55 typical deep learning models on ImageNet with diverse architectures (e. g., CNNs, Transformers) and learning algorithms (e. g., normal supervised training, pre-training, adversarial training) under numerous adversarial attacks and out-of-distribution (OOD) datasets.
3 code implementations • NeurIPS 2023 • Zebin You, Yong Zhong, Fan Bao, Jiacheng Sun, Chongxuan Li, Jun Zhu
In an effort to further advance semi-supervised generative and classification tasks, we propose a simple yet effective training strategy called dual pseudo training (DPT), built upon strong semi-supervised learners and diffusion models.
no code implementations • 10 Feb 2023 • Peng Cui, Yang Yue, Zhijie Deng, Jun Zhu
Deep neural networks (DNNs) have achieved remarkable success in a variety of computer vision tasks, where massive labeled images are routinely required for model optimization.
1 code implementation • 5 Feb 2023 • Chenyu Zheng, Guoqiang Wu, Fan Bao, Yue Cao, Chongxuan Li, Jun Zhu
Theoretically, the paper considers the surrogate loss instead of the zero-one loss in analyses and generalizes the classical results from binary cases to multiclass ones.
1 code implementation • 31 Jan 2023 • Liyuan Wang, Xingxing Zhang, Hang Su, Jun Zhu
To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime.
no code implementations • 10 Jan 2023 • Kexuan Li, Jun Zhu, Anthony R. Ives, Volker C. Radeloff, Fangfang Wang
To be specific, we use a sparsely connected deep neural network with rectified linear unit (ReLU) activation function to estimate the unknown regression function that describes the relationship between response and covariates in the presence of spatial dependence.
1 code implementation • CVPR 2023 • Yinpeng Dong, Caixin Kang, Jinlai Zhang, Zijian Zhu, Yikai Wang, Xiao Yang, Hang Su, Xingxing Wei, Jun Zhu
3D object detection is an important task in autonomous driving to perceive the surroundings.
no code implementations • 1 Dec 2022 • Fan Bao, Chongxuan Li, Jiacheng Sun, Jun Zhu
Extensive empirical evidence demonstrates that conditional generative models are easier to train and perform better than unconditional ones by exploiting the labels of data.
1 code implementation • 28 Nov 2022 • Shilong Liu, Yaoyuan Liang, Feng Li, Shijia Huang, Hao Zhang, Hang Su, Jun Zhu, Lei Zhang
As phrase extraction can be regarded as a $1$D text segmentation problem, we formulate PEG as a dual detection problem and propose a novel DQ-DETR model, which introduces dual queries to probe different features from image and text for object prediction and phrase mask prediction.
Ranked #11 on
Referring Expression Comprehension
on RefCOCO
1 code implementation • 15 Nov 2022 • Zhongkai Hao, Songming Liu, Yichi Zhang, Chengyang Ying, Yao Feng, Hang Su, Jun Zhu
Recent work shows that it provides potential benefits for machine learning models by incorporating the physical prior and collected data, which makes the intersection of machine learning and physics become a prevailing paradigm.
1 code implementation • 2 Nov 2022 • Jinali Zhang, Yinpeng Dong, Jun Zhu, Jihong Zhu, Minchi Kuang, Xiaming Yuan
Extensive experiments show that the SS attack proposed in this paper can be seamlessly combined with the existing state-of-the-art (SOTA) 3D point cloud attack methods to form more powerful attack methods, and the SS attack improves the transferability over 3. 6 times compare to the baseline.
no code implementations • 2 Nov 2022 • Yao Feng, Yuhong Jiang, Hang Su, Dong Yan, Jun Zhu
Model-based reinforcement learning usually suffers from a high sample complexity in training the world model, especially for the environments with complex dynamics.
Model-based Reinforcement Learning
reinforcement-learning
+2
3 code implementations • 2 Nov 2022 • Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu
The commonly-used fast sampler for guided sampling is DDIM, a first-order diffusion ODE solver that generally needs 100 to 250 steps for high-quality samples.
no code implementations • 29 Oct 2022 • Ziyu Wang, Yucen Luo, Yueru Li, Jun Zhu, Bernhard Schölkopf
For nonparametric conditional moment models, efficient estimation often relies on preimposed conditions on various measures of ill-posedness of the hypothesis space, which are hard to validate when flexible models are used.
no code implementations • 27 Oct 2022 • Yibo Miao, Yinpeng Dong, Jun Zhu, Xiao-Shan Gao
For naturalness, we constrain the adversarial example to be $\epsilon$-isometric to the original one by adopting the Gaussian curvature as a surrogate metric guaranteed by a theoretical analysis.
1 code implementation • 23 Oct 2022 • Zhijie Deng, Feng Zhou, Jun Zhu
Laplace approximation (LA) and its linearized variant (LLA) enable effortless adaptation of pretrained deep neural networks to Bayesian neural networks.
1 code implementation • 23 Oct 2022 • Zhijie Deng, Jiaxin Shi, Hao Zhang, Peng Cui, Cewu Lu, Jun Zhu
Unlike prior spectral methods such as Laplacian Eigenmap that operate in a nonparametric manner, Neural Eigenmap leverages NeuralEF to parametrically model eigenfunctions using a neural network.
1 code implementation • 8 Oct 2022 • Yinpeng Dong, Shouwei Ruan, Hang Su, Caixin Kang, Xingxing Wei, Jun Zhu
Recent studies have demonstrated that visual recognition models lack robustness to distribution shift.
1 code implementation • 6 Oct 2022 • Songming Liu, Zhongkai Hao, Chengyang Ying, Hang Su, Jun Zhu, Ze Cheng
We present a unified hard-constraint framework for solving geometrically complex PDEs with neural networks, where the most commonly used Dirichlet, Neumann, and Robin boundary conditions (BCs) are considered.
1 code implementation • 30 Sep 2022 • Jianyun Xu, Zhenwei Miao, Da Zhang, Hongyu Pan, Kaixuan Liu, Peihan Hao, Jun Zhu, Zhengyang Sun, Hongmin Li, Xin Zhan
By employing INT on CenterPoint, we can get around 7% (Waymo) and 15% (nuScenes) performance boost with only 2~4ms latency overhead, and currently SOTA on the Waymo 3D Detection leaderboard.
2 code implementations • 30 Sep 2022 • Fan Bao, Min Zhao, Zhongkai Hao, Peiyao Li, Chongxuan Li, Jun Zhu
Inverse molecular design is critical in material science and drug discovery, where the generated molecules should satisfy certain desirable properties.
1 code implementation • 29 Sep 2022 • Huayu Chen, Cheng Lu, Chengyang Ying, Hang Su, Jun Zhu
To address this problem, we adopt a generative approach by decoupling the learned policy into two parts: an expressive generative behavior model and an action evaluation model.
3 code implementations • CVPR 2023 • Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, Jun Zhu
We evaluate U-ViT in unconditional and class-conditional image generation, as well as text-to-image generation tasks, where U-ViT is comparable if not superior to a CNN-based U-Net of a similar size.
Ranked #4 on
Text-to-Image Generation
on MS COCO
no code implementations • 15 Sep 2022 • Zhongkai Hao, Chengyang Ying, Hang Su, Jun Zhu, Jian Song, Ze Cheng
In this paper, we present a novel bi-level optimization framework to resolve the challenge by decoupling the optimization of the targets and constraints.
1 code implementation • 15 Sep 2022 • Chengyang Ying, Zhongkai Hao, Xinning Zhou, Hang Su, Dong Yan, Jun Zhu
In this paper, we reveal that the instability is also related to a new notion of Reuse Bias of IS -- the bias in off-policy evaluation caused by the reuse of the replay buffer for evaluation and optimization.
no code implementations • 11 Aug 2022 • Qihan Guo, Siwei Wang, Jun Zhu
We study an extension of standard bandit problem in which there are R layers of experts.
no code implementations • 3 Aug 2022 • Wenkai Li, Cheng Feng, Ting Chen, Jun Zhu
In this work, to tackle this important challenge, we firstly investigate the robustness of commonly used deep TSAD methods with contaminated training data which provides a guideline for applying these methods when the provided training data are not guaranteed to be anomaly-free.
1 code implementation • 14 Jul 2022 • Min Zhao, Fan Bao, Chongxuan Li, Jun Zhu
Further, we provide an alternative explanation of the EGSDE as a product of experts, where each of the three experts (corresponding to the SDE and two feature extractors) solely contributes to faithfulness or realism.
Ranked #1 on
Image-to-Image Translation
on AFHQ (Wild to Dog)
2 code implementations • 13 Jul 2022 • Liyuan Wang, Xingxing Zhang, Qian Li, Jun Zhu, Yi Zhong
Continual learning requires incremental compatibility with a sequence of tasks.
2 code implementations • 12 Jul 2022 • Wentse Chen, Shiyu Huang, Yuan Chiang, Tim Pearce, Wei-Wei Tu, Ting Chen, Jun Zhu
We propose Diversity-Guided Policy Optimization (DGPO), an on-policy algorithm that discovers multiple strategies for solving a given task.
no code implementations • 18 Jun 2022 • Siwei Wang, Jun Zhu
To make the algorithm efficient, they usually use the sum of upper confidence bounds within arm set $S$ to represent the upper confidence bound of $S$, which can be much larger than the tight upper confidence bound of $S$ and leads to a much higher complexity than necessary, since the empirical means of different arms in $S$ are independent.
1 code implementation • 17 Jun 2022 • Siyu Wang, Jianfei Chen, Chongxuan Li, Jun Zhu, Bo Zhang
In this work, we propose Integer-only Discrete Flows (IODF), an efficient neural compressor with integer-only arithmetic.
1 code implementation • 16 Jun 2022 • Cheng Lu, Kaiwen Zheng, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu
To fill up this gap, we show that the negative likelihood of the ODE can be bounded by controlling the first, second, and third-order score matching errors; and we further present a novel high-order denoising score matching method to enable maximum likelihood training of score-based diffusion ODEs.
1 code implementation • 15 Jun 2022 • Fan Bao, Chongxuan Li, Jiacheng Sun, Jun Zhu, Bo Zhang
Thus, the generation performance on a subset of timesteps is crucial, which is greatly influenced by the covariance design in DPMs.
1 code implementation • 9 Jun 2022 • Chengyang Ying, Xinning Zhou, Hang Su, Dong Yan, Ning Chen, Jun Zhu
Though deep reinforcement learning (DRL) has obtained substantial success, it may encounter catastrophic failures due to the intrinsic uncertainty of both transition and observation.
no code implementations • 9 Jun 2022 • Zhongkai Hao, Chengyang Ying, Yinpeng Dong, Hang Su, Jun Zhu, Jian Song
Under the GSmooth framework, we present a scalable algorithm that uses a surrogate image-to-image network to approximate the complex transformation.
no code implementations • 9 Jun 2022 • Weikai Yang, Xi Ye, Xingxing Zhang, Lanxi Xiao, Jiazhi Xia, Zhongyuan Wang, Jun Zhu, Hanspeter Pfister, Shixia Liu
The base learners and labeled samples (shots) in an ensemble few-shot classifier greatly affect the model performance.
4 code implementations • 2 Jun 2022 • Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, Jun Zhu
In this work, we propose an exact formulation of the solution of diffusion ODEs.
1 code implementation • 28 May 2022 • Shih-Han Chan, Yinpeng Dong, Jun Zhu, Xiaolu Zhang, Jun Zhou
We propose four kinds of backdoor attacks for object detection task: 1) Object Generation Attack: a trigger can falsely generate an object of the target class; 2) Regional Misclassification Attack: a trigger can change the prediction of a surrounding object to the target class; 3) Global Misclassification Attack: a single trigger can change the predictions of all objects in an image to the target class; and 4) Object Disappearance Attack: a trigger can make the detector fail to detect the object of the target class.
1 code implementation • 26 May 2022 • Tim Pearce, Jong-Hyeon Jeong, Yichen Jia, Jun Zhu
To offer theoretical insight into our algorithm, we show firstly that it can be interpreted as a form of expectation-maximisation, and secondly that it exhibits a desirable `self-correcting' property.
1 code implementation • 22 May 2022 • Ziyu Wang, Yuhao Zhou, Jun Zhu
We investigate nonlinear instrumental variable (IV) regression given high-dimensional instruments.
1 code implementation • Findings (NAACL) 2022 • Jun Zhu, Céline Hudelot
Works on learning job title representation are mainly based on \textit{Job-Transition Graph}, built from the working history of talents.
no code implementations • 30 Apr 2022 • Zhijie Deng, Feng Zhou, Jianfei Chen, Guoqiang Wu, Jun Zhu
In this way, we relate DE to Bayesian inference to enjoy reliable Bayesian uncertainty.
2 code implementations • 30 Apr 2022 • Zhijie Deng, Jiaxin Shi, Jun Zhu
Learning the principal eigenfunctions of an integral operator defined by a kernel and a data distribution is at the core of many machine learning problems.
no code implementations • 26 Mar 2022 • Sha Yuan, Hanyu Zhao, Shuai Zhao, Jiahong Leng, Yangxiao Liang, Xiaozhi Wang, Jifan Yu, Xin Lv, Zhou Shao, Jiaao He, Yankai Lin, Xu Han, Zhenghao Liu, Ning Ding, Yongming Rao, Yizhao Gao, Liang Zhang, Ming Ding, Cong Fang, Yisen Wang, Mingsheng Long, Jing Zhang, Yinpeng Dong, Tianyu Pang, Peng Cui, Lingxiao Huang, Zheng Liang, HuaWei Shen, HUI ZHANG, Quanshi Zhang, Qingxiu Dong, Zhixing Tan, Mingxuan Wang, Shuo Wang, Long Zhou, Haoran Li, Junwei Bao, Yingwei Pan, Weinan Zhang, Zhou Yu, Rui Yan, Chence Shi, Minghao Xu, Zuobai Zhang, Guoqiang Wang, Xiang Pan, Mengjie Li, Xiaoyu Chu, Zijun Yao, Fangwei Zhu, Shulin Cao, Weicheng Xue, Zixuan Ma, Zhengyan Zhang, Shengding Hu, Yujia Qin, Chaojun Xiao, Zheni Zeng, Ganqu Cui, Weize Chen, Weilin Zhao, Yuan YAO, Peng Li, Wenzhao Zheng, Wenliang Zhao, Ziyi Wang, Borui Zhang, Nanyi Fei, Anwen Hu, Zenan Ling, Haoyang Li, Boxi Cao, Xianpei Han, Weidong Zhan, Baobao Chang, Hao Sun, Jiawen Deng, Chujie Zheng, Juanzi Li, Lei Hou, Xigang Cao, Jidong Zhai, Zhiyuan Liu, Maosong Sun, Jiwen Lu, Zhiwu Lu, Qin Jin, Ruihua Song, Ji-Rong Wen, Zhouchen Lin, LiWei Wang, Hang Su, Jun Zhu, Zhifang Sui, Jiajun Zhang, Yang Liu, Xiaodong He, Minlie Huang, Jian Tang, Jie Tang
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
1 code implementation • 13 Mar 2022 • Yinpeng Dong, Shuyu Cheng, Tianyu Pang, Hang Su, Jun Zhu
However, the existing methods inevitably suffer from low attack success rates or poor query efficiency since it is difficult to estimate the gradient in a high-dimensional input space with limited information.
no code implementations • 13 Mar 2022 • Jialian Li, Tongzheng Ren, Dong Yan, Hang Su, Jun Zhu
Our goal is to identify a near-optimal robust policy for the perturbed testing environment, which introduces additional technical difficulties as we need to simultaneously estimate the training environment uncertainty from samples and find the worst-case perturbation for testing.
no code implementations • 9 Mar 2022 • Xiao Yang, Yinpeng Dong, Tianyu Pang, Zihao Xiao, Hang Su, Jun Zhu
It is therefore imperative to develop a framework that can enable a comprehensive evaluation of the vulnerability of face recognition in the physical world.
16 code implementations • 7 Mar 2022 • Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M. Ni, Heung-Yeung Shum
Compared to other models on the leaderboard, DINO significantly reduces its model size and pre-training data size while achieving better results.
Ranked #2 on
Object Detection
on SA-Det-100k
1 code implementation • 21 Feb 2022 • Tianyu Pang, Min Lin, Xiao Yang, Jun Zhu, Shuicheng Yan
The trade-off between robustness and accuracy has been widely studied in the adversarial literature.
1 code implementation • ICLR 2022 • Liyuan Wang, Xingxing Zhang, Kuo Yang, Longhui Yu, Chongxuan Li, Lanqing Hong, Shifeng Zhang, Zhenguo Li, Yi Zhong, Jun Zhu
In this work, we propose memory replay with data compression (MRDC) to reduce the storage cost of old training samples and thus increase their amount that can be stored in the memory buffer.
8 code implementations • ICLR 2022 • Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, Lei Zhang
We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR.
Ranked #102 on
Object Detection
on COCO minival
2 code implementations • ICLR 2022 • Fan Bao, Chongxuan Li, Jun Zhu, Bo Zhang
In this work, we present a surprising result that both the optimal reverse variance and the corresponding optimal KL divergence of a DPM have analytic forms w. r. t.
no code implementations • CVPR 2022 • Yunlong Wang, Hongyu Pan, Jun Zhu, Yu-Huan Wu, Xin Zhan, Kun Jiang, Diange Yang
In this paper, we propose a novel Spatial-Temporal Integrated network with Bidirectional Enhancement, BE-STI, to improve the temporal motion prediction performance by spatial semantic features, which points out an efficient way to combine semantic segmentation and motion prediction.
no code implementations • CVPR 2022 • Hongyang Gu, Jianmin Li, Guangyuan Fu, Chifong Wong, Xinghao Chen, Jun Zhu
In this paper, we propose a novel method, AutoLoss-GMS, to search the better loss function in the space of generalized margin-based softmax loss function for person re-identification automatically.
no code implementations • 9 Nov 2021 • Jun Zhu, Gautier Viaud, Céline Hudelot
The second module learns job seeker representations.
1 code implementation • NeurIPS 2021 • Liyuan Wang, Mingtian Zhang, Zhongfan Jia, Qian Li, Chenglong Bao, Kaisheng Ma, Jun Zhu, Yi Zhong
Without accessing to the old training samples, knowledge transfer from the old tasks to each new task is difficult to determine, which might be either positive or negative.
1 code implementation • 17 Oct 2021 • Yuefeng Chen, Xiaofeng Mao, Yuan He, Hui Xue, Chao Li, Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Hang Su, Jun Zhu, Fangcheng Liu, Chao Zhang, Hongyang Zhang, Yichi Zhang, Shilong Liu, Chang Liu, Wenzhao Xiang, Yajie Wang, Huipeng Zhou, Haoran Lyu, Yidan Xu, Zixuan Xu, Taoyu Zhu, Wenjun Li, Xianfeng Gao, Guoqiu Wang, Huanqian Yan, Ying Guo, Chaoning Zhang, Zheng Fang, Yang Wang, Bingyang Fu, Yunfei Zheng, Yekui Wang, Haorong Luo, Zhen Yang
Many works have investigated the adversarial attacks or defenses under the settings where a bounded and imperceptible perturbation can be added to the input.
1 code implementation • 15 Oct 2021 • Yinpeng Dong, Qi-An Fu, Xiao Yang, Wenzhao Xiang, Tianyu Pang, Hang Su, Jun Zhu, Jiayu Tang, Yuefeng Chen, Xiaofeng Mao, Yuan He, Hui Xue, Chao Li, Ye Liu, Qilong Zhang, Lianli Gao, Yunrui Yu, Xitong Gao, Zhe Zhao, Daquan Lin, Jiadong Lin, Chuanbiao Song, ZiHao Wang, Zhennan Wu, Yang Guo, Jiequan Cui, Xiaogang Xu, Pengguang Chen
Due to the vulnerability of deep neural networks (DNNs) to adversarial examples, a large number of defense techniques have been proposed to alleviate this problem in recent years.
no code implementations • 13 Oct 2021 • Xiao Yang, Yinpeng Dong, Wenzhao Xiang, Tianyu Pang, Hang Su, Jun Zhu
The vulnerability of deep neural networks to adversarial examples has motivated an increasing number of defense strategies for promoting model robustness.
1 code implementation • 9 Oct 2021 • Shiyu Huang, Wenze Chen, Longfei Zhang, Shizhen Xu, Ziyang Li, Fengming Zhu, Deheng Ye, Ting Chen, Jun Zhu
To the best of our knowledge, Tikick is the first learning-based AI system that can take over the multi-agent Google Research Football full game, while previous work could either control a single agent or experiment on toy academic scenarios.
1 code implementation • 8 Oct 2021 • Shiyu Huang, Bin Wang, Dong Li, Jianye Hao, Ting Chen, Jun Zhu
In this work, we propose a new algorithm for circuit routing, named Ranking Cost, which innovatively combines search-based methods (i. e., A* algorithm) and learning-based methods (i. e., Evolution Strategies) to form an efficient and trainable router.
no code implementations • ICML Workshop AML 2021 • Yichi Zhang, Zijian Zhu, Xiao Yang, Jun Zhu
To address this issue, we propose a novel method of Adversarial Semantic Contour (ASC) guided by object contour as prior.
no code implementations • 29 Sep 2021 • Yichi Zhou, Shihong Song, Huishuai Zhang, Jun Zhu, Wei Chen, Tie-Yan Liu
In contextual bandit, one major challenge is to develop theoretically solid and empirically efficient algorithms for general function classes.
no code implementations • 29 Sep 2021 • Zhijie Deng, Feng Zhou, Jianfei Chen, Guoqiang Wu, Jun Zhu
Deep Ensemble (DE) is a flexible, feasible, and effective alternative to Bayesian neural networks (BNNs) for uncertainty estimation in deep learning.
1 code implementation • ICML Workshop AML 2021 • Zhengyi Wang, Zhongkai Hao, Ziqiao Wang, Hang Su, Jun Zhu
In this work, we propose Cluster Attack -- a Graph Injection Attack (GIA) on node classification, which injects fake nodes into the original graph to degenerate the performance of graph neural networks (GNNs) on certain victim nodes while affecting the other nodes as little as possible.
1 code implementation • 29 Jul 2021 • Jiayi Weng, Huayu Chen, Dong Yan, Kaichao You, Alexis Duburcq, Minghao Zhang, Yi Su, Hang Su, Jun Zhu
In this paper, we present Tianshou, a highly modularized Python library for deep reinforcement learning (DRL) that uses PyTorch as its backend.
3 code implementations • 22 Jul 2021 • Shilong Liu, Lei Zhang, Xiao Yang, Hang Su, Jun Zhu
The use of Transformer is rooted in the need of extracting local discriminative features adaptively for different labels, which is a strongly desired property due to the existence of multiple objects in one image.
Ranked #1 on
Multi-Label Classification
on PASCAL VOC 2012
1 code implementation • NeurIPS 2021 • Shuyu Cheng, Guoqiang Wu, Jun Zhu
Finally, our theoretical results are confirmed by experiments on several numerical benchmarks as well as adversarial attacks.
1 code implementation • ICML Workshop AML 2021 • Xiao Yang, Yinpeng Dong, Tianyu Pang, Hang Su, Jun Zhu
Transfer-based adversarial attacks can evaluate model robustness in the black-box setting.
1 code implementation • 30 Jun 2021 • You Qiaoben, Chengyang Ying, Xinning Zhou, Hang Su, Jun Zhu, Bo Zhang
In this paper, we provide a framework to better understand the existing methods by reformulating the problem of adversarial attacks on reinforcement learning in the function space.
no code implementations • 29 Jun 2021 • Yichi Zhou, Shihong Song, Huishuai Zhang, Jun Zhu, Wei Chen, Tie-Yan Liu
However, it is in general unknown how to deriveefficient and effective EE trade-off methods for non-linearcomplex tasks, suchas contextual bandit with deep neural network as the reward function.
no code implementations • CVPR 2021 • Zihao Xiao, Xianfeng Gao, Chilin Fu, Yinpeng Dong, Wei Gao, Xiaolu Zhang, Jun Zhou, Jun Zhu
However, deep CNNs are vulnerable to adversarial patches, which are physically realizable and stealthy, raising new security concerns on the real-world applications of these models.
no code implementations • CVPR 2021 • Zhenwei Miao, Jikai Chen, Hongyu Pan, Ruiwen Zhang, Kaixuan Liu, Peihan Hao, Jun Zhu, Yang Wang, Xin Zhan
Quantization-based methods are widely used in LiDAR points 3D object detection for its efficiency in extracting context information.
1 code implementation • NeurIPS 2021 • Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, Jun Zhu
Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy.
no code implementations • ICML Workshop AML 2021 • You Qiaoben, Xinning Zhou, Chengyang Ying, Jun Zhu
Deep reinforcement learning (DRL) policies are vulnerable to the adversarial attack on their observations, which may mislead real-world RL agents to catastrophic failures.