no code implementations • ICML 2020 • Mingjie Li, Lingshen He, Zhouchen Lin

By viewing ResNet as an explicit Euler discretization of an ordinary differential equation (ODE), for the first time, we find that the adversarial robustness of ResNet is connected to the numerical stability of the corresponding dynamic system.

no code implementations • ICML 2020 • Yuchao Cai, Hanyuan Hang, Hanfang Yang, Zhouchen Lin

In this paper, we propose a boosting algorithm for regression problems called \textit{boosted histogram transform for regression} (BHTR) based on histogram transforms composed of random rotations, stretchings, and translations.

no code implementations • 17 Jul 2024 • Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, Zhouchen Lin

Despite the efforts of some online training methods, tackling spatial credit assignments by alternatives with comparable performance as spatial BP remains a significant problem.

no code implementations • 17 Jun 2024 • Yang Chen, Cong Fang, Zhouchen Lin, Bing Liu

Foundation Models (FMs) have demonstrated remarkable insights into the relational dynamics of the world, leading to the crucial question: how do these models acquire an understanding of world hybrid relations?

1 code implementation • 27 May 2024 • Mingqing Xiao, Yixin Zhu, Di He, Zhouchen Lin

Spiking neural networks (SNNs) are investigated as biologically inspired models of neural computation, distinguished by their computational capability and energy efficiency due to precise spiking times and sparse spikes with event-driven computation.

no code implementations • 29 Feb 2024 • Yang Chen, Yitao Liang, Zhouchen Lin

Causality has been combined with machine learning to produce robust representations for domain generalization.

1 code implementation • 19 Feb 2024 • Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, Zhouchen Lin

Neuromorphic computing with spiking neural networks is promising for energy-efficient artificial intelligence (AI) applications.

no code implementations • 1 Feb 2024 • Huan Li, Zhouchen Lin

Although adaptive gradient methods have been extensively used in deep learning, their convergence rates proved in the literature are all slower than that of SGD, particularly with respect to their dependence on the dimension.

no code implementations • CVPR 2024 • Yikang Li, Yeqing Qiu, Yuxuan Chen, Lingshen He, Zhouchen Lin

In this paper we build affine equivariant networks based on differential invariants from the viewpoint of symmetric PDEs without discretizing or sampling the group.

no code implementations • 9 Oct 2023 • Haotong Yang, Fanxu Meng, Zhouchen Lin, Muhan Zhang

Furthermore, by generalizing this structure to the hierarchical case, we demonstrate that models can achieve task composition, further reducing the space needed to learn from linear to logarithmic, thereby effectively learning on complex reasoning involving multiple steps.

no code implementations • 23 Sep 2023 • Pengyun Yue, Hanzhen Zhao, Cong Fang, Di He, LiWei Wang, Zhouchen Lin, Song-Chun Zhu

With distributed machine learning being a prominent technique for large-scale machine learning tasks, communication complexity has become a major bottleneck for speeding up training and scaling up machine numbers.

no code implementations • 22 Aug 2023 • Yuxuan Du, Yibo Yang, Tongliang Liu, Zhouchen Lin, Bernard Ghanem, DaCheng Tao

Understanding the dynamics of large quantum systems is hindered by the curse of dimensionality.

2 code implementations • 3 Aug 2023 • Yibo Yang, Haobo Yuan, Xiangtai Li, Jianlong Wu, Lefei Zhang, Zhouchen Lin, Philip Torr, DaCheng Tao, Bernard Ghanem

Beyond the normal case, long-tail class incremental learning and few-shot class incremental learning are also proposed to consider the data imbalance and data scarcity, respectively, which are common in real-world implementations and further exacerbate the well-known problem of catastrophic forgetting.

no code implementations • 29 May 2023 • Yi Hu, Haotong Yang, Zhouchen Lin, Muhan Zhang

We also consider the ensemble of code prompting and CoT prompting to combine the strengths of both.

1 code implementation • 22 May 2023 • Long Yang, Zhixiong Huang, Fenghao Lei, Yucun Zhong, Yiming Yang, Cong Fang, Shiting Wen, Binbin Zhou, Zhouchen Lin

Popular reinforcement learning (RL) algorithms tend to produce a unimodal policy distribution, which weakens the expressiveness of complicated policy and decays the ability of exploration.

no code implementations • 10 May 2023 • Bruce X. B. Yu, Jianlong Chang, Haixin Wang, Lingbo Liu, Shijie Wang, Zhiyu Wang, Junfan Lin, Lingxi Xie, Haojie Li, Zhouchen Lin, Qi Tian, Chang Wen Chen

With the surprising development of pre-trained visual foundation models, visual tuning jumped out of the standard modus operandi that fine-tunes the whole pre-trained model or just the fully connected layer.

no code implementations • 17 Mar 2023 • Haixin Wang, Jianlong Chang, Xiao Luo, Jinan Sun, Zhouchen Lin, Qi Tian

Despite recent competitive performance across a range of vision tasks, vision Transformers still have an issue of heavy computational costs.

1 code implementation • 8 Mar 2023 • Yifei Wang, Qi Zhang, Tianqi Du, Jiansheng Yang, Zhouchen Lin, Yisen Wang

In recent years, contrastive learning achieves impressive results on self-supervised visual representation learning, but there still lacks a rigorous understanding of its learning dynamics.

no code implementations • 2 Mar 2023 • Shihong Ding, Hanze Dong, Cong Fang, Zhouchen Lin, Tong Zhang

To circumvent this difficulty, we examine the problem of identifying a mixed Nash equilibrium, where strategies are randomized and characterized by probability distributions over continuous domains. To this end, we propose PArticle-based Primal-dual ALgorithm (PAPAL) tailored for a weakly entropy-regularized min-max optimization over probability distributions.

1 code implementation • ICCV 2023 • Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, Zhi-Quan Luo

In particular, our method achieves state-of-the-art accuracy on ImageNet, while the memory cost and training time are reduced by more than 70% and 50%, respectively, compared with BPTT.

1 code implementation • 6 Feb 2023 • Yibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin, Philip Torr, DaCheng Tao

In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF).

1 code implementation • ICLR 2023 • Yibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin, Philip Torr, DaCheng Tao

In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF).

Ranked #3 on Few-Shot Class-Incremental Learning on CUB-200-2011

1 code implementation • 1 Feb 2023 • Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Yisen Wang, Zhouchen Lin

In this paper, we study spike-based implicit differentiation on the equilibrium state (SPIDE) that extends the recently proposed training method, implicit differentiation on the equilibrium state (IDE), for supervised learning with purely spike-based computation, which demonstrates the potential for energy-efficient training of SNNs.

1 code implementation • 3 Jan 2023 • Xiangtai Li, Shilin Xu, Yibo Yang, Haobo Yuan, Guangliang Cheng, Yunhai Tong, Zhouchen Lin, Ming-Hsuan Yang, DaCheng Tao

Third, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross-attention scheme to boost part segmentation qualities further.

1 code implementation • 12 Oct 2022 • Yibo Yang, Hong Wang, Haobo Yuan, Zhouchen Lin

With NIO, we improve the classification performance of a variety of neural architectures on CIFAR-10, CIFAR-100, and ImageNet.

1 code implementation • 9 Oct 2022 • Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, Zhouchen Lin

With OTTT, it is the first time that two mainstream supervised SNN training methods, BPTT with SG and spike representation-based training, are connected, and meanwhile in a biologically plausible form.

Ranked #3 on Event data classification on CIFAR10-DVS

1 code implementation • 9 Oct 2022 • Mingqing Xiao, Shuxin Zheng, Chang Liu, Zhouchen Lin, Tie-Yan Liu

To be specific, we develop invertible models to generate valid degraded images and meanwhile transform the distribution of lost contents to the fixed distribution of a latent variable during the forward degradation.

1 code implementation • 19 Sep 2022 • Haotong Yang, Zhouchen Lin, Muhan Zhang

However, evaluation of knowledge graph completion (KGC) models often ignores the incompleteness -- facts in the test set are ranked against all unknown triplets which may contain a large number of missing facts not included in the KG yet.

4 code implementations • 13 Aug 2022 • Xingyu Xie, Pan Zhou, Huan Li, Zhouchen Lin, Shuicheng Yan

Adan first reformulates the vanilla Nesterov acceleration to develop a new Nesterov momentum estimation (NME) method, which avoids the extra overhead of computing gradient at the extrapolation point.

1 code implementation • 7 Aug 2022 • Zhengyang Shen, Tao Hong, Qi She, Jinwen Ma, Zhouchen Lin

Steerable models can provide very general and flexible equivariance by formulating equivariance requirements in the language of representation theory and feature fields, which has been recognized to be effective for many vision tasks.

1 code implementation • 29 Jun 2022 • Qi Chen, Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin

Moreover, we show that the optimization-induced variants of our models can boost the performance and improve training stability and efficiency as well.

1 code implementation • 20 Jun 2022 • Yang Hu, Xiyuan Wang, Zhouchen Lin, Pan Li, Muhan Zhang

As pointed out by previous works, this two-step procedure results in low discriminating power, as 1-WL-GNNs by nature learn node-level representations instead of link-level.

no code implementations • 27 May 2022 • Zenan Ling, Xingyu Xie, Qiuhao Wang, Zongpeng Zhang, Zhouchen Lin

A deep equilibrium model (DEQ) is implicitly defined through an equilibrium point of an infinite-depth weight-tied model with an input-injection.

no code implementations • 26 May 2022 • Mingjie Li, Hao Kong, Zhouchen Lin

Furthermore, we analyze the constraints of the inversion layer to ensure the output stability of the network to a certain extent.

1 code implementation • CVPR 2022 • Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, Zhi-Quan Luo

In this paper, we propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance that is competitive to ANNs yet with low latency.

no code implementations • 26 Mar 2022 • Sha Yuan, Hanyu Zhao, Shuai Zhao, Jiahong Leng, Yangxiao Liang, Xiaozhi Wang, Jifan Yu, Xin Lv, Zhou Shao, Jiaao He, Yankai Lin, Xu Han, Zhenghao Liu, Ning Ding, Yongming Rao, Yizhao Gao, Liang Zhang, Ming Ding, Cong Fang, Yisen Wang, Mingsheng Long, Jing Zhang, Yinpeng Dong, Tianyu Pang, Peng Cui, Lingxiao Huang, Zheng Liang, HuaWei Shen, HUI ZHANG, Quanshi Zhang, Qingxiu Dong, Zhixing Tan, Mingxuan Wang, Shuo Wang, Long Zhou, Haoran Li, Junwei Bao, Yingwei Pan, Weinan Zhang, Zhou Yu, Rui Yan, Chence Shi, Minghao Xu, Zuobai Zhang, Guoqiang Wang, Xiang Pan, Mengjie Li, Xiaoyu Chu, Zijun Yao, Fangwei Zhu, Shulin Cao, Weicheng Xue, Zixuan Ma, Zhengyan Zhang, Shengding Hu, Yujia Qin, Chaojun Xiao, Zheni Zeng, Ganqu Cui, Weize Chen, Weilin Zhao, Yuan YAO, Peng Li, Wenzhao Zheng, Wenliang Zhao, Ziyi Wang, Borui Zhang, Nanyi Fei, Anwen Hu, Zenan Ling, Haoyang Li, Boxi Cao, Xianpei Han, Weidong Zhan, Baobao Chang, Hao Sun, Jiawen Deng, Chujie Zheng, Juanzi Li, Lei Hou, Xigang Cao, Jidong Zhai, Zhiyuan Liu, Maosong Sun, Jiwen Lu, Zhiwu Lu, Qin Jin, Ruihua Song, Ji-Rong Wen, Zhouchen Lin, LiWei Wang, Hang Su, Jun Zhu, Zhifang Sui, Jiajun Zhang, Yang Liu, Xiaodong He, Minlie Huang, Jian Tang, Jie Tang

With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.

1 code implementation • 25 Mar 2022 • Yifei Wang, Qi Zhang, Yisen Wang, Jiansheng Yang, Zhouchen Lin

Our theory suggests an alternative understanding of contrastive learning: the role of aligning positive samples is more like a surrogate task than an ultimate goal, and the overlapped augmented views (i. e., the chaos) create a ladder for contrastive learning to gradually learn class-separated representations.

no code implementations • ICLR 2022 • Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin

On the other hand, our unified framework can be extended to the unsupervised scenario, which interprets unsupervised contrastive learning as an important sampling of CEM.

1 code implementation • 17 Mar 2022 • Yibo Yang, Shixiang Chen, Xiangtai Li, Liang Xie, Zhouchen Lin, DaCheng Tao

Modern deep neural networks for classification usually jointly learn a backbone for representation and a linear classifier to output the logit of each class.

Ranked #27 on Long-tail Learning on CIFAR-10-LT (ρ=100)

1 code implementation • 27 Jan 2022 • Huan Li, Zhouchen Lin

They do not invoke negative curvature exploitation or minimization of regularized surrogate functions as the subroutines.

no code implementations • NeurIPS 2021 • Lingshen He, Yiming Dong, Yisen Wang, DaCheng Tao, Zhouchen Lin

Attention mechanism has shown great performance and efficiency in a lot of deep learning models, in which relative position encoding plays a crucial role.

1 code implementation • NeurIPS 2021 • Lingshen He, Yuxuan Chen, Zhengyang Shen, Yiming Dong, Yisen Wang, Zhouchen Lin

Group equivariant CNNs (G-CNNs) that incorporate more equivariance can significantly improve the performance of conventional CNNs.

1 code implementation • NeurIPS 2021 • Zhengyang Geng, Xin-Yu Zhang, Shaojie Bai, Yisen Wang, Zhouchen Lin

This paper focuses on training implicit models of infinite layers.

no code implementations • 3 Nov 2021 • Ke Sun, Mingjie Li, Zhouchen Lin

In this paper, we endeavor to design strategies to achieve universal adversarial robustness.

no code implementations • NeurIPS 2021 • Yifei Wang, Zhengyang Geng, Feng Jiang, Chuming Li, Yisen Wang, Jiansheng Yang, Zhouchen Lin

Multi-view methods learn representations by aligning multiple views of the same image and their performance largely depends on the choice of data augmentation.

no code implementations • ICLR 2022 • Yifei Wang, Qi Zhang, Yisen Wang, Jiansheng Yang, Zhouchen Lin

Our work suggests an alternative understanding of contrastive learning: the role of aligning positive samples is more like a surrogate task than an ultimate goal, and it is the overlapping augmented views (i. e., the chaos) that create a ladder for contrastive learning to gradually learn class-separated representations.

1 code implementation • NeurIPS 2021 • Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Yisen Wang, Zhouchen Lin

In this work, we consider feedback spiking neural networks, which are more brain-like, and propose a novel training method that does not rely on the exact reverse of the forward computation.

no code implementations • ICLR 2022 • Mingjie Li, Yisen Wang, Xingyu Xie, Zhouchen Lin

Works have shown the strong connections between some implicit models and optimization problems.

3 code implementations • ICLR 2021 • Zhengyang Geng, Meng-Hao Guo, Hongxu Chen, Xia Li, Ke Wei, Zhouchen Lin

As an essential ingredient of modern deep learning, attention mechanism, especially self-attention, plays a vital role in the global correlation discovery.

Ranked #7 on Semantic Segmentation on PASCAL VOC 2012 test

no code implementations • 1 Sep 2021 • Hanyuan Hang, Yuchao Cai, Hanfang Yang, Zhouchen Lin

In this paper, we propose an ensemble learning algorithm called \textit{under-bagging $k$-nearest neighbors} (\textit{under-bagging $k$-NN}) for imbalanced classification problems.

1 code implementation • 1 Jul 2021 • Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin

Recently, sampling methods have been successfully applied to enhance the sample quality of Generative Adversarial Networks (GANs).

1 code implementation • 18 Jun 2021 • Qigong Sun, Xiufang Li, Fanhua Shang, Hongying Liu, Kang Yang, Licheng Jiao, Zhouchen Lin

The training of deep neural networks (DNNs) always requires intensive resources for both computation and data storage.

no code implementations • ICML Workshop AML 2021 • Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin

Based on these, we propose principled adversarial sampling algorithms in both supervised and unsupervised scenarios.

no code implementations • 10 Jun 2021 • Jingyi Cui, Hanyuan Hang, Yisen Wang, Zhouchen Lin

In this paper, we propose a density estimation algorithm called \textit{Gradient Boosting Histogram Transform} (GBHT), where we adopt the \textit{Negative Log Likelihood} as the loss function to make the boosting procedure available for the unsupervised tasks.

1 code implementation • 10 Jun 2021 • Hongwei Wen, Jingyi Cui, Hanyuan Hang, Jiabin Liu, Yisen Wang, Zhouchen Lin

As an important branch of weakly supervised learning, partial label learning deals with data where each instance is assigned with a set of candidate labels, whereas only one of them is true.

no code implementations • 3 Jun 2021 • Hanyuan Hang, Tao Huang, Yuchao Cai, Hanfang Yang, Zhouchen Lin

In this paper, we propose a gradient boosting algorithm for large-scale regression problems called \textit{Gradient Boosted Binary Histogram Ensemble} (GBBHE) based on binary histogram partition and ensemble learning.

no code implementations • 27 May 2021 • Xingyu Xie, Qiuhao Wang, Zenan Ling, Xia Li, Yisen Wang, Guangcan Liu, Zhouchen Lin

In this paper, we investigate an emerging question: can an implicit equilibrium model's equilibrium point be regarded as the solution of an optimization problem?

1 code implementation • 25 May 2021 • Hao He, Xiangtai Li, Yibo Yang, Guangliang Cheng, Yunhai Tong, Lubin Weng, Zhouchen Lin, Shiming Xiang

This module is used to squeeze the object boundary from both inner and outer directions, which contributes to precise mask representation.

1 code implementation • 8 Apr 2021 • Zhengyang Shen, Tiancheng Shen, Zhouchen Lin, Jinwen Ma

Spherical signals exist in many applications, e. g., planetary data, LiDAR scans and digitalization of 3D objects, calling for models that can process spherical data effectively.

no code implementations • 6 Apr 2021 • Huan Li, Zhouchen Lin

We prove the $O((\frac{\gamma}{1-\sigma_{\gamma}})^2\sqrt{\frac{L}{\epsilon}})$ and $O((\frac{\gamma}{1-\sigma_{\gamma}})^{1. 5}\sqrt{\frac{L}{\mu}}\log\frac{1}{\epsilon})$ complexities for the practical single loop accelerated gradient tracking over time-varying graphs when the problems are nonstrongly convex and strongly convex, respectively, where $\gamma$ and $\sigma_{\gamma}$ are two common constants charactering the network connectivity, $\epsilon$ is the desired precision, and $L$ and $\mu$ are the smoothness and strong convexity constants, respectively.

1 code implementation • ICCV 2021 • Huasong Zhong, Jianlong Wu, Chong Chen, Jianqiang Huang, Minghua Deng, Liqiang Nie, Zhouchen Lin, Xian-Sheng Hua

On the other hand, a novel graph-based contrastive learning strategy is proposed to learn more compact clustering assignments.

1 code implementation • CVPR 2021 • Xiangtai Li, Hao He, Xia Li, Duo Li, Guangliang Cheng, Jianping Shi, Lubin Weng, Yunhai Tong, Zhouchen Lin

Experimental results on three different aerial segmentation datasets suggest that the proposed method is more effective and efficient than state-of-the-art general semantic segmentation methods.

1 code implementation • NeurIPS 2021 • Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin

Graph Convolutional Networks (GCNs) have attracted more and more attentions in recent years.

no code implementations • CVPR 2021 • Yibo Yang, Shan You, Hongyang Li, Fei Wang, Chen Qian, Zhouchen Lin

Our method enables differentiable sparsification, and keeps the derived architecture equivalent to that of Engine-cell, which further improves the consistency between search and evaluation.

1 code implementation • 27 Jan 2021 • Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, Zhouchen Lin

Bi-Level Optimization (BLO) is originated from the area of economic game theory and then introduced into the optimization community.

no code implementations • 1 Jan 2021 • Yifei Wang, Yisen Wang, Jiansheng Yang, Zhouchen Lin

Recently, sampling methods have been successfully applied to enhance the sample quality of Generative Adversarial Networks (GANs).

no code implementations • 1 Jan 2021 • Xingyu Xie, Hao Kong, Jianlong Wu, Guangcan Liu, Zhouchen Lin

First of all, to perform matrix inverse, we provide a differentiable yet efficient way, named LD-Minv, which is a learnable deep neural network (DNN) with each layer being an $L$-th order matrix polynomial.

no code implementations • 1 Jan 2021 • Jiabin Liu, Hanyuan Hang, Bo wang, Xin Shen, Zhouchen Lin

Learning from label proportions (LLP), where the training data are arranged in form of groups with only label proportions provided instead of the exact labels, is an important weakly supervised learning paradigm in machine learning.

no code implementations • 1 Jan 2021 • Yibo Yang, Shan You, Hongyang Li, Fei Wang, Chen Qian, Zhouchen Lin

The Engine-cell is differentiable for architecture search, while the Transit-cell only transits the current sub-graph by architecture derivation.

1 code implementation • 6 Nov 2020 • Xiangtai Li, Xia Li, Ansheng You, Li Zhang, Guangliang Cheng, Kuiyuan Yang, Yunhai Tong, Zhouchen Lin

Instead of propagating information on the spatial map, we first learn to squeeze the input feature into a channel-wise global vector and perform reasoning within the single vector where the computation cost can be significantly reduced.

1 code implementation • NeurIPS 2020 • Yibo Yang, Hongyang Li, Shan You, Fei Wang, Chen Qian, Zhouchen Lin

By doing so, our network for search at each update satisfies the sparsity constraint and is efficient to train.

no code implementations • 9 Sep 2020 • Huan Li, Zhouchen Lin, Yongchun Fang

Our stochastic gradient computation complexities are the same as the ones of single-machine VR methods, such as SAG, SAGA, and SVRG, and our communication complexities keep the same as those of EXTRA and DIGing, respectively.

3 code implementations • ICML 2020 • Zhengyang Shen, Lingshen He, Zhouchen Lin, Jinwen Ma

In implementation, we discretize the system using the numerical schemes of PDOs, deriving approximately equivariant convolutions (PDO-eConvs).

Ranked #1 on Image Classification on MNIST-rot-12

2 code implementations • ECCV 2020 • Xiangtai Li, Xia Li, Li Zhang, Guangliang Cheng, Jianping Shi, Zhouchen Lin, Shaohua Tan, Yunhai Tong

Our insight is that appealing performance of semantic segmentation requires \textit{explicitly} modeling the object \textit{body} and \textit{edge}, which correspond to the high and low frequency of the image.

1 code implementation • ICML 2020 • Xingyu Xie, Hao Kong, Jianlong Wu, Wayne Zhang, Guangcan Liu, Zhouchen Lin

While successful in many fields, deep neural networks (DNNs) still suffer from some open problems such as bad local minima and unsatisfactory generalization performance.

no code implementations • 14 Jun 2020 • Bing Yu, Ke Sun, He Wang, Zhouchen Lin, Zhanxing Zhu

The scarcity of class-labeled data is a ubiquitous bottleneck in many machine learning problems.

10 code implementations • ECCV 2020 • Mingqing Xiao, Shuxin Zheng, Chang Liu, Yaolong Wang, Di He, Guolin Ke, Jiang Bian, Zhouchen Lin, Tie-Yan Liu

High-resolution digital images are usually downscaled to fit various display screens or save the cost of storage and bandwidth, meanwhile the post-upscaling is adpoted to recover the original resolutions or the details in the zoom-in images.

no code implementations • CVPR 2020 • Xia Li, Yibo Yang, Qijie Zhao, Tiancheng Shen, Zhouchen Lin, Hong Liu

The convolution operation suffers from a limited receptive filed, while global modeling is fundamental to dense prediction tasks, such as semantic segmentation.

no code implementations • 24 Feb 2020 • Huan Li, Zhouchen Lin

EXTRA is a popular method for dencentralized distributed optimization and has broad applications.

no code implementations • 8 Dec 2019 • Hanyuan Hang, Zhouchen Lin, Xiaoyu Liu, Hongwei Wen

Instead, we apply kernel histogram transforms (KHT) equipped with smoother regressors such as support vector machines (SVMs), and it turns out that both single and ensemble KHT enjoy almost optimal convergence rates.

no code implementations • 23 Nov 2019 • Yibo Yang, Jianlong Wu, Hongyang Li, Xia Li, Tiancheng Shen, Zhouchen Lin

We establish a stability condition for ResNets with step sizes and weight parameters, and point out the effects of step sizes on the stability and performance.

no code implementations • 21 Nov 2019 • Ke Sun, Bing Yu, Zhouchen Lin, Zhanxing Zhu

Regularization plays a crucial role in machine learning models, especially for deep neural networks.

1 code implementation • 18 Nov 2019 • Yibo Yang, Hongyang Li, Xia Li, Qijie Zhao, Jianlong Wu, Zhouchen Lin

In order to overcome the lack of supervision, we introduce a differentiable module to resolve the overlap between any pair of instances.

Ranked #8 on Panoptic Segmentation on Cityscapes test

no code implementations • 26 Oct 2019 • Hao Kong, Canyi Lu, Zhouchen Lin

Recently, the \textit{Tensor Nuclear Norm~(TNN)} regularization based on t-SVD has been widely used in various low tubal-rank tensor recovery tasks.

1 code implementation • ICLR 2021 • Ke Sun, Zhanxing Zhu, Zhouchen Lin

The design of deep graph models still remains to be investigated and the crucial part is how to explore and exploit the knowledge from different hops of neighbors in an efficient way.

Ranked #2 on Node Classification on MS ACADEMIC

5 code implementations • ICCV 2019 • Xia Li, Zhisheng Zhong, Jianlong Wu, Yibo Yang, Zhouchen Lin, Hong Liu

It is designed to compute the representation of each position by a weighted sum of the features at all positions.

Ranked #5 on Semantic Segmentation on BDD100K val

no code implementations • 18 Jun 2019 • Zhisheng Zhong, Fangyin Wei, Zhouchen Lin, Chao Zhang

Furthermore, we propose that weight tensors in networks with proper order and balanced dimension are easier to be compressed.

1 code implementation • 15 May 2019 • Xingyu Xie, Jianlong Wu, Zhisheng Zhong, Guangcan Liu, Zhouchen Lin

Recently, a number of learning-based optimization methods that combine data-driven architectures with the classical optimization algorithms have been proposed and explored, showing superior empirical performance in solving various ill-posed inverse problems, but there is still a scarcity of rigorous analysis about the convergence behaviors of learning-based optimization.

no code implementations • CVPR 2019 • Junjian Zhang, Chun-Guang Li, Chong You, Xianbiao Qi, Honggang Zhang, Jun Guo, Zhouchen Lin

However, the applicability of subspace clustering has been limited because practical visual data in raw form do not necessarily lie in such linear subspaces.

Ranked #2 on Image Clustering on Extended Yale-B

1 code implementation • ICCV 2019 • Jianlong Wu, Keyu Long, Fei Wang, Chen Qian, Cheng Li, Zhouchen Lin, Hongbin Zha

Recent developed deep unsupervised methods allow us to jointly learn representation and cluster unlabelled data.

Ranked #7 on Image Clustering on Tiny-ImageNet

1 code implementation • 28 Feb 2019 • Ke Sun, Zhouchen Lin, Zhanxing Zhu

In this paper, we propose a novel training algorithm for Graph Convolutional Network, called Multi-Stage Self-Supervised(M3S) Training Algorithm, combined with self-supervised learning approach, focusing on improving the generalization performance of GCNs on graphs with few labeled nodes.

no code implementations • 28 Feb 2019 • Ke Sun, Zhanxing Zhu, Zhouchen Lin

In this paper, we present a systematic study on adversarial examples from three aspects: the amount of training data, task-dependent and model-specific factors.

no code implementations • 28 Feb 2019 • Ke Sun, Zhouchen Lin, Hantao Guo, Zhanxing Zhu

The effectiveness of Graph Convolutional Networks (GCNs) has been demonstrated in a wide range of graph-based machine learning tasks.

no code implementations • 28 Feb 2019 • Ke Sun, Zhanxing Zhu, Zhouchen Lin

In this work, we propose a novel defense mechanism called Boundary Conditional GAN to enhance the robustness of deep neural networks against adversarial examples.

no code implementations • 1 Feb 2019 • Cong Fang, Zhouchen Lin, Tong Zhang

In this paper, we give a sharp analysis for Stochastic Gradient Descent (SGD) and prove that SGD is able to efficiently escape from saddle points and find an $(\epsilon, O(\epsilon^{0. 5}))$-approximate second-order stationary point in $\tilde{O}(\epsilon^{-3. 5})$ stochastic gradient computations for generic nonconvex optimization problems, when the objective function satisfies gradient-Lipschitz, Hessian-Lipschitz, and dispersive noise assumptions.

no code implementations • NeurIPS 2018 • Cong Fang, Chris Junchi Li, Zhouchen Lin, Tong Zhang

Specially, we prove that the SPIDER-SFO algorithm achieves a gradient computation cost of $\mathcal{O}\left( \min( n^{1/2} \epsilon^{-2}, \epsilon^{-3} ) \right)$ to find an $\epsilon$-approximate first-order stationary point.

no code implementations • 5 Nov 2018 • Jia Li, Cong Fang, Zhouchen Lin

LPOM is block multi-convex in all layer-wise weights and activations.

no code implementations • 11 Oct 2018 • Fanhua Shang, James Cheng, Yuanyuan Liu, Zhi-Quan Luo, Zhouchen Lin

The heavy-tailed distributions of corrupted outliers and singular values of all channels in low-level vision have proven effective priors for many applications such as background modeling, photometric stereo and image alignment.

no code implementations • 3 Oct 2018 • Huan Li, Yibo Yang, Dongmin Chen, Zhouchen Lin

In this paper, we propose the hypothesis that the neural network structure design can be inspired by optimization algorithms and a faster optimization algorithm may lead to a better neural network structure.

no code implementations • NeurIPS 2018 • Zhisheng Zhong, Tiancheng Shen, Yibo Yang, Zhouchen Lin, Chao Zhang

To solve these problems, we propose the Super-Resolution CliqueNet (SRCliqueNet) to reconstruct the high resolution (HR) image with better textural details in the wavelet domain.

no code implementations • 16 Aug 2018 • Risheng Liu, Shichao Cheng, Yi He, Xin Fan, Zhouchen Lin, Zhongxuan Luo

Moreover, there is a lack of rigorous analysis about the convergence behaviors of these reimplemented iterations, and thus the significance of such methods is a little bit vague.

no code implementations • ECCV 2018 • Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, Hongbin Zha

In heavy rain, rain streaks have various directions and shapes, which can be regarded as the accumulation of multiple rain streak layers.

Ranked #7 on Single Image Deraining on Test2800

no code implementations • 10 Jul 2018 • Jianlong Wu, Zhouchen Lin, Hongbin Zha

In this paper, we focus on the Markov chain based spectral clustering method and propose a novel essential tensor learning method to explore the high order correlations for multi-view representation.

no code implementations • NeurIPS 2018 • Cong Fang, Chris Junchi Li, Zhouchen Lin, Tong Zhang

For stochastic first-order method, combining SPIDER with normalized gradient descent, we propose two new algorithms, namely SPIDER-SFO and SPIDER-SFO\textsuperscript{+}, that solve non-convex stochastic optimization problems using stochastic gradients only.

1 code implementation • 7 Jun 2018 • Canyi Lu, Jiashi Feng, Zhouchen Lin, Shuicheng Yan

Specifically, we show that by solving a TNN minimization problem, the underlying tensor of size $n_1\times n_2\times n_3$ with tubal rank $r$ can be exactly recovered when the given number of Gaussian measurements is $O(r(n_1+n_2-r)n_3)$.

no code implementations • 23 May 2018 • Canyi Lu, Jiashi Feng, Zhouchen Lin, Tao Mei, Shuicheng Yan

Second, we observe that many existing methods approximate the block diagonal representation matrix by using different structure priors, e. g., sparsity and low-rankness, which are indirect.

1 code implementation • 10 Apr 2018 • Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, Shuicheng Yan

Equipped with the new tensor nuclear norm, we then solve the TRPCA problem by solving a convex program and provide the theoretical guarantee for the exact recovery.

3 code implementations • CVPR 2018 • Yibo Yang, Zhisheng Zhong, Tiancheng Shen, Zhouchen Lin

In contrast to prior networks, there are both forward and backward connections between any two layers in the same block.

no code implementations • 27 Feb 2018 • Cong Fang, Yameng Huang, Zhouchen Lin

$O(1/\epsilon)$) convergence rate for non-strongly convex functions, and $O(\sqrt{\kappa}\log(1/\epsilon))$ (v. s.

no code implementations • ICLR 2018 • Chen Xu, Jianqiang Yao, Zhouchen Lin, Wenwu Ou, Yuanbin Cao, Zhirong Wang, Hongbin Zha

Recurrent neural networks have achieved excellent performance in many applications.

no code implementations • 8 Dec 2017 • Canyi Lu, Jiashi Feng, Zhouchen Lin, Shuicheng Yan

Experimental analysis on several real data sets verifies the effectiveness of our method.

no code implementations • CVPR 2016 • Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, Shuicheng Yan

In this work, we prove that under certain suitable assumptions, we can recover both the low-rank and the sparse components exactly by simply solving a convex program whose objective is a weighted combination of the tensor nuclear norm and the $\ell_1$-norm, i. e., $\min_{{\mathcal{L}},\ {\mathcal{E}}} \ \|{{\mathcal{L}}}\|_*+\lambda\|{{\mathcal{E}}}\|_1, \ \text{s. t.}

no code implementations • 25 Nov 2016 • Chen Xu, Zhouchen Lin, Hongbin Zha

In this paper, we show that for any $p$, $p_1$, and $p_2 >0$ satisfying $1/p=1/p_1+1/p_2$, there is an equivalence between the Schatten-$p$ norm of one matrix and the Schatten-$p_1$ and the Schatten-$p_2$ norms of its two factor matrices.

no code implementations • 17 Aug 2016 • Xiang Zhang, Jiarui Sun, Siwei Ma, Zhouchen Lin, Jian Zhang, Shiqi Wang, Wen Gao

Therefore, introducing an accurate rate-constraint in sparse coding and dictionary learning becomes meaningful, which has not been fully exploited in the context of sparse representation.

no code implementations • 8 Jul 2016 • Liansheng Zhuang, Zihan Zhou, Jingwen Yin, Shenghua Gao, Zhouchen Lin, Yi Ma, Nenghai Yu

In the literature, most existing graph-based semi-supervised learning (SSL) methods only use the label information of observed samples in the label propagation stage, while ignoring such valuable information when learning the graph.

no code implementations • CVPR 2016 • Jinshan Pan, Zhouchen Lin, Zhixun Su, Ming-Hsuan Yang

Estimating blur kernels from real world images is a challenging problem as the linear image formation assumption does not hold when significant outliers, such as saturated pixels and non-Gaussian noise, are present.

no code implementations • 12 Jan 2016 • Yuqing Hou, Zhouchen Lin, Jin-Ge Yao

Annotating images with tags is useful for indexing and retrieving images.

no code implementations • 2 Jan 2016 • Xinglin Piao, Yongli Hu, Junbin Gao, Yanfeng Sun, Zhouchen Lin, Bao-Cai Yin

A new submodule clustering method via sparse and low-rank representation for multi-way data is proposed in this paper.

1 code implementation • 18 Dec 2015 • Li Shen, Zhouchen Lin, Qingming Huang

Learning deeper convolutional neural networks becomes a tendency in recent years.

Ranked #8 on Long-tail Learning on VOC-MLT

no code implementations • ICCV 2015 • Chun-Guang Li, Zhouchen Lin, Honggang Zhang, Jun Guo

State of the art approaches for Semi-Supervised Learning (SSL) usually follow a two-stage framework -- constructing an affinity matrix from the data and then propagating the partial labels on this affinity matrix to infer those unknown labels.

no code implementations • NeurIPS 2015 • Huan Li, Zhouchen Lin

However, it is still unknown whether the usual APG can ensure the convergence to a critical point in nonconvex programming.

no code implementations • 25 Nov 2015 • Chen Xu, Zhouchen Lin, Zhenyu Zhao, Hongbin Zha

We propose a new majorization-minimization (MM) method for non-smooth and non-convex programs, which is general enough to include the existing MM methods.

no code implementations • 21 Nov 2015 • Canyi Lu, Shuicheng Yan, Zhouchen Lin

Spectral Clustering (SC) is one of the most widely used methods for data clustering.

no code implementations • 14 Nov 2015 • Canyi Lu, Huan Li, Zhouchen Lin, Shuicheng Yan

The Augmented Lagragian Method (ALM) and Alternating Direction Method of Multiplier (ADMM) have been powerful optimization methods for general convex programming subject to linear constraint.

no code implementations • 23 Oct 2015 • Canyi Lu, Jinhui Tang, Shuicheng Yan, Zhouchen Lin

The nuclear norm is widely used as a convex surrogate of the rank function in compressive sensing for low rank matrix recovery with its applications in image recovery and signal processing.

no code implementations • 13 Aug 2015 • Canyi Lu, Huan Li, Zhouchen Lin

To the best of our knowledge, this is the first work which directly minimizes the mutual coherence of the projected dictionary with a convergence guarantee.

no code implementations • 25 Jun 2015 • Hongyang Zhang, Zhouchen Lin, Chao Zhang

As an application, we also find that the solutions to extended robust Low-Rank Representation and to our extended robust MC are mutually expressible, so both our theory and algorithm can be applied to the subspace clustering problem with missing values under certain conditions.

no code implementations • 10 Jun 2015 • Yuqing Hou, Zhouchen Lin

Tag-based image retrieval (TBIR) has drawn much attention in recent years due to the explosive amount of digital images and crowdsourcing tags.

no code implementations • CVPR 2015 • Baohua Li, Ying Zhang, Zhouchen Lin, Huchuan Lu

Therefore, we propose Mixture of Gaussian Regression (MoG Regression) for subspace clustering by modeling noise as a Mixture of Gaussians (MoG).

no code implementations • CVPR 2015 • Zhizhong Li, Deli Zhao, Zhouchen Lin, Edward Y. Chang

In the line search step, R3MC approximates the minimum point on the searching curve by minimizing on the line tangent to the curve.

no code implementations • 18 Jan 2015 • Canyi Lu, Jinhui Tang, Min Lin, Liang Lin, Shuicheng Yan, Zhouchen Lin

In this paper, we study the robust subspace clustering problem, which aims to cluster the given possibly noisy data points into their underlying subspaces.

no code implementations • 18 Jan 2015 • Canyi Lu, Jiashi Feng, Zhouchen Lin, Shuicheng Yan

In this work, we argue that both sparsity and the grouping effect are important for subspace segmentation.

no code implementations • 6 Dec 2014 • Hongyang Zhang, Zhouchen Lin, Chao Zhang, Junbin Gao

More specifically, we discover that once a solution to one of the models is obtained, we can obtain the solutions to other models in closed-form formulations.

no code implementations • 6 Dec 2014 • Canyi Lu, Changbo Zhu, Chunyan Xu, Shuicheng Yan, Zhouchen Lin

This work studies the Generalized Singular Value Thresholding (GSVT) operator ${\text{Prox}}_{g}^{{\sigma}}(\cdot)$, \begin{equation*} {\text{Prox}}_{g}^{{\sigma}}(B)=\arg\min\limits_{X}\sum_{i=1}^{m}g(\sigma_{i}(X)) + \frac{1}{2}||X-B||_{F}^{2}, \end{equation*} associated with a nonconvex function $g$ defined on the singular values of $X$.

no code implementations • 3 Sep 2014 • Liansheng Zhuang, Shenghua Gao, Jinhui Tang, Jingjing Wang, Zhouchen Lin, Yi Ma

This paper aims at constructing a good graph for discovering intrinsic data structures in a semi-supervised learning setting.

no code implementations • CVPR 2014 • Chunyu Wang, Yizhou Wang, Zhouchen Lin, Alan L. Yuille, Wen Gao

We address the challenges in three ways: (i) We represent a 3D pose as a linear combination of a sparse set of bases learned from 3D human skeletons.

Ranked #29 on 3D Human Pose Estimation on HumanEva-I

no code implementations • CVPR 2014 • Jiashi Feng, Zhouchen Lin, Huan Xu, Shuicheng Yan

Most current state-of-the-art subspace segmentation methods (such as SSC and LRR) resort to alternative structural priors (such as sparseness and low-rankness) to construct the affinity matrix.

no code implementations • CVPR 2014 • Risheng Liu, Junjie Cao, Zhouchen Lin, Shiguang Shan

Then by optimizing a discrete submodular function constrained with this LESD and a uniform matroid, the saliency seeds (i. e., boundary conditions) can be learnt for this image, thus achieving an optimal PDE system to model the evolution of visual saliency.

no code implementations • CVPR 2014 • Han Hu, Zhouchen Lin, Jianjiang Feng, Jie zhou

Based on our analysis, we propose the SMooth Representation (SMR) model.

no code implementations • CVPR 2014 • Canyi Lu, Jinhui Tang, Shuicheng Yan, Zhouchen Lin

We observe that all the existing nonconvex penalty functions are concave and monotonically increasing on $[0,\infty)$.

no code implementations • 28 Apr 2014 • Canyi Lu, Yunchao Wei, Zhouchen Lin, Shuicheng Yan

This paper proposes the Proximal Iteratively REweighted (PIRE) algorithm for solving a general problem, which involves a large body of nonconvex sparse and structured sparse related problems.

no code implementations • 29 Jan 2014 • Canyi Lu, Zhouchen Lin, Shuicheng Yan

Our convergence proof of IRLS is more general than previous one which depends on the special properties of the Schatten-$p$ norm and $\ell_{2, q}$-norm.

no code implementations • 18 Oct 2013 • Zhouchen Lin, Risheng Liu, Huan Li

However, the traditional alternating direction method (ADM) and its linearized version (LADM, obtained by linearizing the quadratic penalty term) are for the two-block case and cannot be naively generalized to solve the multi-block case.

no code implementations • 23 Apr 2013 • Hongyang Zhang, Zhouchen Lin, Chao Zhang

For several rank minimization problems, such a replacement has been theoretically proven to be valid, i. e., the solution to nuclear norm minimization problem is also the solution to rank minimization problem.

no code implementations • NeurIPS 2011 • Zhouchen Lin, Risheng Liu, Zhixun Su

It suffers from $O(n^3)$ computation complexity due to the matrix-matrix multiplications and matrix inversions, even if partial SVD is used.

Optimization and Control

no code implementations • 26 Aug 2011 • Risheng Liu, Zhouchen Lin, Siming Wei, Zhixun Su

In this paper, we propose a novel algorithm, called $l_1$ filtering, for \emph{exactly} solving PCP with an $O(r^2(m+n))$ complexity, where $m\times n$ is the size of data matrix and $r$ is the rank of the matrix to recover, which is supposed to be much smaller than $m$ and $n$.

no code implementations • 2 Dec 2010 • Zhouchen Lin, Siming Wei

Recent years have witnessed the popularity of using rank minimization as a regularizer for various signal processing and machine learning problems.

1 code implementation • 14 Oct 2010 • Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, Yi Ma

In this work we address the subspace recovery problem.

no code implementations • 26 Sep 2010 • Zhouchen Lin, Minming Chen, Yi Ma

This paper proposes scalable and fast algorithms for solving the Robust PCA problem, namely recovering a low-rank matrix with an unknown fraction of its entries being arbitrarily corrupted.

Optimization and Control Numerical Analysis Systems and Control

no code implementations • NeurIPS 2009 • Wenming Zheng, Zhouchen Lin

The method of common spatio-spectral patterns (CSSPs) is an extension of common spatial patterns (CSPs) by utilizing the technique of delay embedding to alleviate the adverse effects of noises and artifacts on the electroencephalogram (EEG) classification.

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.