Search Results for author: Fengxiang He

Found 47 papers, 20 papers with code

Decentralized SGD and Average-direction SAM are Asymptotically Equivalent

1 code implementation5 Jun 2023 Tongtian Zhu, Fengxiang He, KaiXuan Chen, Mingli Song, DaCheng Tao

Decentralized stochastic gradient descent (D-SGD) allows collaborative learning on massive devices simultaneously without the control of a central server.

Improving Heterogeneous Model Reuse by Density Estimation

1 code implementation23 May 2023 Anke Tang, Yong Luo, Han Hu, Fengxiang He, Kehua Su, Bo Du, Yixin Chen, DaCheng Tao

This paper studies multiparty learning, aiming to learn a model using the private data of different participants.

Density Estimation Selection bias

Learning to Generalize Provably in Learning to Optimize

1 code implementation22 Feb 2023 Junjie Yang, Tianlong Chen, Mingkang Zhu, Fengxiang He, DaCheng Tao, Yingbin Liang, Zhangyang Wang

While the optimizer generalization has been recently studied, the optimizee generalization (or learning to generalize) has not been rigorously studied in the L2O context, which is the aim of this paper.

Global Nash Equilibrium in Non-convex Multi-player Game: Theory and Algorithms

no code implementations19 Jan 2023 Guanpu Chen, Gehui Xu, Fengxiang He, Yiguang Hong, Leszek Rutkowski, DaCheng Tao

This paper takes conjugate transformation to the formulation of non-convex multi-player games, and casts the complementary problem into a variational inequality (VI) problem with a continuous pseudo-gradient mapping.

Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach

1 code implementation2 Nov 2022 Kaiwen Yang, Yanchao Sun, Jiahao Su, Fengxiang He, Xinmei Tian, Furong Huang, Tianyi Zhou, DaCheng Tao

In experiments, we show that our method consistently brings non-trivial improvements to the three aforementioned learning tasks from both efficiency and final performance, either or not combined with strong pre-defined augmentations, e. g., on medical images when domain knowledge is unavailable and the existing augmentation techniques perform poorly.

Data Augmentation Representation Learning

Benefits of Permutation-Equivariance in Auction Mechanisms

no code implementations11 Oct 2022 Tian Qin, Fengxiang He, Dingfeng Shi, Wenbing Huang, DaCheng Tao

Designing an incentive-compatible auction mechanism that maximizes the auctioneer's revenue while minimizes the bidders' ex-post regret is an important yet intricate problem in economics.

Bridged Transformer for Vision and Point Cloud 3D Object Detection

1 code implementation CVPR 2022 Yikai Wang, TengQi Ye, Lele Cao, Wenbing Huang, Fuchun Sun, Fengxiang He, DaCheng Tao

Recently, there is a trend of leveraging multiple sources of input data, such as complementing the 3D point cloud with 2D images that often have richer color and fewer noises.

3D Object Detection object-detection

InFi: End-to-End Learning to Filter Input for Resource-Efficiency in Mobile-Centric Inference

1 code implementation28 Sep 2022 Mu Yuan, Lan Zhang, Fengxiang He, Xueting Tong, Miao-Hui Song, Zhengyuan Xu, Xiang-Yang Li

Previous efforts have tailored effective solutions for many applications, but left two essential questions unanswered: (1) theoretical filterability of an inference workload to guide the application of input filtering techniques, thereby avoiding the trial-and-error cost for resource-constrained mobile applications; (2) robust discriminability of feature embedding to allow input filtering to be widely effective for diverse inference tasks and input content.

Super-model ecosystem: A domain-adaptation perspective

no code implementations30 Aug 2022 Fengxiang He, DaCheng Tao

We model the super-model paradigm as a two-stage diffusion process: (1) in the pre-training stage, the model parameter diffuses from random initials and converges to a steady distribution; and (2) in the fine-tuning stage, the model parameter is transported to another steady distribution.

Domain Adaptation

Topology-aware Generalization of Decentralized SGD

1 code implementation25 Jun 2022 Tongtian Zhu, Fengxiang He, Lan Zhang, Zhengyang Niu, Mingli Song, DaCheng Tao

Our theory indicates that the generalizability of D-SGD is positively correlated with the spectral gap, and can explain why consensus control in initial training phase can ensure better generalization.

Understanding deep learning via decision boundary

no code implementations3 Jun 2022 Shiye Lei, Fengxiang He, Yancheng Yuan, DaCheng Tao

From the theoretical view, two lower bounds based on algorithm DB variability are proposed and do not explicitly depend on the sample size.

DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training

1 code implementation1 Jun 2022 Rong Dai, Li Shen, Fengxiang He, Xinmei Tian, DaCheng Tao

In this work, we propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL, which employs personalized sparse masks to customize sparse local models on the edge.

Personalized Federated Learning

Robust Unlearnable Examples: Protecting Data Against Adversarial Learning

1 code implementation28 Mar 2022 Shaopeng Fu, Fengxiang He, Yang Liu, Li Shen, DaCheng Tao

To address this concern, methods are proposed to make data unlearnable for deep learning models by adding a type of error-minimizing noise.

Exploring High-Order Structure for Robust Graph Structure Learning

no code implementations22 Mar 2022 Guangqian Yang, Yibing Zhan, Jinlong Li, Baosheng Yu, Liu Liu, Fengxiang He

In this paper, we analyze the adversarial attack on graphs from the perspective of feature smoothness which further contributes to an efficient new adversarial defensive algorithm for GNNs.

Adversarial Attack Graph structure learning +1

Achieving Personalized Federated Learning with Sparse Local Models

no code implementations27 Jan 2022 Tiansheng Huang, Shiwei Liu, Li Shen, Fengxiang He, Weiwei Lin, DaCheng Tao

To counter this issue, personalized FL (PFL) was proposed to produce dedicated local models for each individual user.

Personalized Federated Learning

Text-to-Image Synthesis Based on Object-Guided Joint-Decoding Transformer

no code implementations CVPR 2022 Fuxiang Wu, Liu Liu, Fusheng Hao, Fengxiang He, Jun Cheng

Object-guided text-to-image synthesis aims to generate images from natural language descriptions built by two-step frameworks, i. e., the model generates the layout and then synthesizes images from the layout and captions.

Image Generation

Siamese Network with Interactive Transformer for Video Object Segmentation

1 code implementation28 Dec 2021 Meng Lan, Jing Zhang, Fengxiang He, Lefei Zhang

Semi-supervised video object segmentation (VOS) refers to segmenting the target object in remaining frames given its annotation in the first frame, which has been actively studied in recent years.

Semantic Segmentation Semi-Supervised Video Object Segmentation +1

Visual Semantics Allow for Textual Reasoning Better in Scene Text Recognition

1 code implementation AAAI 2022 2021 Yue He, Chen Chen, Jing Zhang, Juhua Liu, Fengxiang He, Chaoyue Wang, Bo Du

Technically, given the character segmentation maps predicted by a VR model, we construct a subgraph for each instance, where nodes represent the pixels in it and edges are added between nodes based on their spatial similarity.

Ranked #3 on Scene Text Recognition on SVTP (using extra training data)

Language Modelling Scene Text Recognition

Self-Ensembling GAN for Cross-Domain Semantic Segmentation

1 code implementation15 Dec 2021 Yonghao Xu, Fengxiang He, Bo Du, DaCheng Tao, Liangpei Zhang

In SE-GAN, a teacher network and a student network constitute a self-ensembling model for generating semantic segmentation maps, which together with a discriminator, forms a GAN.

Semantic Segmentation

Spatial-Temporal-Fusion BNN: Variational Bayesian Feature Layer

no code implementations12 Dec 2021 Shiye Lei, Zhuozhuo Tu, Leszek Rutkowski, Feng Zhou, Li Shen, Fengxiang He, DaCheng Tao

Bayesian neural networks (BNNs) have become a principal approach to alleviate overconfident predictions in deep learning, but they often suffer from scaling issues due to a large number of distribution parameters.

Adversarial Robustness Variational Inference

Spectral Complexity-scaled Generalization Bound of Complex-valued Neural Networks

no code implementations7 Dec 2021 Haowen Chen, Fengxiang He, Shiye Lei, DaCheng Tao

The bound scales with the spectral complexity, the dominant factor of which is the spectral norm product of weight matrices.


Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction

1 code implementation4 Dec 2021 Yikai Wang, Fuchun Sun, Wenbing Huang, Fengxiang He, DaCheng Tao

For the application of dense image prediction, the validity of CEN is tested by four different scenarios: multimodal fusion, cycle multimodal fusion, multitask learning, and multimodal multitask learning.

Semantic Segmentation

Decision boundary variability and generalization in neural networks

no code implementations29 Sep 2021 Shiye Lei, Fengxiang He, Yancheng Yuan, DaCheng Tao

Two new notions, algorithm DB variability and $(\epsilon, \eta)$-data DB variability, are proposed to measure the decision boundary variability from the algorithm and data perspectives.

Generalizable Learning to Optimize into Wide Valleys

no code implementations29 Sep 2021 Junjie Yang, Tianlong Chen, Mingkang Zhu, Fengxiang He, DaCheng Tao, Yingbin Liang, Zhangyang Wang

Learning to optimize (L2O) has gained increasing popularity in various optimization tasks, since classical optimizers usually require laborious, problem-specific design and hyperparameter tuning.

Huber Additive Models for Non-stationary Time Series Analysis

no code implementations ICLR 2022 Yingjie Wang, Xianrui Zhong, Fengxiang He, Hong Chen, DaCheng Tao

Moreover, the error bound for non-stationary time series contains a discrepancy measure for the shifts of the data distributions over time.

Additive models Causal Discovery +3

Robust Unlearnable Examples: Protecting Data Privacy Against Adversarial Learning

no code implementations ICLR 2022 Shaopeng Fu, Fengxiang He, Yang Liu, Li Shen, DaCheng Tao

To address this concern, methods are proposed to make data unlearnable for deep learning models by adding a type of error-minimizing noise.

On Heterogeneously Distributed Data, Sparsity Matters

no code implementations29 Sep 2021 Tiansheng Huang, Shiwei Liu, Li Shen, Fengxiang He, Weiwei Lin, DaCheng Tao

Federated learning (FL) is particularly vulnerable to heterogeneously distributed data, since a common global model in FL may not adapt to the heterogeneous data distribution of each user.

Personalized Federated Learning

Bayesian Inference Forgetting

1 code implementation16 Jan 2021 Shaopeng Fu, Fengxiang He, Yue Xu, DaCheng Tao

This paper proposes a {\it Bayesian inference forgetting} (BIF) framework to realize the right to be forgotten in Bayesian inference.

Bayesian Inference Variational Inference

Neural networks behave as hash encoders: An empirical study

1 code implementation14 Jan 2021 Fengxiang He, Shiye Lei, Jianmin Ji, DaCheng Tao

We then define an {\it activation hash phase chart} to represent the space expanded by {model size}, training time, training sample size, and the encoding properties, which is divided into three canonical regions: {\it under-expressive regime}, {\it critically-expressive regime}, and {\it sufficiently-expressive regime}.

Robustness, Privacy, and Generalization of Adversarial Training

1 code implementation25 Dec 2020 Fengxiang He, Shaopeng Fu, Bohan Wang, DaCheng Tao

This measure can be approximate empirically by an asymptotically consistent empirical estimator, {\it empirical robustified intensity}.

Generalization Bounds Privacy Preserving

Recent advances in deep learning theory

no code implementations20 Dec 2020 Fengxiang He, DaCheng Tao

Deep learning is usually described as an experiment-driven field under continuous criticizes of lacking theoretical foundations.

Bayesian Inference Ethics +1

Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting

1 code implementation12 Nov 2020 Zeke Xie, Fengxiang He, Shaopeng Fu, Issei Sato, DaCheng Tao, Masashi Sugiyama

Thus it motivates us to design a similar mechanism named {\it artificial neural variability} (ANV), which helps artificial neural networks learn some advantages from ``natural'' neural networks.


Tighter Generalization Bounds for Iterative Differentially Private Learning Algorithms

no code implementations18 Jul 2020 Fengxiang He, Bohan Wang, DaCheng Tao

This paper studies the relationship between generalization and privacy preservation in iterative learning algorithms by two sequential steps.

Federated Learning Generalization Bounds

Understanding Generalization in Recurrent Neural Networks

no code implementations ICLR 2020 Zhuozhuo Tu, Fengxiang He, DaCheng Tao

We first present a new generalization bound for recurrent neural networks based on matrix 1-norm and Fisher-Rao norm.

Generalization Bounds LEMMA

Piecewise linear activations substantially shape the loss surfaces of neural networks

no code implementations ICLR 2020 Fengxiang He, Bohan Wang, DaCheng Tao

This result holds for any neural network with arbitrary depth and arbitrary piecewise linear activation functions (excluding linear functions) under most loss functions in practice.

Control Batch Size and Learning Rate to Generalize Well: Theoretical and Empirical Evidence

no code implementations NeurIPS 2019 Fengxiang He, Tongliang Liu, DaCheng Tao

Specifically, we prove a PAC-Bayes generalization bound for neural networks trained by SGD, which has a positive correlation with the ratio of batch size to learning rate.

Why ResNet Works? Residuals Generalize

no code implementations2 Apr 2019 Fengxiang He, Tongliang Liu, DaCheng Tao

This paper studies the influence of residual connections on the hypothesis complexity of the neural network in terms of the covering number of its hypothesis space.

Instance-Dependent PU Learning by Bayesian Optimal Relabeling

no code implementations7 Aug 2018 Fengxiang He, Tongliang Liu, Geoffrey I. Webb, DaCheng Tao

Specifically, by treating the unlabelled data as noisy negative examples, we could automatically label a group positive and negative examples whose labels are identical to the ones assigned by a Bayesian optimal classifier with a consistency guarantee.

Cannot find the paper you are looking for? You can Submit a new open access paper.