Search Results for author: Tiancong Chen

Found 5 papers, 1 papers with code

Stability Based Generalization Bounds for Exponential Family Langevin Dynamics

no code implementations9 Jan 2022 Arindam Banerjee, Tiancong Chen, Xinyan Li, Yingxue Zhou

Recent years have seen advances in generalization bounds for noisy stochastic algorithms, especially stochastic gradient Langevin dynamics (SGLD) based on stability (Mou et al., 2018; Li et al., 2020) and information theoretic approaches (Xu and Raginsky, 2017; Negrea et al., 2019; Steinke and Zakynthinou, 2020).

Generalization Bounds

Early Stopping for Deep Image Prior

1 code implementation11 Dec 2021 Hengkang Wang, Taihui Li, Zhong Zhuang, Tiancong Chen, Hengyue Liang, Ju Sun

In this regard, the majority of DIP works for vision tasks only demonstrates the potential of the models -- reporting the peak performance against the ground truth, but provides no clue about how to operationally obtain near-peak performance without access to the groundtruth.

De-randomized PAC-Bayes Margin Bounds: Applications to Non-convex and Non-smooth Predictors

no code implementations23 Feb 2020 Arindam Banerjee, Tiancong Chen, Yingxue Zhou

Existing approaches for deterministic non-smooth deep nets typically need to bound the Lipschitz constant of such deep nets but such bounds are quite large, may even increase with the training set size yielding vacuous generalization bounds.

Generalization Bounds

Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization

no code implementations24 Jul 2019 Xinyan Li, Qilong Gu, Yingxue Zhou, Tiancong Chen, Arindam Banerjee

(2) how can we characterize the stochastic optimization dynamics of SGD with fixed and adaptive step sizes and diagonal pre-conditioning based on the first and second moments of SGs?

Stochastic Optimization

Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms

no code implementations NeurIPS 2020 Xiangyi Chen, Tiancong Chen, Haoran Sun, Zhiwei Steven Wu, Mingyi Hong

We show that these algorithms are non-convergent whenever there is some disparity between the expected median and mean over the local gradients.

Federated Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.