Search Results for author: Yefan Zhou

Found 5 papers, 4 papers with code

Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance

no code implementations17 Jul 2024 Haiquan Lu, Xiaotian Liu, Yefan Zhou, Qunli Li, Kurt Keutzer, Michael W. Mahoney, Yujun Yan, Huanrui Yang, Yaoqing Yang

We discover a trade-off between sharpness and diversity: minimizing the sharpness in the loss landscape tends to diminish the diversity of individual members within the ensemble, adversely affecting the ensemble's improvement.

Diversity

MD tree: a model-diagnostic tree grown on loss landscape

1 code implementation24 Jun 2024 Yefan Zhou, Jianlong Chen, Qinxue Cao, Konstantin Schürholt, Yaoqing Yang

This paper considers "model diagnosis", which we formulate as a classification problem.

Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training

1 code implementation NeurIPS 2023 Yefan Zhou, Tianyu Pang, Keqin Liu, Charles H. Martin, Michael W. Mahoney, Yaoqing Yang

In particular, the learning rate, which can be interpreted as a temperature-like parameter within the statistical mechanics of learning, plays a crucial role in neural network training.

Scheduling

A Three-regime Model of Network Pruning

1 code implementation28 May 2023 Yefan Zhou, Yaoqing Yang, Arin Chang, Michael W. Mahoney

Our approach uses temperature-like and load-like parameters to model the impact of neural network (NN) training hyperparameters on pruning performance.

Efficient Neural Network Hyperparameter Optimization +1

Cannot find the paper you are looking for? You can Submit a new open access paper.