Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization

no code implementations • 12 Jun 2024 • Yuhang Cai, Jingfeng Wu, Song Mei, Michael Lindsey, Peter L. Bartlett

The typical training of neural networks using large stepsize gradient descent (GD) under the logistic loss often involves two distinct phases, where the empirical risk oscillates in the first phase but decreases monotonically in the second phase.

Multi-band oscillations emerge from a simple spiking network

no code implementations • 29 Jun 2022 • Tianyi Wu, Yuhang Cai, Ruilin Zhang, Zhongyi Wang, Louis Tao, Zhuo-Cheng Xiao

These results suggest a simple geometric mechanism behind the emergence of multi-band oscillations without appealing to oscillatory inputs or multiple synaptic or neuronal timescales.

Model Reduction Captures Stochastic Gamma Oscillations on Low-Dimensional Manifolds

no code implementations • 5 Jan 2021 • Yuhang Cai, Tianyi Wu, Louis Tao, Zhuo-Cheng Xiao

Here we propose a suite of Markovian model reduction methods with varying levels of complexity and applied it to spiking network models exhibiting heterogeneous dynamical regimes, ranging from homogeneous firing to strong synchrony in the gamma band.