Search Results for author:
Found 3 papers, 1 papers with code
Cuttlefish: Low-Rank Model Training without All the Tuning
Cuttlefish leverages the observation that after a few epochs of full-rank training, the stable rank (i. e., an approximation of the true rank) of each layer stabilizes at a constant value.
MRL: Learning to Mix with Attention and Convolutions
To achieve an efficient mix, we exploit the domain-wide receptive field provided by self-attention for regional-scale mixing and convolutional kernels restricted to local scale for local-scale mixing.
Ranked #1 on
Multi-tissue Nucleus Segmentation
on CoNSeP
Massively Distributed SGD: ImageNet/ResNet-50 Training in a Flash
Scaling the distributed deep learning to a massive GPU cluster level is challenging due to the instability of the large mini-batch training and the overhead of the gradient synchronization.
