Search Results for author:
Found 3 papers, 1 papers with code
Exploring Low Rank Training of Deep Neural Networks
Training deep neural networks in low rank, i. e. with factorised layers, is of particular interest to the community: it offers efficiency over unfactorised training in terms of both memory consumption and training time.
Scalable Training of Language Models using JAX pjit and TPUv4
Modern large language models require distributed training strategies due to their size.
Learning Sparse Networks Using Targeted Dropout
Before computing the gradients for each weight update, targeted dropout stochastically selects a set of units or weights to be dropped using a simple self-reinforcing sparsity criterion and then computes the gradients for the remaining weights.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
