Search Results for author:
Found 4 papers, 0 papers with code
Train Where the Data is: A Case for Bandwidth Efficient Coded Training
Most of the model training is performed on high performance compute nodes and the training data is stored near these nodes for faster training.
GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Training
Data parallelism can boost the training speed of convolutional neural networks (CNN), but could suffer from significant communication costs caused by gradient aggregation.
Pipe-SGD: A Decentralized Pipelined SGD Framework for Distributed Deep Net Training
Distributed training of deep nets is an important technique to address some of the present day computing challenges like memory consumption and computational demands.
PolyShard: Coded Sharding Achieves Linearly Scaling Efficiency and Security Simultaneously
In particular, we propose PolyShard: ``polynomially coded sharding'' scheme that achieves information-theoretic upper bounds on the efficiency of the storage, system throughput, as well as on trust, thus enabling a truly scalable system.
Cryptography and Security Distributed, Parallel, and Cluster Computing Information Theory Information Theory
