Search Results for author:
Found 5 papers, 3 papers with code
Adaptive Elastic Training for Sparse Deep Learning on Heterogeneous Multi-GPU Servers
We address these challenges with Adaptive SGD, an adaptive elastic model averaging stochastic gradient descent algorithm for heterogeneous multi-GPUs that is characterized by dynamic scheduling, adaptive batch size scaling, and normalized model merging.
Embedded Self-Distillation in Compact Multi-Branch Ensemble Network for Remote Sensing Scene Classification
On this issue, we embed self-distillation (SD) method to transfer knowledge from ensemble network to main-branch in it.
MGML: Multi-Granularity Multi-Level Feature Ensemble Network for Remote Sensing Scene Classification
To avoid the interference from confusing information, we propose Multi-granularity Multi-Level Feature Ensemble Module (MGML-FEM) which can provide diverse predictions by full-channel feature generator (FC-FG).
Heterogeneous CPU+GPU Stochastic Gradient Descent Algorithms
In order to allow for a principled exploration of the design space, we first introduce a generic deep learning framework that exploits the difference in computational power and memory hierarchy between CPU and GPU through asynchronous message passing.
Stochastic Gradient Descent on Highly-Parallel Architectures
The choice between synchronous GPU and asynchronous CPU depends on the task and the characteristics of the data.
