Search Results for author:
Found 4 papers, 2 papers with code
Reduce, Reuse, Recycle: Improving Training Efficiency with Distillation
We conducted a series of experiments to investigate whether and how distillation can be used to accelerate training using ResNet-50 trained on ImageNet and BERT trained on C4 with a masked language modeling objective and evaluated on GLUE, using common enterprise hardware (8x NVIDIA A100).
Measure Twice, Cut Once: Quantifying Bias and Fairness in Deep Neural Networks
Algorithmic bias is of increasing concern, both to the research community, and society at large.
Simon Says: Evaluating and Mitigating Bias in Pruned Neural Networks with Knowledge Distillation
In recent years the ubiquitous deployment of AI has posed great concerns in regards to algorithmic bias, discrimination, and fairness.
Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression
The experimental results running on an AMD server with four Geforce RTX 2080Ti GPUs show that our algorithm can achieve 3x speedup plus 19% energy savings on VGG distillation, and 3. 5x speedup plus 29% energy savings on ResNet distillation, both with negligible accuracy loss.
