Search Results for author:
Found 12 papers, 3 papers with code
Wukong: Towards a Scaling Law for Large-Scale Recommendation
Scaling laws play an instrumental role in the sustainable improvement in model quality.
Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendation
We study a mismatch between the deep learning recommendation models' flat architecture, common distributed training paradigm and hierarchical data center topology.
Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models
In this work, we explore a "pre-train, and search" paradigm for efficient sharding.
Self-discipline on multiple channels
Comparative experimental results on both datasets show that SMC-2 outperforms Label Smoothing Regularizaion and Self-distillation From The Last Mini-batch on all models, and outperforms the state-of-the-art Sharpness-Aware Minimization method on 83% of the models. Compatibility of SMC-2 and data augmentation experimental results show that using both SMC-2 and data augmentation improves the generalization ability of the model between 0. 28% and 1. 80% compared to using only data augmentation.
PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
It is widely acknowledged that large models have the potential to deliver superior performance across a broad range of domains.
DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction
To overcome the challenge brought by DHEN's deeper and multi-layer structure in training, we propose a novel co-designed training system that can further improve the training efficiency of DHEN.
Characterizing and Taming Resolution in Convolutional Neural Networks
Image resolution has a significant effect on the accuracy and computational, storage, and bandwidth costs of computer vision model inference.
Cloud Collectives: Towards Cloud-aware Collectives forML Workloads with Rank Reordering
ML workloads are becoming increasingly popular in the cloud.
Accelerating SpMM Kernel with Cache-First Edge Sampling for Graph Neural Networks
To evaluate ES-SpMM's performance, we integrated it with a popular GNN framework, DGL, and tested it using representative GNN models and datasets.
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers.
Parameter Hub: a Rack-Scale Parameter Server for Distributed Deep Neural Network Training
Distributed deep neural network (DDNN) training constitutes an increasingly important workload that frequently runs in the cloud.
