Search Results for author:
Found 9 papers, 0 papers with code
Optimizing Distributed Training on Frontier for Large Language Models
For the training of the 175 Billion parameter model and the 1 Trillion parameter model, we achieved $100\%$ weak scaling efficiency on 1024 and 3072 MI250X GPUs, respectively.
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences.
Prediction of $\textrm{CO}_2$ Adsorption in Nano-Pores with Graph Neural Networks
We investigate the graph-based convolutional neural network approach for predicting and ranking gas adsorption properties of crystalline Metal-Organic Framework (MOF) adsorbents for application in post-combustion capture of $\textrm{CO}_2$.
AI-aided multiscale modeling of physiologically-significant blood clots
We have developed an AI-aided multiple time stepping (AI-MTS) algorithm and multiscale modeling framework (AI-MSM) and implemented them on the Summit-like supercomputer, AIMOS.
Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum
We propose a momentum method for such model averaging approaches.
CASTELO: Clustered Atom Subtypes aidEd Lead Optimization -- a combined machine learning and molecular modeling method
Drug discovery is a multi-stage process that comprises two costly major steps: pre-clinical research and clinical trials.
Accelerating Data Loading in Deep Neural Network Training
Our model suggests that I/O rate limits the scalability of distributed training, which inspires us to design a locality-aware data loading method.
A Distributed Hierarchical SGD Algorithm with Sparse Global Reduction
Reducing communication in training large-scale machine learning applications on distributed platform is still a big challenge.
On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization
We establish the convergence results of K-AVG for nonconvex objectives and explain why the K-step delay is necessary and leads to better performance than traditional parallel stochastic gradient descent which is a special case of K-AVG with $K=1$.
