Search Results for author:
Found 18 papers, 4 papers with code
Pretraining Billion-scale Geospatial Foundational Models on Frontier
Although large FMs have demonstrated significant impact in natural language processing and computer vision, efforts toward FMs for geospatial applications have been restricted to smaller size models, as pretraining larger models requires very large computing resources equipped with state-of-the-art hardware accelerators.
The Case for Co-Designing Model Architectures with Hardware
While GPUs are responsible for training the vast majority of state-of-the-art deep learning models, the implications of their architecture are often overlooked when designing new deep learning (DL) models.
Optimizing Distributed Training on Frontier for Large Language Models
For the training of the 175 Billion parameter model and the 1 Trillion parameter model, we achieved $100\%$ weak scaling efficiency on 1024 and 3072 MI250X GPUs, respectively.
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences.
Atomic structure generation from reconstructing structural fingerprints
These methods would help identify or, in the case of generative models, even create novel crystal structures of materials with a set of specified functional properties to then be synthesized or isolated in the laboratory.
Stable Parallel Training of Wasserstein Conditional Generative Adversarial Neural Networks
We propose a stable, parallel approach to train Wasserstein Conditional Generative Adversarial Neural Networks (W-CGANs) under the constraint of a fixed computational budget.
Stable Anderson Acceleration for Deep Learning
Anderson acceleration (AA) is an extrapolation technique designed to speed-up fixed-point iterations like those arising from the iterative training of DL models.
MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems
Scientific communities are increasingly adopting machine learning and deep learning models in their applications to accelerate scientific insights.
Neural network based order parameter for phase transitions and its applications in high-entropy alloys
We propose that the Manhattan distance in the VAE latent space can serve as a generic order parameter for order-disorder phase transitions.
Scalable Balanced Training of Conditional Generative Adversarial Neural Networks on Image Data
We propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models.
Data optimization for large batch distributed training of deep neural networks
Distributed training in deep learning (DL) is common practice as data and models grow.
Distributed Training and Optimization Of Neural Networks
Deep learning models are yielding increasingly better performances thanks to multiple factors.
Integrating Deep Learning in Domain Sciences at Exascale
This paper discusses the necessities of an HPC deep learning framework and how those needs can be provided (e. g., as in MagmaDNN) through a deep integration with existing HPC libraries, such as MAGMA and its modular memory management, MPI, CuBLAS, CuDNN, MKL, and HIP.
Exascale Deep Learning for Scientific Inverse Problems
We introduce novel communication strategies in synchronous distributed Deep Learning consisting of decentralized gradient reduction orchestration and computational graph-aware grouping of gradient tensors.
Y-net: A Physics-constrained and Semi-supervised Learning Approach to the Phase Problem in Computational Electron Imaging
The phase problem in diffraction physics is one of the oldest inverse problems in all of science.
A scalable constructive algorithm for the optimization of neural network architectures
We propose a new scalable method to optimize the architecture of an artificial neural network.
Robust data-driven approach for predicting the configurational energy of high entropy alloys
In this study, a robust data-driven framework based on Bayesian approaches is proposed and demonstrated on the accurate and efficient prediction of configurational energy of high entropy alloys.
Defining Big Data Analytics Benchmarks for Next Generation Supercomputers
The design and construction of high performance computing (HPC) systems relies on exhaustive performance analysis and benchmarking.
Performance
