Search Results for author:
Found 63 papers, 10 papers with code
ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability
As the largest model of its kind, ORBIT surpasses the current climate AI foundation model size by a thousandfold.
Network architecture search of X-ray based scientific applications
Our NAS and HPS of (1) BraggNN achieves a 31. 03\% improvement in bragg peak detection accuracy with a 87. 57\% reduction in model size, and (2) PtychoNN achieves a 16. 77\% improvement in model accuracy and a 12. 82\% reduction in model size when compared to the baseline PtychoNN model.
AI Competitions and Benchmarks: Dataset Development
Initially, we develop the tasks involved in dataset development and offer insights into their effective management (including requirements, design, implementation, evaluation, distribution, and maintenance).
Streamlining Ocean Dynamics Modeling with Fourier Neural Operators: A Multiobjective Hyperparameter and Architecture Optimization Approach
The experimental results show that the optimal set of hyperparameters enhanced model performance in single timestepping forecasting and greatly exceeded the baseline configuration in the autoregressive rollout for long-horizon forecasting up to 30 days.
The Unreasonable Effectiveness Of Early Discarding After One Epoch In Neural Network Hyperparameter Optimization
To reach high performance with deep learning, hyperparameter optimization (HPO) is essential.
Transfer-Learning-Based Autotuning Using Gaussian Copula
We introduce the first generative TL-based autotuning approach based on the Gaussian copula (GC) to model the high-performing regions of the search space from prior data and then generate high-performing configurations for new tasks.
Optimizing Distributed Training on Frontier for Large Language Models
For the training of the 175 Billion parameter model and the 1 Trillion parameter model, we achieved $100\%$ weak scaling efficiency on 1024 and 3072 MI250X GPUs, respectively.
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences.
Self-supervised Learning for Anomaly Detection in Computational Workflows
To address this problem, we introduce an autoencoder-driven self-supervised learning~(SSL) approach that learns a summary statistic from unlabeled workflow data and estimates the normal behavior of the computational workflow in the latent space.
Parallel Multi-Objective Hyperparameter Optimization with Uniform Normalization and Bounded Objectives
Machine learning (ML) methods offer a wide range of configurable hyperparameters that have a significant influence on their performance.
Comparing Llama-2 and GPT-3 LLMs for HPC kernels generation
We evaluate the use of the open-source Llama-2 model for generating well-known, high-performance computing kernels (e. g., AXPY, GEMV, GEMM) on different parallel programming models and languages (e. g., C++: OpenMP, OpenMP Offload, OpenACC, CUDA, HIP; Fortran: OpenMP, OpenMP Offload, OpenACC; Python: numpy, Numba, pyCUDA, cuPy; and Julia: Threads, CUDA. jl, AMDGPU. jl).
Improving Performance in Continual Learning Tasks using Bio-Inspired Architectures
The ability to learn continuously from an incoming data stream without catastrophic forgetting is critical to designing intelligent systems.
Is One Epoch All You Need For Multi-Fidelity Hyperparameter Optimization?
Hyperparameter optimization (HPO) is crucial for fine-tuning machine learning models but can be computationally expensive.
Uncertainty Quantification for Molecular Property Predictions with Graph Neural Architecture Search
To that end, we introduce AutoGNNUQ, an automated uncertainty quantification (UQ) approach for molecular property prediction.
Evaluation of OpenAI Codex for HPC Parallel Programming Models Kernel Generation
We evaluate AI-assisted generative capabilities on fundamental numerical kernels in high-performance computing (HPC), including AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG.
Learning Continually on a Sequence of Graphs -- The Dynamical System Way
However, the literature is quite sparse, when the data corresponding to a CL task is nonEuclidean-- data , such as graphs, point clouds or manifold, where the notion of similarity in the sense of Euclidean metric does not hold.
ytopt: Autotuning Scientific Applications for Energy Efficiency at Large Scales
As we enter the exascale computing era, efficiently utilizing power and optimizing the performance of scientific applications under power and energy constraints has become critical and challenging.
Application of probabilistic modeling and automated machine learning framework for high-dimensional stress field
Modern computational methods, involving highly sophisticated mathematical formulations, enable several tasks like modeling complex physical phenomenon, predicting key properties and design optimization.
Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles
We demonstrate the feasibility of this framework for two tasks - forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature.
Analyzing the impact of climate change on critical infrastructure from the scientific literature: A weakly supervised NLP approach
In comparison with a months-long process of subject-matter expert labeling, we assign category labels to the whole corpus using weak supervision and supervised learning in about 13 hours.
Unified Probabilistic Neural Architecture and Weight Ensembling Improves Model Robustness
To this end, we propose a Unified probabilistic architecture and weight ensembling Neural Architecture Search (UraeNAS) that leverages advances in probabilistic neural architecture search and approximate Bayesian inference to generate ensembles form the joint distribution of neural network architectures and weights.
HPC Storage Service Autotuning Using Variational-Autoencoder-Guided Asynchronous Bayesian Optimization
Distributed data storage services tailored to specific applications have grown popular in the high-performance computing (HPC) community as a way to address I/O and storage challenges.
Explainable Graph Pyramid Autoformer for Long-Term Traffic Forecasting
Moreover, most of the existing deep learning traffic forecasting models are black box, presenting additional challenges related to explainability and interpretability.
Asynchronous Decentralized Bayesian Optimization for Large Scale Hyperparameter Optimization
Bayesian optimization (BO) is a promising approach for hyperparameter optimization of deep neural networks (DNNs), where each model training can take minutes to hours.
Multifidelity Reinforcement Learning with Control Variates
The proposed estimator, which is based on the method of control variates, is used to design a multifidelity Monte Carlo RL (MFMCRL) algorithm that improves the learning of the agent in the high-fidelity environment.
Sequential Bayesian Neural Subnetwork Ensembles
Deep neural network ensembles that appeal to model diversity have been used successfully to improve predictive performance and model robustness in several applications.
Deep-Ensemble-Based Uncertainty Quantification in Spatiotemporal Graph Neural Networks for Traffic Forecasting
Our approach uses a scalable Bayesian optimization method to perform hyperparameter optimization, selects a set of high-performing configurations, fits a generative model to capture the joint distributions of the hyperparameter configurations, and trains an ensemble of models by sampling a new set of hyperparameter configurations from the generative model.
Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems
We present a data-driven modeling method that accurately captures shocks and chaotic dynamics by proposing a novel architecture, stabilized neural ordinary differential equation (ODE).
Sparsity-Inducing Categorical Prior Improves Robustness of the Information Bottleneck
In addition, it provides a mechanism for learning a joint distribution of the latent variable and the sparsity and hence can account for the complete uncertainty in the latent space.
Multi-fidelity reinforcement learning framework for shape optimization
One key limitation of conventional DRL methods is their episode-hungry nature which proves to be a bottleneck for tasks which involve costly evaluations of a numerical forward model.
A data-centric weak supervised learning for highway traffic incident detection
To that end, we focus on a data-centric approach to improve the accuracy and reduce the false alarm rate of traffic incident detection on highways.
Modeling Design and Control Problems Involving Neural Network Surrogates
We consider nonlinear optimization problems that involve surrogate models represented by neural networks.
AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification
However, building ensembles of neural networks is a challenging task because, in addition to choosing the right neural architecture or hyperparameters for each member of the ensemble, there is an added cost of training each model.
AutoPhaseNN: Unsupervised Physics-aware Deep Learning of 3D Nanoscale Bragg Coherent Diffraction Imaging
The problem of phase retrieval, or the algorithmic recovery of lost phase information from measured intensity alone, underlies various imaging methods from astronomy to nanoscale imaging.
Formalizing the Generalization-Forgetting Trade-off in Continual Learning
We formulate the continual learning (CL) problem via dynamic programming and model the trade-off between catastrophic forgetting and generalization as a two-player sequential game.
Customized Monte Carlo Tree Search for LLVM/Polly's Composable Loop Optimization Transformations
The search space exposed by the transformation pragmas is a tree, wherein each node represents a specific combination of loop transformations that can be applied to the code resulting from the parent node's loop transformations.
Autotuning PolyBench Benchmarks with LLVM Clang/Polly Loop Optimization Pragmas Using Bayesian Optimization (extended version)
In this paper, we develop a ytopt autotuning framework that leverages Bayesian optimization to explore the parameter space search and compare four different supervised learning methods within Bayesian optimization and evaluate their effectiveness.
A cross-study analysis of drug response prediction in cancer cell lines
To provide a more rigorous assessment of model generalizability between different studies, we use machine learning to analyze five publicly available cell line-based data sets: NCI60, CTRP, GDSC, CCLE and gCSI.
Learning Symbolic Expressions: Mixed-Integer Formulations, Cuts, and Heuristics
This problem is referred to as symbolic regression.
Data-Driven Random Access Optimization in Multi-Cell IoT Networks with NOMA
To enable a capacity-optimal network, a novel formulation of random channel access management is proposed, in which the transmission probability of each IoT device is tuned to maximize the geometric mean of users' expected capacity.
Meta-Continual Learning Via Dynamic Programming
Meta continual learning algorithms seek to train a model when faced with similar tasks observed in a sequential manner.
AgEBO-Tabular: Joint Neural Architecture and Hyperparameter Search with Autotuned Data-Parallel Training for Tabular Data
Neural architecture search (NAS) is an AutoML approach that generates and evaluates multiple neural network architectures concurrently and improves the accuracy of the generated models iteratively.
Autotuning PolyBench Benchmarks with LLVM Clang/Polly Loop Optimization Pragmas Using Bayesian Optimization
An autotuning is an approach that explores a search space of possible implementations/configurations of a kernel or an application by selecting and evaluating a subset of implementations/configurations on a target platform and/or use models to identify a high performance implementation/configuration.
Dynamic Graph Neural Network for Traffic Forecasting in Wide Area Networks
Wide area networking infrastructures (WANs), particularly science and research WANs, are the backbone for moving large volumes of scientific data between experimental facilities and data centers.
Graph Neural Network Architecture Search for Molecular Property Prediction
Neural architecture search (NAS) is a promising approach to discover high-performing neural network architectures automatically.
Meta Continual Learning via Dynamic Programming
Meta continual learning algorithms seek to train a model when faced with similar tasks observed in a sequential manner.
Neuromodulated Neural Architectures with Local Error Signals for Memory-Constrained Online Continual Learning
Using high performing configurations metalearned in the single task learning setting, we achieve superior continual learning performance on Split-MNIST, and Split-CIFAR-10 data as compared with other memory-constrained learning approaches, and match that of the state-of-the-art memory-intensive replay-based approaches.
Multilayer Neuromodulated Architectures for Memory-Constrained Online Continual Learning
We focus on the problem of how to achieve online continual learning under memory-constrained conditions where the input data may not be known \emph{a priori}.
A Gradient-Aware Search Algorithm for Constrained Markov Decision Processes
In this brief, we first prove that the optimization objective in the dual linear program of a finite CMDP is a piece-wise linear convex function (PWLC) with respect to the Lagrange penalty multipliers.
Towards On-Chip Bayesian Neuromorphic Learning
e-prop 1 is a promising learning algorithm that tackles this with Broadcast Alignment (a technique where network weights are replaced with random weights during feedback) and accumulated local information.
Transfer Learning with Graph Neural Networks for Short-Term Highway Traffic Forecasting
To that end, we develop a new transfer learning approach for DCRNN, where a single model trained on data-rich regions of the highway network can be used to forecast traffic on unseen regions of the highway network.
Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV based Random Access IoT Networks with NOMA
In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers.
Learning to Optimize Variational Quantum Circuits to Solve Combinatorial Problems
Proposed recently, the Quantum Approximate Optimization Algorithm (QAOA) is considered as one of the leading candidates for demonstrating quantum advantage in the near term.
Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems
The Quantum Approximate Optimization Algorithm (QAOA) is arguably one of the leading quantum algorithms that can outperform classical state-of-the-art methods in the near term.
A Modular Deep Learning Pipeline for Galaxy-Scale Strong Gravitational Lens Detection and Modeling
Upcoming large astronomical surveys are expected to capture an unprecedented number of strong gravitational lensing systems.
Value-Added Chemical Discovery Using Reinforcement Learning
Computer-assisted synthesis planning aims to help chemists find better reaction pathways faster.
Graph-Partitioning-Based Diffusion Convolutional Recurrent Neural Network for Large-Scale Traffic Forecasting
We demonstrate the efficacy of the graph-partitioning-based DCRNN approach to model the traffic on a large California highway network with 11, 160 sensor locations.
MaLTESE: Large-Scale Simulation-Driven Machine Learning for Transient Driving Cycles
In order to reduce the number of dynamometer tests required for calibrating modern engines, a large-scale simulation-driven machine learning approach is presented in this work.
Using recurrent neural networks for nonlinear component computation in advection-dominated reduced-order models
Rapid simulations of advection-dominated problems are vital for multiple engineering and geophysical applications.
Site-specific graph neural network for predicting protonation energy of oxygenate molecules
These oxygenated molecules have adequate carbon, hydrogen, and oxygen atoms that can be used for developing new value-added molecules (chemicals or transportation fuels).
Scalable Reinforcement-Learning-Based Neural Architecture Search for Cancer Deep Learning Research
Cancer is a complex disease, the understanding and treatment of which are being aided through increases in the volume of collected data and in the scale of deployed computing power.
Neuromorphic Architecture Optimization for Task-Specific Dynamic Learning
Our results show that optimal learning rules can be dataset-dependent even within similar tasks.
Neuromorphic Acceleration for Approximate Bayesian Inference on Neural Networks via Permanent Dropout
We use these results to demonstrate the feasibility of conducting the inference phase with permanent dropout on spiking neural networks, mitigating the technique's computational and energy burden, which is essential for its use at scale or on edge platforms.
