Search Results for author:
Found 19 papers, 8 papers with code
Learning to Compose SuperWeights for Neural Parameter Allocation Search
To address this, we generate layer weights by learning to compose sets of SuperWeights, which represent a group of trainable parameters.
Cached Operator Reordering: A Unified View for Fast GNN Training
Graph Neural Networks (GNNs) are a powerful tool for handling structured graph data and addressing tasks such as node classification, graph classification, and clustering.
STen: Productive and Efficient Sparsity in PyTorch
As deep learning models grow, sparsity is becoming an increasingly critical component of deep neural networks, enabling improved performance and reduced storage.
Spatial Mixture-of-Experts
Many data have an underlying dependence on spatial location; it may be weather on the Earth, a simulation on a mesh, or a registered image.
Neural Graph Databases
In general, LPG2vec enables combining predictive power of the most powerful GNNs with the full scope of information encoded in the LPG model, paving the way for neural graph databases, a class of systems where the vast complexity of maintained data will benefit from modern and future graph machine learning methods.
ENS-10: A Dataset For Post-Processing Ensemble Weather Forecasts
We propose the ENS-10 prediction correction task for improving the forecast quality at a 48-hour lead time through ensemble post-processing.
A Data-Centric Optimization Framework for Machine Learning
Rapid progress in deep learning is leading to a diverse set of quickly changing models, with a dramatically growing demand for compute.
MixtureEnsembles: Leveraging Parameter Sharing for Efficient Ensembles
We improve on these methods with MixtureEnsembles, which learns to factorize ensemble members with shared parameters by constructing each layer with a linear combination of templates.
Learning Combinatorial Node Labeling Algorithms
We present a novel neural architecture to solve graph optimization problems where the solution consists of arbitrary node labels, allowing us to solve hard problems like graph coloring.
Motif Prediction with Graph Neural Networks
We also successfully apply our architecture for predicting more arbitrary clusters and communities, illustrating its potential for graph mining beyond motif analysis.
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
The growing energy and performance costs of deep learning have driven the community to reduce the size of neural networks by selectively pruning components.
Clairvoyant Prefetching for Distributed Machine Learning I/O
I/O is emerging as a major bottleneck for machine learning training, especially in distributed environments.
The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism
We present scalable hybrid-parallel algorithms for training large-scale 3D convolutional neural networks.
Data Movement Is All You Need: A Case Study on Optimizing Transformers
Transformers are one of the most important machine learning workloads today.
Neural Parameter Allocation Search
We introduce Neural Parameter Allocation Search (NPAS), a novel task where the goal is to train a neural network given an arbitrary, fixed parameter budget.
Deep Learning for Post-Processing Ensemble Weather Forecasts
Applied to global data, our mixed models achieve a relative improvement in ensemble forecast skill (CRPS) of over 14%.
Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging
For evaluation, we train ResNet-50 on ImageNet; Transformer for machine translation; and deep reinforcement learning for navigation at scale.
Predicting Weather Uncertainty with Deep Convnets
Modern weather forecast models perform uncertainty quantification using ensemble prediction systems, which collect nonparametric statistics based on multiple perturbed simulations.
Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelism
We also see an emerging need to handle datasets with very large samples, where memory requirements for training are large.
