Search Results for author:
Found 6 papers, 5 papers with code
A critical look at the evaluation of GNNs under heterophily: Are we really making progress?
Graphs without this property are called heterophilous, and it is typically assumed that specialized methods are required to achieve strong performance on such graphs.
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
Many deep learning applications benefit from using large models with billions of parameters.
Training Transformers Together
The infrastructure necessary for training state-of-the-art models is becoming overly expensive, which makes training such models affordable only to large corporations and institutions.
Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees
Due to these considerations, it is important to equip existing methods with strategies that would allow to reduce the volume of transmitted information during training while obtaining a model of comparable quality.
Secure Distributed Training at Scale
Training such models requires a lot of computational resources (e. g., HPC clusters) that are not available to small research groups and independent researchers.
Distributed Deep Learning in Open Collaborations
Modern deep learning applications require increasingly more compute to train state-of-the-art models.
