Search Results for author:
Found 12 papers, 8 papers with code
LASER: Linear Compression in Wireless Distributed Optimization
On the latter, we obtain $50$-$64 \%$ improvement in perplexity over our baselines for noisy channels.
MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks
Predicting multiple real-world tasks in a single model often requires a particularly diverse feature space.
Beyond spectral gap (extended): The role of the topology in decentralized learning
This paper aims to paint an accurate picture of sparsely-connected distributed optimization.
Modular Clinical Decision Support Networks (MoDN) -- Updatable, Interpretable, and Portable Predictions for Evolving Clinical Environments
Data-driven Clinical Decision Support Systems (CDSS) have the potential to improve and standardise care with personalised probabilistic guidance.
Beyond spectral gap: The role of the topology in decentralized learning
In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster.
RelaySum for Decentralized Deep Learning on Heterogeneous Data
A key challenge, primarily in decentralized deep learning, remains the handling of differences between the workers' local data distributions.
Practical Low-Rank Communication Compression in Decentralized Deep Learning
Lossy gradient compression has become a practical tool to overcome the communication bottleneck in centrally coordinated distributed training of machine learning models.
PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning
Lossy gradient compression has become a practical tool to overcome the communication bottleneck in centrally coordinated distributed training of machine learning models.
Optimizer Benchmarking Needs to Account for Hyperparameter Tuning
The performance of optimizers, particularly in deep learning, depends considerably on their chosen hyperparameter configuration.
On the Tunability of Optimizers in Deep Learning
There is no consensus yet on the question whether adaptive gradient methods like Adam are easier to use than non-adaptive optimization methods like SGD.
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization
We study gradient compression methods to alleviate the communication bottleneck in data-parallel distributed optimization.
Kernel-predicting convolutional networks for denoising monte carlo renderings.
In a second approach, we introduce a novel, kernel-prediction network which uses the CNN to estimate the local weighting kernels used to compute each denoised pixel from its neighbors.
