Search Results for author:
Found 11 papers, 6 papers with code
Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loop
A key structural element in both of these methods is the inclusion of an outer loop at the beginning of which a full pass over the training data is made in order to compute the exact gradient, which is then used to construct a variance-reduced estimator of the gradient.
Natural Compression for Distributed Deep Learning
Our technique is applied individually to all entries of the to-be-compressed update vector and works by randomized rounding to the nearest (negative or positive) power of two, which can be computed in a "natural" way by ignoring the mantissa.
Optimal Client Sampling for Federated Learning
We show that importance can be measured using only the norm of the update and give a formula for optimal client participation.
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
FjORD alleviates the problem of client system heterogeneity by tailoring the model width to the client's capabilities.
A Field Guide to Federated Optimization
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection.
Granger Causality using Neural Networks
It helps in answering the question whether one time series is helpful in forecasting.
Balancing Privacy and Performance for Private Federated Learning Algorithms
Our findings reveal a direct correlation between the optimal number of local steps, communication rounds, and a set of variables, e. g the DP privacy budget and other problem parameters, specifically in the context of strongly convex optimization.
Maestro: Uncovering Low-Rank Structures via Trainable Decomposition
We further apply our technique on DNNs and empirically illustrate that Maestro enables the extraction of lower footprint models that preserve model performance while allowing for graceful accuracy-latency tradeoff for the deployment to devices of different capabilities.
Efficient Conformal Prediction under Data Heterogeneity
Conformal Prediction (CP) stands out as a robust framework for uncertainty quantification, which is crucial for ensuring the reliability of predictions.
Rethink Model Re-Basin and the Linear Mode Connectivity
Recent studies suggest that with sufficiently wide models, most SGD solutions can, up to permutation, converge into the same basin.
Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad
Adaptive methods are extremely popular in machine learning as they make learning rate tuning less expensive.
