Search Results for author:
Found 5 papers, 1 papers with code
DiPaCo: Distributed Path Composition
Progress in machine learning (ML) has been fueled by scaling neural network models.
Asynchronous Local-SGD Training for Language Modeling
Local stochastic gradient descent (Local-SGD), also referred to as federated averaging, is an approach to distributed optimization where each device performs more than one SGD update per communication.
DiLoCo: Distributed Low-Communication Training of Language Models
In this work, we propose a distributed optimization algorithm, Distributed Low-Communication (DiLoCo), that enables training of language models on islands of devices that are poorly connected.
A data-driven approach for learning to control computers
It would be useful for machines to use computers as humans do so that they can aid us in everyday tasks.
Imitating Interactive Intelligence
These evaluations convincingly demonstrate that interactive training and auxiliary losses improve agent behaviour beyond what is achieved by supervised learning of actions alone.
