Search Results for author:
Found 6 papers, 0 papers with code
Evolving Machine Learning Algorithms From Scratch
However, this progress has largely focused on the architecture of neural networks, where it has relied on sophisticated expert-designed layers as building blocks---or similarly restrictive search spaces.
Brainformers: Trading Simplicity for Efficiency
Using this insight, we develop a complex block, named Brainformer, that consists of a diverse sets of layers such as sparsely gated feed-forward layers, dense feed-forward layers, attention layers, and various forms of layer normalization and activation functions.
The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink
Four best practices can reduce ML training energy by up to 100x and CO2 emissions up to 1000x.
Searching for Efficient Transformers for Language Modeling
For example, at a 500M parameter size, Primer improves the original T5 architecture on C4 auto-regressive language modeling, reducing the training cost by 4X.
Carbon Emissions and Large Neural Network Training
To help reduce the carbon footprint of ML, we believe energy usage and CO2e should be a key metric in evaluating models, and we are collaborating with MLPerf developers to include energy usage during training and inference in this industry standard benchmark.
Improving image generative models with human interactions
GANs provide a framework for training generative models which mimic a data distribution.
