Search Results for author:
Found 14 papers, 3 papers with code
Scalable Multi-Domain Adaptation of Language Models using Modular Experts
MoDE is a mixture-of-experts architecture that augments a general PLMs with modular, domain-specialized experts.
Brainformers: Trading Simplicity for Efficiency
Using this insight, we develop a complex block, named Brainformer, that consists of a diverse sets of layers such as sparsely gated feed-forward layers, dense feed-forward layers, attention layers, and various forms of layer normalization and activation functions.
GiPH: Generalizable Placement Learning for Adaptive Heterogeneous Computing
Careful placement of a computational application within a target device cluster is critical for achieving low application completion time.
Lifelong Language Pretraining with Distribution-Specialized Experts
Compared to existing lifelong learning approaches, Lifelong-MoE achieves better few-shot performance on 19 downstream NLP tasks.
Mixture-of-Experts with Expert Choice Routing
Prior work allocates a fixed number of experts to each token using a top-k function regardless of the relative importance of different tokens.
A Transferable Approach for Partitioning Machine Learning Models on Multi-Chip-Modules
Partitioning ML graphs for MCMs is particularly hard as the search space grows exponentially with the number of chiplets available and the number of nodes in the neural network.
An Evaluation of Edge TPU Accelerators for Convolutional Neural Networks
Then, we extensively evaluate three classes of Edge TPUs, covering different computing ecosystems, that are either currently deployed in Google products or are the product pipeline, across 423K unique convolutional neural networks.
Rethinking Co-design of Neural Architectures and Hardware Accelerators
In our work, we target the optimization of hardware and software configurations on an industry-standard edge accelerator.
Apollo: Transferable Architecture Exploration
We further show that by transferring knowledge between target architectures with different design constraints, Apollo is able to find optimal configurations faster and often with better objective value (up to 25% improvements).
NAHAS: Neural Architecture and Hardware Accelerator Search
In this paper, we study the importance of co-designing neural architectures and hardware accelerators.
Transferable Graph Optimizers for ML Compilers
Most compilers for machine learning (ML) frameworks need to solve many correlated optimization problems to generate efficient machine code.
Chip Placement with Deep Reinforcement Learning
To achieve these results, we pose placement as a Reinforcement Learning (RL) problem and train an agent to place the nodes of a chip netlist onto a chip canvas.
GDP: Generalized Device Placement for Dataflow Graphs
Runtime and scalability of large neural networks can be significantly affected by the placement of operations in their dataflow graphs on suitable devices.
In-Datacenter Performance Analysis of a Tensor Processing Unit
Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters' NN inference demand.
