Search Results for author:
Found 10 papers, 4 papers with code
LUT Tensor Core: Lookup Table Enables Efficient Low-Bit LLM Inference Acceleration
Then, LUT Tensor Core proposes the hardware design featuring an elongated tiling shape design to enhance table reuse and a bit-serial design to support various precision combinations in mpGEMM.
Scaling Deep Learning Computation over the Inter-Core Connected Intelligence Processor with T10
As AI chips incorporate numerous parallelized cores to scale deep learning (DL) computing, inter-core communication is enabled recently by employing high-bandwidth and low-latency interconnect links on the chip (e. g., Graphcore IPU).
T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edge
Weight quantization is crucial for reducing the memory footprint of LLMs on devices.
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Recent research, such as BitNet, is paving the way for a new era of 1-bit Large Language Models (LLMs).
BitNet: Scaling 1-bit Transformers for Large Language Models
The increasing size of large language models has posed challenges for deployment and raised concerns about environmental impact due to high energy consumption.
FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement
We first present an empirical analysis on the problems and opportunities of training MoE models, which motivates us to overcome the routing imbalance and fluctuation problems by a dynamic expert management and device placement mechanism.
PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation
Dynamic sparsity, where the sparsity patterns are unknown until runtime, poses a significant challenge to deep learning.
EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate
Then it diversifies the experts and continues to train the MoE with a novel Dense-to-Sparse gate (DTS-Gate).
Architectural Implications of Graph Neural Networks
Graph neural networks (GNN) represent an emerging line of deep learning models that operate on graph structures.
Towards Efficient Large-Scale Graph Neural Network Computing
This evolution has led to large graph-based irregular and sparse models that go beyond what existing deep learning frameworks are designed for.
