INT-FP-QSim: Mixed Precision and Formats For Large Language Models and Vision Transformers

1 code implementation • 7 Jul 2023 • Lakshmi Nair, Mikhail Bernadskiy, Arulselvan Madhavan, Craig Chan, Ayon Basumallik, Darius Bunandar

To supplement this ongoing effort, we propose INT-FP-QSim: an open-source simulator that enables flexible evaluation of LLMs and vision transformers at various numerical precisions and formats.

Quantization