Search Results for author: Yiqiao Wang

Found 1 papers, 1 papers with code

Scalable MatMul-free Language Modeling

1 code implementation4 Jun 2024 Rui-Jie Zhu, Yu Zhang, Ethan Sifferman, Tyler Sheaves, Yiqiao Wang, Dustin Richmond, Peng Zhou, Jason K. Eshraghian

Our experiments show that our proposed MatMul-free models achieve performance on-par with state-of-the-art Transformers that require far more memory during inference at a scale up to at least 2. 7B parameters.

Language Modelling

Cannot find the paper you are looking for? You can Submit a new open access paper.