no code implementations • 18 Feb 2024 • Ninglu Shao, Shitao Xiao, Zheng Liu, Peitian Zhang
2) Strong sample efficiency of training, which enables the embedding model to be learned in a cost-effective way.
1 code implementation • 15 Jan 2024 • Ninglu Shao, Shitao Xiao, Zheng Liu, Peitian Zhang
Extensible Tokenization stands as a midware in between of the tokenized context and the LLM, which transforms the raw token embeddings into the extensible embeddings.
1 code implementation • 7 Jan 2024 • Peitian Zhang, Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye, Zhicheng Dou
Although the context window can be extended through fine-tuning, it will result in a considerable cost at both training and inference time, and exert an unfavorable impact to the LLM's original capabilities.
1 code implementation • 3 May 2023 • Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongxiang Sun, Xiao Zhang, Jun Xu
The debut of ChatGPT has recently attracted the attention of the natural language processing (NLP) community and beyond.