$\text{Memory}^3$: Language Modeling with Explicit Memory

no code implementations • 1 Jul 2024 • Hongkang Yang, Zehao Lin, Wenjin Wang, Hao Wu, Zhiyu Li, Bo Tang, Wenqiang Wei, Jinbo Wang, Zeyun Tang, Shichao Song, Chenyang Xi, Yu Yu, Kai Chen, Feiyu Xiong, Linpeng Tang, Weinan E

The model is named $\text{Memory}^3$, since explicit memory is the third form of memory in LLMs after implicit memory (model parameters) and working memory (context key-values).

Language Modelling RAG +1

Deep Exponential Families

no code implementations • 10 Nov 2014 • Rajesh Ranganath, Linpeng Tang, Laurent Charlin, David M. Blei

We describe \textit{deep exponential families} (DEFs), a class of latent variable models that are inspired by the hidden structures used in deep neural networks.

Variational Inference