Search Results for author:
Found 1 papers, 1 papers with code
Learn Your Tokens: Word-Pooled Tokenization for Language Modeling
Language models typically tokenize text into subwords, using a deterministic, hand-engineered heuristic of combining characters into longer surface-level strings such as 'ing' or whole words.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
