Search Results for author:
Found 2 papers, 2 papers with code
TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer
TransNormerLLM evolves from the previous linear attention architecture TransNormer by making advanced modifications that include positional embedding, linear attention acceleration, gating mechanisms, tensor normalization, and inference acceleration and stabilization.
cosFormer: Rethinking Softmax in Attention
As one of its core components, the softmax attention helps to capture long-range dependencies yet prohibits its scale-up due to the quadratic space and time complexity to the sequence length.
Ranked #4 on
Offline RL
on D4RL
Cannot find the paper you are looking for? You can
Submit a new open access paper.
