Search Results for author:
Found 4 papers, 2 papers with code
The Impact of Positional Encoding on Length Generalization in Transformers
In this paper, we conduct a systematic empirical study comparing the length generalization performance of decoder-only Transformers with five different position encoding approaches including Absolute Position Embedding (APE), T5's Relative PE, ALiBi, and Rotary, in addition to Transformers without positional encoding (NoPE).
Measuring the Knowledge Acquisition-Utilization Gap in Pretrained Language Models
We propose a systematic framework to measure parametric knowledge utilization in PLMs.
The Curious Case of Absolute Position Embeddings
Transformer language models encode the notion of word order using positional information.
Paraphrase Generation by Learning How to Edit from Samples
With its novel editor module, the model then paraphrases the input sequence by editing it using the extracted relations between the retrieved pair of sentences.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
