1 code implementation • 6 Jun 2024 • Lars Hillebrand, Prabhupad Pradhan, Christian Bauckhage, Rafet Sifa
We introduce "pointer-guided segment ordering" (SO), a novel pre-training technique aimed at enhancing the contextual understanding of paragraph-level text representations in large language models.