Search Results for author:
Found 11 papers, 6 papers with code
Training-Free Long-Context Scaling of Large Language Models
The ability of Large Language Models (LLMs) to process and generate coherent text is markedly weakened when the number of input tokens exceeds their pretraining length.
Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
In this paper, we seek to empirically investigate knowledge transfer from larger to smaller models through a parametric perspective.
Scaling Laws of RoPE-based Extrapolation
The extrapolation capability of Large Language Models (LLMs) based on Rotary Position Embedding is currently a topic of considerable interest.
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
Recently, there has been growing interest in extending the context length of large language models (LLMs), aiming to effectively process long inputs of one turn or conversations with more extensive histories.
Optimizing Non-Autoregressive Transformers with Contrastive Learning
In this paper, we propose to ease the difficulty of modality learning via sampling from the model distribution instead of the data distribution.
COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization
Traditional training paradigms for extractive and abstractive summarization systems always only use token-level or sentence-level training objectives.
CoNT: Contrastive Neural Text Generation
We validate CoNT on five generation tasks with ten benchmarks, including machine translation, summarization, code comment generation, data-to-text generation and commonsense generation.
$\mathcal{Y}$-Tuning: An Efficient Tuning Paradigm for Large-Scale Pre-Trained Models via Label Representation Learning
With the success of large-scale pre-trained models (PTMs), how efficiently adapting PTMs to downstream tasks has attracted tremendous attention, especially for PTMs with billions of parameters.
TURNER: The Uncertainty-based Retrieval Framework for Chinese NER
Chinese NER is a difficult undertaking due to the ambiguity of Chinese characters and the absence of word boundaries.
RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarization
Existing summarization systems mostly generate summaries purely relying on the content of the source document.
Enhancing Scientific Papers Summarization with Citation Graph
Previous work for text summarization in scientific domain mainly focused on the content of the input document, but seldom considering its citation network.
