Search Results for author:
Found 7 papers, 2 papers with code
A Silver Bullet or a Compromise for Full Attention? A Comprehensive Study of Gist Token-based Context Compression
In this work, we provide a thorough investigation of gist-based context compression methods to improve long-context processing in large language models.
Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models
In this work, we provide a detailed analysis of this issue and identify that unusually high attention entropy can be a key factor.
Knowledge Boundary of Large Language Models: A Survey
Although large language models (LLMs) store vast amount of knowledge in their parameters, they still have limitations in the memorization and utilization of certain knowledge, leading to undesired behaviors such as generating untruthful and inaccurate responses.
On the Transformations across Reward Model, Parameter Update, and In-Context Prompt
Despite the general capabilities of pre-trained large language models (LLMs), they still need further adaptation to better serve practical applications.
Consecutive Batch Model Editing with HooK Layers
As the typical retraining paradigm is unacceptably time- and resource-consuming, researchers are turning to model editing to find an effective way that supports both consecutive and batch scenarios to edit the model behavior directly.
WatME: Towards Lossless Watermarking Through Lexical Redundancy
Text watermarking has emerged as a pivotal technique for identifying machine-generated text.
DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Text
Specifically, we design a novel node memory scheme and aggregate the information over the depth dimension instead of the breadth dimension of the graph, which empowers the ability to collect long dependencies without stacking multiple layers.
