Search Results for author:
Found 7 papers, 5 papers with code
Parameter-Efficient Checkpoint Merging via Metrics-Weighted Averaging
Checkpoint merging is a technique for combining multiple model snapshots into a single superior model, potentially reducing training time for large language models.
Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers
Motivated by this approach, we propose Cross-Architecture Transfer Learning (XATL), in which the weights of the shared components between LCI and self-attention-based transformers, such as layernorms, MLPs, input/output embeddings, are directly transferred to the new architecture from already pre-trained model parameters.
AbsInstruct: Eliciting Abstraction Ability from LLMs through Explanation Tuning with Plausibility Estimation
Abstraction ability is crucial in human intelligence, which can also benefit various tasks in NLP study.
AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph
Cognitive research indicates that abstraction ability is essential in human intelligence, which remains under-explored in language models.
KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection
Large Language Models (LLMs) have demonstrated remarkable human-level natural language generation capabilities.
CKBP v2: Better Annotation and Reasoning for Commonsense Knowledge Base Population
We show that CKBP v2 serves as a challenging and representative evaluation dataset for the CSKB Population task, while its development set aids in selecting a population model that leads to improved knowledge acquisition for downstream commonsense reasoning.
Benchmarking Commonsense Knowledge Base Population with an Effective Evaluation Dataset
Experimental results show that generalizing commonsense reasoning on unseen assertions is inherently a hard task.
