Search Results for author:
Found 16 papers, 13 papers with code
FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion
Heterogeneous model fusion enhances the performance of LLMs by integrating the knowledge and capabilities of multiple structurally diverse models.
FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion
The FuseChat-3. 0 training pipeline consists of two key stages: (1) supervised fine-tuning (SFT) to align the target and source model distributions, and (2) Direct Preference Optimization (DPO) to apply preferences from multiple source LLMs to fine-tune the target model.
Advantage-Guided Distillation for Preference Alignment in Small Language Models
Instead of directly applying existing alignment techniques to SLMs, we propose to utilize a well-aligned teacher LLM to guide the alignment process for these models, thereby facilitating the transfer of the teacher's knowledge of human preferences to the student model.
Weighted-Reward Preference Optimization for Implicit Model Fusion
To address distributional deviations between the source and target LLMs, WRPO introduces a progressive adaptation strategy that gradually shifts reliance on preferred examples from the target LLM to the source LLMs.
FuseChat: Knowledge Fusion of Chat Models
In this work, we propose a new framework for the knowledge fusion of chat LLMs through two main stages, resulting in FuseChat.
ProFuser: Progressive Fusion of Large Language Models
While fusing the capacities and advantages of various large language models (LLMs) offers a pathway to construct more powerful and versatile models, a fundamental challenge is to properly select advantageous model during the training.
Self-Evolution Fine-Tuning for Policy Optimization
In this paper, we introduce self-evolution fine-tuning (SEFT) for policy optimization, with the aim of eliminating the need for annotated samples while retaining the stability and efficiency of SFT.
BlockPruner: Fine-grained Pruning for Large Language Models
While various layer pruning methods have been developed based on this insight, they generally overlook the finer-grained redundancies within the layers themselves.
Knowledge Fusion of Chat LLMs: A Preliminary Technical Report
Recently, FuseLLM introduced the concept of knowledge fusion to transfer the collective knowledge of multiple structurally varied LLMs into a target LLM through lightweight continual training.
Knowledge Verification to Nip Hallucination in the Bud
While large language models (LLMs) have demonstrated exceptional performance across various tasks following human alignment, they may still generate responses that sound plausible but contradict factual knowledge, a phenomenon known as hallucination.
Knowledge Fusion of Large Language Models
In this paper, we introduce the notion of knowledge fusion for LLMs, aimed at combining the capabilities of existing LLMs and transferring them into a single LLM.
PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection
Drawing inspiration from Psychological Questionnaires, which are carefully designed by psychologists to evaluate individual personality traits through a series of targeted items, we argue that these items can be regarded as a collection of well-structured chain-of-thought (CoT) processes.
Explore-Instruct: Enhancing Domain-Specific Instruction Coverage through Active Exploration
Instruction-tuning can be substantially optimized through enhanced diversity, resulting in models capable of handling a broader spectrum of tasks.
Retrieval-Generation Alignment for End-to-End Task-Oriented Dialogue System
The results demonstrate that when combined with meta knowledge, the response generator can effectively leverage high-quality knowledge records from the retriever and enhance the quality of generated responses.
Clustering-Aware Negative Sampling for Unsupervised Sentence Representation
Contrastive learning has been widely studied in sentence representation learning.
Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog
Retrieving proper domain knowledge from an external database lies at the heart of end-to-end task-oriented dialog systems to generate informative responses.
