Search Results for author:
Found 21 papers, 15 papers with code
Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Model
In this study, we introduce CT-LLM, a 2B large language model (LLM) that illustrates a pivotal shift towards prioritizing the Chinese language in developing LLMs.
The Fine Line: Navigating Large Language Model Pretraining with Down-streaming Capability Analysis
To address the aforementioned limitations, this paper undertakes a comprehensive comparison of model capabilities at various pretraining intermediate checkpoints.
EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models
This paper introduces EasyJailbreak, a unified framework simplifying the construction and evaluation of jailbreak attacks against LLMs.
ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages
Tool learning is widely acknowledged as a foundational approach or deploying large language models (LLMs) in real-world scenarios.
Navigating the OverKill in Large Language Models
Large language models are meticulously aligned to be both helpful and harmless.
Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback
This reliance limits the applicability of RLHF and hinders the development of professional assistants tailored to diverse human preferences.
RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning
To bridge this gap, we introduce RoTBench, a multi-level benchmark for evaluating the robustness of LLMs in tool learning.
Secrets of RLHF in Large Language Models Part II: Reward Modeling
We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset and fully leverage high-quality preference data.
ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios
Furthermore, a sole emphasis on outcomes disregards the intricate capabilities essential for LLMs to effectively utilize tools.
LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks.
RealBehavior: A Framework for Faithfully Characterizing Foundation Models' Human-like Behavior Mechanisms
Reports of human-like behaviors in foundation models are growing, with psychological theories providing enduring tools to investigate these behaviors.
TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
In this paper, we introduce TRACE, a novel benchmark designed to evaluate continual learning in LLMs.
Echotune: A Modular Extractor Leveraging the Variable-Length Nature of Speech in ASR Tasks
The Transformer architecture has proven to be highly effective for Automatic Speech Recognition (ASR) tasks, becoming a foundational component for a plethora of research in the domain.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Secrets of RLHF in Large Language Models Part I: PPO
Therefore, we explore the PPO-max, an advanced version of PPO algorithm, to efficiently improve the training stability of the policy model.
DSRM: Boost Textual Adversarial Training with Distribution Shift Risk Minimization
Adversarial training is one of the best-performing methods in improving the robustness of deep language models.
On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection
Detecting adversarial samples that are carefully crafted to fool the model is a critical step to socially-secure applications.
Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement
To enhance the multi-step reasoning capabilities of large language models, researchers have extensively explored prompting methods, notably the Chain-of-Thought (CoT) method which explicitly elicits human-like rationales.
Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model
Pretrained language models have achieved remarkable success in various natural language processing tasks.
CausalAPM: Generalizable Literal Disentanglement for NLU Debiasing
Dataset bias, i. e., the over-reliance on dataset-specific literal heuristics, is getting increasing attention for its detrimental effect on the generalization ability of NLU models.
Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding
Dataset bias has attracted increasing attention recently for its detrimental effect on the generalization ability of fine-tuned models.
Decorrelate Irrelevant, Purify Relevant: Overcome Textual Spurious Correlations from a Feature Perspective
Most of the existing debiasing methods often identify and weaken these samples with biased features (i. e., superficial surface features that cause such spurious correlations).
