Search Results for author:
Found 48 papers, 15 papers with code
Domain-Lifelong Learning for Dialogue State Tracking via Knowledge Preservation Networks
This paradigm is often impractical in real-world applications since online dialogue systems usually involve continually emerging new data and domains.
Prejudge-Before-Think: Enhancing Large Language Models at Test-Time by Process Prejudge Reasoning
In this paper, we introduce a new \emph{process prejudge} strategy in LLM reasoning to demonstrate that bootstrapping with process prejudge allows the LLM to adaptively anticipate the errors encountered when advancing the subsequent reasoning steps, similar to people sometimes pausing to think about what mistakes may occur and how to avoid them, rather than relying solely on trial and error.
NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables
Processing structured tabular data, particularly lengthy tables, constitutes a fundamental yet challenging task for large language models (LLMs).
SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversity
Existing pretraining data mixing methods for large language models (LLMs) typically follow a domain-wise methodology, a top-down process that first determines domain weights and then performs uniform data sampling across each domain.
Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective
Direct Preference Optimization (DPO) has gained attention as an efficient alternative to reinforcement learning from human feedback (RLHF) for aligning large language models (LLMs) with human preferences.
C2T: A Classifier-Based Tree Construction Method in Speculative Decoding
The growing scale of Large Language Models (LLMs) has exacerbated inference latency and computational costs.
MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures
The remarkable performance of large language models (LLMs) in various language tasks has attracted considerable attention.
Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration
We propose DPT-Agent, a novel language agent framework that integrates System 1 and System 2 for efficient real-time simultaneous human-AI collaboration.
FRAMES: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy
Large language models (LLMs) have significantly advanced human language understanding and generation, with pretraining data quality and organization being crucial to their performance.
FIRE: Flexible Integration of Data Quality Ratings for Effective Pre-Training
FIRE aligns multiple quality signals into a unified space, and integrates diverse data quality raters to provide a comprehensive quality signal for each data point.
Who's the MVP? A Game-Theoretic Evaluation Benchmark for Modular Attribution in LLM Agents
Large Language Model (LLM) agents frameworks often employ modular architectures, incorporating components such as planning, reasoning, action execution, and reflection to tackle complex tasks.
Revisit Self-Debugging with Self-Generated Tests for Code Generation
Large language models (LLMs) have shown significant advancements in code generation, but still face challenges on tasks beyond their basic capabilities.
Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
Current large language models (LLMs) generally utilize a consistent data distribution throughout the entire pretraining process.
AgentRefine: Enhancing Agent Generalization through Refinement Tuning
We analyze that the poor generalization ability comes from overfitting to several manual agent environments and a lack of adaptation to new situations.
Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
The quadratic computational complexity of the attention mechanism in current Large Language Models (LLMs) renders inference with long contexts prohibitively expensive.
PrefixKV: Adaptive Prefix KV Cache is What Vision Instruction-Following Models Need for Efficient Generation
With an adaptive layer-wise KV retention recipe based on binary search, the maximum contextual information can thus be preserved in each layer, facilitating the generation.
Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
Experiments show that the method improves the actor's exploration efficiency and solution diversity, especially on challenging queries, leading to a stronger reasoning model.
Predictor-Corrector Enhanced Transformers with Exponential Moving Average Coefficient Learning
First, we introduce a predictor-corrector learning framework to minimize truncation errors, which consists of a high-order predictor and a multistep corrector.
Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling
Self-improvement methods enable large language models (LLMs) to generate solutions themselves and iteratively train on filtered, high-quality rationales.
Multi-Programming Language Sandbox for LLMs
We introduce MPLSandbox, an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs).
FIRP: Faster LLM inference via future intermediate representation prediction
Recent advancements in Large Language Models (LLMs) have shown remarkable performance across a wide range of tasks.
EPS-MoE: Expert Pipeline Scheduler for Cost-Efficient MoE Inference
The Mixture-of-Experts (MoE) model has emerged as a prominent architecture in the field of Large Language Models (LLMs), providing a better balance between model performance and computational efficiency.
LogicPro: Improving Complex Logical Reasoning via Program-Guided Learning
In this paper, we present a novel approach, called LogicPro, to enhance Large Language Models (LLMs) complex Logical reasoning through Program Examples.
Length Desensitization in Direct Preference Optimization
Direct Preference Optimization (DPO) is widely utilized in the Reinforcement Learning from Human Feedback (RLHF) phase to align Large Language Models (LLMs) with human preferences, thereby enhancing both their harmlessness and efficacy.
How Do Your Code LLMs Perform? Empowering Code Instruction Tuning with High-Quality Data
Based on our selected data, we present XCoder, a family of models finetuned from LLaMA3.
ReMamba: Equip Mamba with Effective Long-Sequence Modeling
While the Mamba architecture demonstrates superior inference efficiency and competitive performance on short-context natural language processing (NLP) tasks, empirical evidence suggests its capacity to comprehend long contexts is limited compared to transformer-based models.
EAGLE: Elevating Geometric Reasoning through LLM-empowered Visual Instruction Tuning
To address these limitations, we propose EAGLE, a novel two-stage end-to-end visual enhancement MLLM framework designed to ElevAte Geometric reasoning through LLM-Empowered visual instruction tuning.
SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models
As large language models (LLMs) continue to advance in capability and influence, ensuring their security and preventing harmful outputs has become crucial.
Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs
Extensive experiments on 12 mainstream MLLMs, such as GPT-4V and Gemini-Pro Vision, demonstrate that these models exhibit significant hallucinations on Hallu-PI, which is not observed in unperturbed scenarios.
Graph-Structured Speculative Decoding
We apply GSD across a range of LLMs, including a 70-billion parameter LLaMA-2 model, and observe a remarkable speedup of 1. 73$\times$ to 1. 96$\times$, significantly surpassing standard speculative decoding.
What's Wrong with Your Code Generated by Large Language Models? An Extensive Study
The increasing development of large language models (LLMs) in code generation has drawn significant attention among researchers.
Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism
The recent advancements in large language models (LLMs) have been extraordinary, yet the escalating inference costs associated with them present challenges in real-world applications.
LLMs Know What They Need: Leveraging a Missing Information Guided Framework to Empower Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) demonstrates great value in alleviating outdated knowledge or hallucination by supplying LLMs with updated and relevant knowledge.
Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration
In this paper, we propose a novel parallel decoding approach, namely \textit{hidden transfer}, which decodes multiple successive tokens simultaneously in a single forward pass.
What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbation
Our findings reveal several connections between the properties of perturbations and LLM performance, providing insights into the failure cases of uniform quantization and suggesting potential solutions to improve the robustness of LLM quantization.
Unraveling the Mystery of Scaling Laws: Part I
However, the original scaling law paper by OpenAI did not disclose the complete details necessary to derive the precise scaling law formulas, and their conclusions are only based on models containing up to 1. 5 billion parameters.
Learning or Self-aligning? Rethinking Instruction Fine-tuning
Instruction Fine-tuning~(IFT) is a critical phase in building large language models~(LLMs).
Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection
Out-of-domain (OOD) intent detection aims to examine whether the user's query falls outside the predefined domain of the system, which is crucial for the proper functioning of task-oriented dialogue (TOD) systems.
DolphCoder: Echo-Locating Code Large Language Models with Diverse and Multi-Objective Instruction Tuning
It learns diverse instruction targets and combines a code evaluation objective to enhance its code generation ability.
Improving Input-label Mapping with Demonstration Replay for In-context Learning
The effectiveness of ICL can be attributed to the strong language modeling capabilities of large language models (LLMs), which enable them to learn the mapping between input and labels based on in-context demonstrations.
Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression
In particular, our approach extracts knowledge from LLMs to construct a knowledge store, from which the small-scale model can retrieve relevant information and leverage it for effective inference.
APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection
Detecting out-of-domain (OOD) intents from user queries is essential for a task-oriented dialogue system.
Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT
The tasks of out-of-domain (OOD) intent discovery and generalized intent discovery (GID) aim to extend a closed intent classifier to open-world intent sets, which is crucial to task-oriented dialogue (TOD) systems.
Confidence Calibration for Intent Detection via Hyperspherical Space and Rebalanced Accuracy-Uncertainty Loss
Experiments on the open datasets verify that our model outperforms the existing calibration methods and achieves a significant improvement on the calibration metric.
Density-Based Dynamic Curriculum Learning for Intent Detection
To handle this problem, we propose a density-based dynamic curriculum learning model.
From Paraphrasing to Semantic Parsing: Unsupervised Semantic Parsing via Synchronous Semantic Decoding
During synchronous decoding: the utterance paraphrasing is constrained by the structure of the logical form, therefore the canonical utterance can be paraphrased controlledly; the semantic decoding is guided by the semantics of the canonical utterance, therefore its logical form can be generated unsupervisedly.
