Search Results for author:
Found 51 papers, 37 papers with code
Adaptive Preconditioners Trigger Loss Spikes in Adam
Loss spikes emerge commonly during training across neural networks of varying architectures and scales when using the Adam optimizer.
Adversarial Preference Learning for Robust LLM Alignment
However, they remain vulnerable to adversarial attacks due to three key limitations: (1) the inefficiency and high cost of human annotation, (2) the vast diversity of potential adversarial attacks, and (3) the risk of feedback bias and reward hacking.
Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles
Although these observations are based on jigsaw puzzles and may vary across other visual tasks, this research contributes a valuable piece of jigsaw to the larger puzzle of collective understanding rule-based visual RL and its potential in multimodal learning.
Scalable Complexity Control Facilitates Reasoning Ability of LLMs
The reasoning ability of large language models (LLMs) has been rapidly advancing in recent years, attracting interest in more fundamental approaches that can reliably enhance their generalizability.
Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models
With the rapid development of Large Language Models (LLMs), aligning these models with human preferences and values is critical to ensuring ethical and safe applications.
CodeFlowBench: A Multi-turn, Iterative Benchmark for Complex Code Generation
Modern software development demands code that is maintainable, testable, and scalable by organizing the implementation into modular components with iterative reuse of existing codes.
xVerify: Efficient Answer Verifier for Reasoning Model Evaluations
To train and evaluate xVerify, we construct the VAR dataset by collecting question-answer pairs generated by multiple LLMs across various datasets, leveraging multiple reasoning models and challenging evaluation sets designed specifically for reasoning model assessment.
MaintainCoder: Maintainable Code Generation Under Dynamic Requirements
Modern code generation has made significant strides in functional correctness and execution efficiency.
RARE: Retrieval-Augmented Reasoning Modeling
Domain-specific intelligence demands specialized knowledge and sophisticated reasoning for problem-solving, posing significant challenges for large language models (LLMs) that struggle with knowledge hallucination and inadequate reasoning capabilities under constrained parameter budgets.
MoC: Mixtures of Text Chunking Learners for Retrieval-Augmented Generation System
Retrieval-Augmented Generation (RAG), while serving as a viable complement to large language models (LLMs), often overlooks the crucial aspect of text chunking within its pipeline.
SEAP: Training-free Sparse Expert Activation Pruning Unlock the Brainpower of Large Language Models
Notably, at 50% pruning, SEAP surpasses both WandA and FLAP by over 20%, and at 20% pruning, it incurs only a 2. 2% performance drop compared to the dense model.
SurveyX: Academic Survey Automation via Large Language Models
Large Language Models (LLMs) have demonstrated exceptional comprehension capabilities and a vast knowledge base, suggesting that LLMs can serve as efficient tools for automated survey generation.
HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generation
Retrieval-Augmented Generation (RAG) systems often struggle with imperfect retrieval, as traditional retrievers focus on lexical or semantic similarity rather than logical relevance.
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Model
However, the incorporation of external and unverified knowledge increases the vulnerability of LLMs because attackers can perform attack tasks by manipulating knowledge.
GRAPHMOE: Amplifying Cognitive Depth of Mixture-of-Experts Network via Introducing Self-Rethinking Mechanism
Additionally, this study explores a novel recurrent routing strategy that may inspire further advancements in enhancing the reasoning capabilities of language models.
Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perception
While Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm for boosting large language models (LLMs) in knowledge-intensive tasks, it often overlooks the crucial aspect of text chunking within its workflow.
FTII-Bench: A Comprehensive Multimodal Benchmark for Flow Text with Image Insertion
Based on this, we introduce the Flow Text with Image Insertion Benchmark (FTII-Bench), which includes 318 high-quality Chinese image-text news articles and 307 high-quality English image-text news articles, covering 10 different news domains.
QAEncoder: Towards Aligned Representation Learning in Question Answering System
Modern QA systems entail retrieval-augmented generation (RAG) for accurate and trustworthy responses.
MM-CamObj: A Comprehensive Multimodal Dataset for Camouflaged Object Scenarios
CamObj-Instruct is collected for fine-tuning the LVLMs with improved instruction-following capabilities, and it includes 11, 363 images and 68, 849 conversations with diverse instructions.
Attention Heads of Large Language Models: A Survey
Since the advent of ChatGPT, Large Language Models (LLMs) have excelled in various tasks but remain as black-box systems.
Controllable Text Generation for Large Language Models: A Survey
This paper systematically reviews the latest advancements in CTG for LLMs, offering a comprehensive definition of its core concepts and clarifying the requirements for control conditions and text quality.
Xinyu: An Efficient LLM-based System for Commentary Generation
To address the advanced requirements, we present an argument ranking model for arguments and establish a comprehensive evidence database that includes up-to-date events and classic books, thereby strengthening the substantiation of the evidence with retrieval augmented generation (RAG) technology.
Internal Consistency and Self-Feedback in Large Language Models: A Survey
In this paper, we use a unified perspective of internal consistency, offering explanations for reasoning deficiencies and hallucinations.
$\text{Memory}^3$: Language Modeling with Explicit Memory
The model is named $\text{Memory}^3$, since explicit memory is the third form of memory in LLMs after implicit memory (model parameters) and working memory (context key-values).
HRDE: Retrieval-Augmented Large Language Models for Chinese Health Rumor Detection and Explainability
HealthRCN is the largest known dataset of Chinese health information rumors to date.
FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models
Large language models (LLMs) excel in generating coherent text, but they often struggle with context awareness, leading to inaccuracies in tasks requiring faithful adherence to provided information.
Improving Generalization and Convergence by Enhancing Implicit Regularization
In this work, we propose an Implicit Regularization Enhancement (IRE) framework to accelerate the discovery of flat solutions in deep learning, thereby improving generalization and convergence.
Empowering Large Language Models to Set up a Knowledge Retrieval Indexer via Self-Learning
Retrieval-Augmented Generation (RAG) offers a cost-effective approach to injecting real-time knowledge into large language models (LLMs).
xFinder: Robust and Pinpoint Answer Extraction for Large Language Models
The continuous advancement of large language models (LLMs) has brought increasing attention to the critical issue of developing fair and reliable methods for evaluating their performance.
Proxy-RLHF: Decoupling Generation and Alignment in Large Language Model with Proxy
Reinforcement Learning from Human Feedback (RLHF) is the prevailing approach to ensure Large Language Models (LLMs) align with human values.
NewsBench: A Systematic Evaluation Framework for Assessing Editorial Capabilities of Large Language Models in Chinese Journalism
We present NewsBench, a novel evaluation framework to systematically assess the capabilities of Large Language Models (LLMs) for editorial capabilities in Chinese journalism.
Controlled Text Generation for Large Language Model with Dynamic Attribute Graphs
In this study, we introduce a pluggable CTG framework for Large Language Models (LLMs) named Dynamic Attribute Graphs-based controlled text generation (DATG).
CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models
For each of these CRUD categories, we have developed comprehensive datasets to evaluate the performance of RAG systems.
Grimoire is All You Need for Enhancing Large Language Models
In this paper, we propose a method SLEICL that involves learning from examples using strong language models and then summarizing and transferring these learned skills to weak language models for inference and application.
UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation
These techniques encompass the use of directed hallucination induction and strategies that deliberately alter authentic text to produce hallucinations.
CodeKGC: Code Language Model for Generative Knowledge Graph Construction
However, large generative language model trained on structured data such as code has demonstrated impressive capability in understanding natural language for structural prediction and reasoning tasks.
On Analyzing the Role of Image for Visual-enhanced Relation Extraction
Multimodal relation extraction is an essential task for knowledge graph construction.
Construction and Applications of Billion-Scale Pre-Trained Multimodal Business Knowledge Graph
We release all the open resources (OpenBG benchmarks) derived from it for the community and report experimental results of KG-centric tasks.
BCRLSP: An Offline Reinforcement Learning Framework for Sequential Targeted Promotion
During deployment, we combine the offline RL model with the LP model to generate a robust policy under the budget constraints.
Disentangled Ontology Embedding for Zero-shot Learning
In this paper, we focus on ontologies for augmenting ZSL, and propose to learn disentangled ontology embeddings guided by ontology properties to capture and utilize more fine-grained class relationships in different aspects.
Relphormer: Relational Graph Transformer for Knowledge Graph Representations
To this end, we propose a new variant of Transformer for knowledge graph representations dubbed Relphormer.
Ranked #3 on
Link Prediction
on FB15k-237
Bridging the Gap between Reality and Ideality of Entity Matching: A Revisiting and Benchmark Re-Construction
Experimental results demonstrate that the assumptions made in the previous benchmark construction process are not coincidental with the open environment, which conceal the main challenges of the task and therefore significantly overestimate the current progress of entity matching.
NeuralKG: An Open Source Library for Diverse Representation Learning of Knowledge Graphs
NeuralKG is an open-source Python-based library for diverse representation learning of knowledge graphs.
From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer
Knowledge graph completion aims to address the problem of extending a KG with missing triples.
Ranked #52 on
Link Prediction
on FB15k-237
Ontology-enhanced Prompt-tuning for Few-shot Learning
Specifically, we develop the ontology transformation based on the external knowledge graph to address the knowledge missing issue, which fulfills and converts structure knowledge to text.
DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population
We present an open-source and extensible knowledge extraction toolkit DeepKE, supporting complicated low-resource, document-level and multimodal scenarios in the knowledge base population.
Knowledge Graph Embedding in E-commerce Applications: Attentive Reasoning, Explanations, and Transferable Rules
We first identity three important desiderata for e-commerce KG systems: 1) attentive reasoning, reasoning over a few target relations of more concerns instead of all; 2) explanation, providing explanations for a prediction to help both users and business operators understand why the prediction is made; 3) transferable rules, generating reusable rules to accelerate the deployment of a KG to new systems.
BCORLE($\lambda$): An Offline Reinforcement Learning and Evaluation Framework for Coupons Allocation in E-commerce Market
Further, a novel offline reinforcement learning method and an off-policy evaluation algorithm are proposed for policy learning and policy evaluation, respectively.
Interpretable performance analysis towards offline reinforcement learning: A dataset perspective
We make it open-source for fair and comprehensive competitions between offline RL algorithms with complete datasets and checkpoints being provided.
Studying Product Competition Using Representation Learning
We show that, compared with state-of-the-art models, our approach is faster, and can produce more accurate demand forecasts and price elasticities.
