Search Results for author: Lei Hou

Found 112 papers, 72 papers with code

How Can Cross-lingual Knowledge Contribute Better to Fine-Grained Entity Typing?

no code implementations Findings (ACL) 2022 Hailong Jin, Tiansi Dong, Lei Hou, Juanzi Li, Hui Chen, Zelin Dai, Qu Yincen

Cross-lingual Entity Typing (CLET) aims at improving the quality of entity type prediction by transferring semantic knowledge learned from rich-resourced languages to low-resourced languages.

Entity Typing Mixture-of-Experts +2

Do Pre-trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation and a Reasonable Approach

1 code implementation Findings (ACL) 2022 Xin Lv, Yankai Lin, Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie zhou

In recent years, pre-trained language models (PLMs) have been shown to capture factual knowledge from massive texts, which encourages the proposal of PLM-based knowledge graph completion (KGC) models.

Knowledge Graph Completion Link Prediction

Invertible Koopman neural operator for data-driven modeling of partial differential equations

no code implementations25 Mar 2025 Yuhong Jin, Andong Cong, Lei Hou, Qiang Gao, Xiangdong Ge, Chonglong Zhu, Yongzhi Feng, Jun Li

Koopman operator theory is a popular candidate for data-driven modeling because it provides a global linearization representation for nonlinear dynamical systems.

Operator learning

MRCEval: A Comprehensive, Challenging and Accessible Machine Reading Comprehension Benchmark

1 code implementation10 Mar 2025 Shengkun Ma, Hao Peng, Lei Hou, Juanzi Li

Existing MRC datasets primarily assess specific aspects of reading comprehension (RC), lacking a comprehensive MRC benchmark.

Machine Reading Comprehension Natural Language Understanding

Sparse Auto-Encoder Interprets Linguistic Features in Large Language Models

no code implementations27 Feb 2025 Yi Jing, Zijun Yao, Lingxu Ran, Hongzhu Guo, Xiaozhi Wang, Lei Hou, Juanzi Li

Large language models (LLMs) excel in tasks that require complex linguistic abilities, such as reference disambiguation and metaphor recognition/generation.

counterfactual Language Modeling +2

Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems

1 code implementation26 Feb 2025 Hao Peng, Yunjia Qi, Xiaozhi Wang, Zijun Yao, Bin Xu, Lei Hou, Juanzi Li

In this paper, we propose agentic reward modeling, a reward system that combines reward models with verifiable correctness signals from different aspects to provide reliable rewards.

Instruction Following

LongWriter-V: Enabling Ultra-Long and High-Fidelity Generation in Vision-Language Models

1 code implementation20 Feb 2025 Shangqing Tu, Yucheng Wang, Daniel Zhang-li, Yushi Bai, Jifan Yu, Yuhao Wu, Lei Hou, Huiqin Liu, Zhiyuan Liu, Bin Xu, Juanzi Li

Moreover, to achieve long outputs that maintain high-fidelity to the input images, we employ Direct Preference Optimization (DPO) to the SFT model.

Pairwise RM: Perform Best-of-N Sampling with Knockout Tournament

1 code implementation22 Jan 2025 Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, Juanzi Li

To address this, we propose a Pairwise Reward Model (Pairwise RM) combined with a knockout tournament for BoN sampling.

Math

LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks

1 code implementation19 Dec 2024 Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

This paper introduces LongBench v2, a benchmark designed to assess the ability of LLMs to handle long-context problems requiring deep understanding and reasoning across real-world multitasks.

8k In-Context Learning +1

EventSum: A Large-Scale Event-Centric Summarization Dataset for Chinese Multi-News Documents

no code implementations16 Dec 2024 Mengna Zhu, Kaisheng Zeng, Mao Wang, Kaiming Xiao, Lei Hou, Hongbin Huang, Juanzi Li

Therefore, we proposed the Event-Centric Multi-Document Summarization (ECS) task, which aims to generate concise and comprehensive summaries of a given event based on multiple related news documents.

Document Summarization Multi-Document Summarization

AtomR: Atomic Operator-Empowered Large Language Models for Heterogeneous Knowledge Reasoning

1 code implementation25 Nov 2024 Amy Xin, Jinxin Liu, Zijun Yao, Zhicheng Lee, Shulin Cao, Lei Hou, Juanzi Li

Drawing inspiration from the graph modeling of knowledge, AtomR leverages large language models (LLMs) to decompose complex questions into combinations of three atomic knowledge operators, significantly enhancing the reasoning process at both the planning and execution stages.

Hallucination Question Answering +2

Constraint Back-translation Improves Complex Instruction Following of Large Language Models

no code implementations31 Oct 2024 Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li

Following the conventional instruction-tuning practice, previous works conduct post-training on complex instruction-response pairs generated by feeding complex instructions to advanced LLMs.

Instruction Following Translation

LongReward: Improving Long-context Large Language Models with AI Feedback

1 code implementation28 Oct 2024 Jiajie Zhang, Zhongni Hou, Xin Lv, Shulin Cao, Zhenyu Hou, Yilin Niu, Lei Hou, Yuxiao Dong, Ling Feng, Juanzi Li

Though significant advancements have been achieved in developing long-context large language models (LLMs), the compromised quality of LLM-synthesized data for supervised fine-tuning (SFT) often affects the long-context performance of SFT models and leads to inherent limitations.

Offline RL Reinforcement Learning (RL)

RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style

1 code implementation21 Oct 2024 Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, Juanzi Li

However, this approach fails to assess reward models on subtle but critical content changes and variations in style, resulting in a low correlation with policy model performance.

Benchmarking Language Modeling +1

Pre-training Distillation for Large Language Models: A Design Space Exploration

no code implementations21 Oct 2024 Hao Peng, Xin Lv, Yushi Bai, Zijun Yao, Jiajie Zhang, Lei Hou, Juanzi Li

Previous work applying KD in the field of large language models (LLMs) typically focused on the post-training phase, where the student LLM learns directly from instructions and corresponding responses generated by the teacher model.

Knowledge Distillation

Awaking the Slides: A Tuning-free and Knowledge-regulated AI Tutoring System via Language Model Coordination

no code implementations11 Sep 2024 Daniel Zhang-li, Zheyuan Zhang, Jifan Yu, Joy Lim Jia Yin, Shangqing Tu, Linlu Gong, Haohua Wang, Zhiyuan Liu, Huiqin Liu, Lei Hou, Juanzi Li

We develop Slide2Lecture, a tuning-free and knowledge-regulated intelligent tutoring system that can (1) effectively convert an input lecture slide into a structured teaching agenda consisting of a set of heterogeneous teaching actions; (2) create and manage an interactive lecture that generates responsive interactions catering to student learning demands while regulating the interactions to follow teaching actions.

Language Modeling Language Modelling

From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents

no code implementations5 Sep 2024 Jifan Yu, Zheyuan Zhang, Daniel Zhang-li, Shangqing Tu, Zhanxin Hao, Rui Miao Li, Haoxuan Li, Yuanchun Wang, Hanming Li, Linlu Gong, Jie Cao, Jiayin Lin, Jinchang Zhou, Fei Qin, Haohua Wang, Jianxiao Jiang, Lijun Deng, Yisi Zhan, Chaojun Xiao, Xusheng Dai, Xuan Yan, Nianyi Lin, Nan Zhang, Ruixin Ni, Yang Dang, Lei Hou, Yu Zhang, Xu Han, Manli Li, Juanzi Li, Zhiyuan Liu, Huiqin Liu, Maosong Sun

Since the first instances of online education, where courses were uploaded to accessible and shared online platforms, this form of scaling the dissemination of human knowledge to reach a broader audience has sparked extensive discussion and widespread adoption.

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

1 code implementation4 Sep 2024 Jiajie Zhang, Yushi Bai, Xin Lv, Wanjun Gu, Danqing Liu, Minhao Zou, Shulin Cao, Lei Hou, Yuxiao Dong, Ling Feng, Juanzi Li

Though current long-context large language models (LLMs) have demonstrated impressive capacities in answering user questions based on extensive text, the lack of citations in their responses makes user verification difficult, leading to concerns about their trustworthiness due to their potential hallucinations.

Question Answering Sentence

OpenEP: Open-Ended Future Event Prediction

no code implementations13 Aug 2024 Yong Guan, Hao Peng, Xiaozhi Wang, Lei Hou, Juanzi Li

For question construction, we pose questions from seven perspectives, including location, time, event development, event outcome, event impact, event response, and other, to facilitate an in-depth analysis and understanding of the comprehensive evolution of events.

Prediction

LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

3 code implementations13 Aug 2024 Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10, 000 words while maintaining output quality.

MAVEN-Fact: A Large-scale Event Factuality Detection Dataset

2 code implementations22 Jul 2024 Chunyang Li, Hao Peng, Xiaozhi Wang, Yunjia Qi, Lei Hou, Bin Xu, Juanzi Li

Thanks to the comprehensive annotations of event arguments and relations in MAVEN, MAVEN-Fact also supports some further analyses and we find that adopting event arguments and relations helps in event factuality detection for fine-tuned models but does not benefit LLMs.

Hallucination

LLMAEL: Large Language Models are Good Context Augmenters for Entity Linking

1 code implementation4 Jul 2024 Amy Xin, Yunjia Qi, Zijun Yao, Fangwei Zhu, Kaisheng Zeng, Xu Bin, Lei Hou, Juanzi Li

Entity Linking (EL) models are well-trained at mapping mentions to their corresponding entities according to a given context.

Data Augmentation Entity Linking

SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation

1 code implementation27 Jun 2024 Zijun Yao, Weijian Qi, Liangming Pan, Shulin Cao, Linmei Hu, Weichuan Liu, Lei Hou, Juanzi Li

This paper introduces Self-aware Knowledge Retrieval (SeaKR), a novel adaptive RAG model that extracts self-aware uncertainty of LLMs from their internal states.

Question Answering RAG +1

Aligning Teacher with Student Preferences for Tailored Training Data Generation

no code implementations27 Jun 2024 Yantao Liu, Zhao Zhang, Zijun Yao, Shulin Cao, Lei Hou, Juanzi Li

Thus, we propose ARTE, dubbed Aligning TeacheR with StudenT PreferencEs, a framework that aligns the teacher model with student preferences to generate tailored training examples for Knowledge Distillation.

In-Context Learning Knowledge Distillation

Finding Safety Neurons in Large Language Models

no code implementations20 Jun 2024 Jianhui Chen, Xiaozhi Wang, Zijun Yao, Yushi Bai, Lei Hou, Juanzi Li

In this paper, we explore the inner mechanisms of safety alignment from the perspective of mechanistic interpretability, focusing on identifying and analyzing safety neurons within LLMs that are responsible for safety behaviors.

Misinformation Red Teaming +1

Knowledge-to-Jailbreak: One Knowledge Point Worth One Attack

1 code implementation17 Jun 2024 Shangqing Tu, Zhuoran Pan, Wenxuan Wang, Zhexin Zhang, Yuliang Sun, Jifan Yu, Hongning Wang, Lei Hou, Juanzi Li

To bridge this gap, we propose a new task, knowledge-to-jailbreak, which aims to generate jailbreaks from domain knowledge to evaluate the safety of LLMs when applied to those domains.

Language Modeling Language Modelling +1

R-Eval: A Unified Toolkit for Evaluating Domain Knowledge of Retrieval Augmented Large Language Models

1 code implementation17 Jun 2024 Shangqing Tu, Yuanchun Wang, Jifan Yu, Yuyang Xie, Yaran Shi, Xiaozhi Wang, Jing Zhang, Lei Hou, Juanzi Li

In this paper, we address the challenges of evaluating RALLMs by introducing the R-Eval toolkit, a Python toolkit designed to streamline the evaluation of different RAG workflows in conjunction with LLMs.

RAG Retrieval

DICE: Detecting In-distribution Contamination in LLM's Fine-tuning Phase for Math Reasoning

1 code implementation6 Jun 2024 Shangqing Tu, Kejian Zhu, Yushi Bai, Zijun Yao, Lei Hou, Juanzi Li

To effectively detect in-distribution contamination, we propose DICE, a novel method that leverages the internal states of LLMs to locate-then-detect the contamination.

Math

A Solution-based LLM API-using Methodology for Academic Information Seeking

1 code implementation24 May 2024 Yuanchun Wang, Jifan Yu, Zijun Yao, Jing Zhang, Yuyang Xie, Shangqing Tu, Yiyang Fu, Youhe Feng, Jinkai Zhang, Jingyao Zhang, Bowen Huang, Yuanyao Li, Huihui Yuan, Lei Hou, Juanzi Li, Jie Tang

Applying large language models (LLMs) for academic API usage shows promise in reducing researchers' academic information seeking efforts.

Event GDR: Event-Centric Generative Document Retrieval

no code implementations11 May 2024 Yong Guan, Dingxiao Liu, Jinchen Ma, Hao Peng, Xiaozhi Wang, Lei Hou, Ru Li

Inspired by this, we propose Event GDR, an event-centric generative document retrieval model, integrating event knowledge into this task.

Information Retrieval Retrieval

TacoERE: Cluster-aware Compression for Event Relation Extraction

no code implementations11 May 2024 Yong Guan, Xiaozhi Wang, Lei Hou, Juanzi Li, Jeff Pan, Jiaoyan Chen, Freddy Lecue

Existing work mainly focuses on directly modeling the entire document, which cannot effectively handle long-range dependencies and information redundancy.

Event Relation Extraction Relation +1

ADELIE: Aligning Large Language Models on Information Extraction

1 code implementation8 May 2024 Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li

Large language models (LLMs) usually fall short on information extraction (IE) tasks and struggle to follow the complex instructions of IE tasks.

MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classification

1 code implementation7 Apr 2024 Kai Sun, Yushi Bai, Ji Qi, Lei Hou, Juanzi Li

This highlights the challenging nature of our benchmark for existing models and the significant gap between the multimodal reasoning capabilities of current models and humans.

Image Comprehension Math +1

Evaluating Generative Language Models in Information Extraction as Subjective Question Correction

1 code implementation4 Apr 2024 Yuchen Fan, Yantao Liu, Zijun Yao, Jifan Yu, Lei Hou, Juanzi Li

(1) The imprecision of existing evaluation metrics that struggle to effectively gauge semantic consistency between model outputs and ground truth, and (2) The inherent incompleteness of evaluation benchmarks, primarily due to restrictive human annotation schemas, resulting in underestimated LLM performances.

Event Extraction Natural Language Inference +1

Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Models

1 code implementation4 Apr 2024 Yantao Liu, Zijun Yao, Xin Lv, Yuchen Fan, Shulin Cao, Jifan Yu, Lei Hou, Juanzi Li

However, knowledge in the document may conflict with the memory of LLMs due to outdated or incorrect knowledge in the LLMs' parameters.

Question Answering

A Cause-Effect Look at Alleviating Hallucination of Knowledge-grounded Dialogue Generation

no code implementations4 Apr 2024 Jifan Yu, Xiaohan Zhang, Yifan Xu, Xuanyu Lei, Zijun Yao, Jing Zhang, Lei Hou, Juanzi Li

Recently, knowledge-grounded dialogue generation models, that intentionally invoke external knowledge resources to more informative responses, are also proven to be effective in reducing hallucination.

counterfactual Counterfactual Reasoning +2

Reverse That Number! Decoding Order Matters in Arithmetic Learning

no code implementations9 Mar 2024 Daniel Zhang-li, Nianyi Lin, Jifan Yu, Zheyuan Zhang, Zijun Yao, Xiaokang Zhang, Lei Hou, Jing Zhang, Juanzi Li

Recent advancements in pretraining have demonstrated that modern Large Language Models (LLMs) possess the capability to effectively learn arithmetic operations.

Event-level Knowledge Editing

1 code implementation20 Feb 2024 Hao Peng, Xiaozhi Wang, Chunyang Li, Kaisheng Zeng, Jiangshan Duo, Yixin Cao, Lei Hou, Juanzi Li

However, natural knowledge updates in the real world come from the occurrences of new events rather than direct changes in factual triplets.

knowledge editing Triplet

CogCoM: Train Large Vision-Language Models Diving into Details through Chain of Manipulations

1 code implementation6 Feb 2024 Ji Qi, Ming Ding, Weihan Wang, Yushi Bai, Qingsong Lv, Wenyi Hong, Bin Xu, Lei Hou, Juanzi Li, Yuxiao Dong, Jie Tang

Drawing inspiration from human cognition in solving visual problems (e. g., marking, zoom in), this paper introduces Chain of Manipulations, a mechanism that enables VLMs to solve problems step-by-step with evidence.

Visual Reasoning

KB-Plugin: A Plug-and-play Framework for Large Language Models to Induce Programs over Low-resourced Knowledge Bases

1 code implementation2 Feb 2024 Jiajie Zhang, Shulin Cao, Linmei Hu, Ling Feng, Lei Hou, Juanzi Li

Secondly, KB-Plugin utilizes abundant annotated data from a rich-resourced KB to train another pluggable module, namely PI plugin, which can help the LLM extract question-relevant schema information from the schema plugin of any KB and utilize this information to induce programs over this KB.

Program induction Self-Supervised Learning

LongAlign: A Recipe for Long Context Alignment of Large Language Models

1 code implementation31 Jan 2024 Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, Juanzi Li

Extending large language models to effectively handle long contexts requires instruction fine-tuning on input sequences of similar length.

Diversity Instruction Following

Time Series Diffusion Method: A Denoising Diffusion Probabilistic Model for Vibration Signal Generation

no code implementations13 Dec 2023 Haiming Yi, Lei Hou, Yuhong Jin, Nasser A. Saeed, Ali Kandil, Hao Duan

However, in the field of vibration signal generation, the criteria for evaluating the quality of the generated signal are different from that of image generation and there is a fundamental difference between them.

Denoising Fault Diagnosis +3

Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions

1 code implementation23 Nov 2023 Shulin Cao, Jiajie Zhang, Jiaxin Shi, Xin Lv, Zijun Yao, Qi Tian, Juanzi Li, Lei Hou

During reasoning, for leaf nodes, LLMs choose a more confident answer from Closed-book QA that employs parametric knowledge and Open-book QA that employs retrieved external knowledge, thus eliminating the negative retrieval problem.

Retrieval

When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks

no code implementations15 Nov 2023 Hao Peng, Xiaozhi Wang, Jianhui Chen, Weikai Li, Yunjia Qi, Zimu Wang, Zhili Wu, Kaisheng Zeng, Bin Xu, Lei Hou, Juanzi Li

In this paper, we find that ICL falls short of handling specification-heavy tasks, which are tasks with complicated and extensive task specifications, requiring several hours for ordinary humans to master, such as traditional information extraction tasks.

In-Context Learning

MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation

1 code implementation15 Nov 2023 Xiaozhi Wang, Hao Peng, Yong Guan, Kaisheng Zeng, Jianhui Chen, Lei Hou, Xu Han, Yankai Lin, Zhiyuan Liu, Ruobing Xie, Jie zhou, Juanzi Li

Understanding events in texts is a core objective of natural language understanding, which requires detecting event occurrences, extracting event arguments, and analyzing inter-event relationships.

All Event Argument Extraction +4

WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models

2 code implementations13 Nov 2023 Shangqing Tu, Yuliang Sun, Yushi Bai, Jifan Yu, Lei Hou, Juanzi Li

To mitigate the potential misuse of large language models (LLMs), recent research has developed watermarking algorithms, which restrict the generation process to leave an invisible trace for watermark detection.

Benchmarking Instruction Following

VidCoM: Fast Video Comprehension through Large Language Models with Multimodal Tools

no code implementations16 Oct 2023 Ji Qi, Kaixuan Ji, Jifan Yu, Duokang Wang, Bin Xu, Lei Hou, Juanzi Li

Building models that comprehends videos and responds specific user instructions is a practical and challenging topic, as it requires mastery of both vision understanding and knowledge reasoning.

Caption Generation Descriptive +3

Mastering the Task of Open Information Extraction with Large Language Models and Consistent Reasoning Environment

no code implementations16 Oct 2023 Ji Qi, Kaixuan Ji, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Lei Hou, Juanzi Li, Bin Xu

Open Information Extraction (OIE) aims to extract objective structured knowledge from natural texts, which has attracted growing attention to build dedicated models with human experience.

In-Context Learning Open Information Extraction

OmniEvent: A Comprehensive, Fair, and Easy-to-Use Toolkit for Event Understanding

1 code implementation25 Sep 2023 Hao Peng, Xiaozhi Wang, Feng Yao, Zimu Wang, Chuzhao Zhu, Kaisheng Zeng, Lei Hou, Juanzi Li

Event understanding aims at understanding the content and relationship of events within texts, which covers multiple complicated information extraction tasks: event detection, event argument extraction, and event relation extraction.

Event Argument Extraction Event Detection +2

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

3 code implementations28 Aug 2023 Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

In this paper, we introduce LongBench, the first bilingual, multi-task benchmark for long context understanding, enabling a more rigorous evaluation of long context understanding.

16k Code Completion +2

LittleMu: Deploying an Online Virtual Teaching Assistant via Heterogeneous Sources Integration and Chain of Teach Prompts

1 code implementation11 Aug 2023 Shangqing Tu, Zheyuan Zhang, Jifan Yu, Chunyang Li, Siyu Zhang, Zijun Yao, Lei Hou, Juanzi Li

However, few MOOC platforms are providing human or virtual teaching assistants to support learning for massive online students due to the complexity of real-world online education scenarios and the lack of training data.

Language Modelling Question Answering +1

VisKoP: Visual Knowledge oriented Programming for Interactive Knowledge Base Question Answering

no code implementations6 Jul 2023 Zijun Yao, Yuanyong Chen, Xin Lv, Shulin Cao, Amy Xin, Jifan Yu, Hailong Jin, Jianjun Xu, Peng Zhang, Lei Hou, Juanzi Li

We present Visual Knowledge oriented Programming platform (VisKoP), a knowledge base question answering (KBQA) system that integrates human into the loop to edit and debug the knowledge base (KB) queries.

Knowledge Base Question Answering Program induction +2

The Devil is in the Details: On the Pitfalls of Event Extraction Evaluation

1 code implementation12 Jun 2023 Hao Peng, Xiaozhi Wang, Feng Yao, Kaisheng Zeng, Lei Hou, Juanzi Li, Zhiyuan Liu, Weixing Shen

In this paper, we check the reliability of EE evaluations and identify three major pitfalls: (1) The data preprocessing discrepancy makes the evaluation results on the same dataset not directly comparable, but the data preprocessing details are not widely noted and specified in papers.

Event Argument Extraction Event Detection +1

Learn to Not Link: Exploring NIL Prediction in Entity Linking

1 code implementation25 May 2023 Fangwei Zhu, Jifan Yu, Hailong Jin, Juanzi Li, Lei Hou, Zhifang Sui

We conduct a series of experiments with the widely used bi-encoder and cross-encoder entity linking models, results show that both types of NIL mentions in training data have a significant influence on the accuracy of NIL prediction.

Entity Linking Prediction

Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering

no code implementations24 May 2023 Jiajie Zhang, Shulin Cao, Tingjia Zhang, Xin Lv, Jiaxin Shi, Qi Tian, Juanzi Li, Lei Hou

To facilitate reasoning, we propose a novel two-stage XQA framework, Reasoning over Hierarchical Question Decomposition Tree (RoHT).

Question Answering

Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction

1 code implementation23 May 2023 Ji Qi, Chuchun Zhang, Xiaozhi Wang, Kaisheng Zeng, Jifan Yu, Jinxin Liu, Jiuding Sun, Yuxiang Chen, Lei Hou, Juanzi Li, Bin Xu

In this paper, we present the first benchmark that simulates the evaluation of open information extraction models in the real world, where the syntactic and expressive distributions under the same knowledge meaning may drift variously.

Language Modeling Language Modelling +2

ChatLog: Carefully Evaluating the Evolution of ChatGPT Across Time

1 code implementation27 Apr 2023 Shangqing Tu, Chunyang Li, Jifan Yu, Xiaozhi Wang, Lei Hou, Juanzi Li

We find some stable features that stay unchanged and apply them on the detection of ChatGPT-generated texts to improve the robustness of cross-version detection.

Natural Language Understanding

MoocRadar: A Fine-grained and Multi-aspect Knowledge Repository for Improving Cognitive Student Modeling in MOOCs

1 code implementation5 Apr 2023 Jifan Yu, Mengying Lu, Qingyang Zhong, Zijun Yao, Shangqing Tu, Zhengshan Liao, Xiaoya Li, Manli Li, Lei Hou, Hai-Tao Zheng, Juanzi Li, Jie Tang

Student modeling, the task of inferring a student's learning characteristics through their interactions with coursework, is a fundamental issue in intelligent education.

cognitive diagnosis Knowledge Tracing

GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation

1 code implementation26 Mar 2023 Ji Qi, Jifan Yu, Teng Tu, Kunyu Gao, Yifan Xu, Xinyu Guan, Xiaozhi Wang, Yuxiao Dong, Bin Xu, Lei Hou, Juanzi Li, Jie Tang, Weidong Guo, Hui Liu, Yu Xu

Despite the recent emergence of video captioning models, how to generate vivid, fine-grained video descriptions based on the background knowledge (i. e., long and informative commentary about the domain-specific scenes with appropriate reasoning) is still far from being solved, which however has great applications such as automatic sports narrative.

Video Captioning

Syntactically Robust Training on Partially-Observed Data for Open Information Extraction

1 code implementation17 Jan 2023 Ji Qi, Yuxiang Chen, Lei Hou, Juanzi Li, Bin Xu

In this paper, we propose a syntactically robust training framework that enables models to be trained on a syntactic-abundant distribution based on diverse paraphrase generation.

Open Information Extraction Paraphrase Generation +2

Answering Complex Logical Queries on Knowledge Graphs via Query Computation Tree Optimization

1 code implementation19 Dec 2022 Yushi Bai, Xin Lv, Juanzi Li, Lei Hou

QTO finds the optimal solution by a forward-backward propagation on the tree-like computation graph, i. e., query computation tree.

Complex Query Answering

Finding Skill Neurons in Pre-trained Transformer-based Language Models

1 code implementation14 Nov 2022 Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, Juanzi Li

Furthermore, we demonstrate the skill neurons are most likely generated in pre-training rather than fine-tuning by showing that the skill neurons found with prompt tuning are also crucial for other fine-tuning methods freezing neuron weights, such as the adapter-based tuning and BitFit.

Network Pruning

MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction

1 code implementation14 Nov 2022 Xiaozhi Wang, Yulin Chen, Ning Ding, Hao Peng, Zimu Wang, Yankai Lin, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, Jie zhou

It contains 103, 193 event coreference chains, 1, 216, 217 temporal relations, 57, 992 causal relations, and 15, 841 subevent relations, which is larger than existing datasets of all the ERE tasks by at least an order of magnitude.

Event Relation Extraction Relation +1

A Survey of Knowledge Enhanced Pre-trained Language Models

no code implementations11 Nov 2022 Linmei Hu, Zeyi Liu, Ziwang Zhao, Lei Hou, Liqiang Nie, Juanzi Li

We introduce appropriate taxonomies respectively for Natural Language Understanding (NLU) and Natural Language Generation (NLG) to highlight these two main tasks of NLP.

Natural Language Understanding Retrieval +3

ConstGCN: Constrained Transmission-based Graph Convolutional Networks for Document-level Relation Extraction

no code implementations8 Oct 2022 Ji Qi, Bin Xu, Kaisheng Zeng, Jinxin Liu, Jifan Yu, Qi Gao, Juanzi Li, Lei Hou

Document-level relation extraction with graph neural networks faces a fundamental graph construction gap between training and inference - the golden graph structure only available during training, which causes that most methods adopt heuristic or syntactic rules to construct a prior graph as a pseudo proxy.

Document-level Relation Extraction graph construction +1

Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsing

no code implementations4 Oct 2022 Lunyiu Nie, Jiuding Sun, Yanlin Wang, Lun Du, Lei Hou, Juanzi Li, Shi Han, Dongmei Zhang, Jidong Zhai

The recent prevalence of pretrained language models (PLMs) has dramatically shifted the paradigm of semantic parsing, where the mapping from natural language utterances to structured logical forms is now formulated as a Seq2Seq task.

Decoder Hallucination +2

Towards a General Pre-training Framework for Adaptive Learning in MOOCs

1 code implementation18 Jul 2022 Qingyang Zhong, Jifan Yu, Zheyuan Zhang, Yiming Mao, Yuquan Wang, Yankai Lin, Lei Hou, Juanzi Li, Jie Tang

Adaptive learning aims to stimulate and meet the needs of individual learners, which requires sophisticated system-level coordination of diverse tasks, including modeling learning resources, estimating student states, and making personalized recommendations.

Knowledge Tracing

GraphQ IR: Unifying the Semantic Parsing of Graph Query Languages with One Intermediate Representation

1 code implementation24 May 2022 Lunyiu Nie, Shulin Cao, Jiaxin Shi, Jiuding Sun, Qi Tian, Lei Hou, Juanzi Li, Jidong Zhai

Subject to the huge semantic gap between natural and formal languages, neural semantic parsing is typically bottlenecked by its complexity of dealing with both input semantics and output syntax.

Few-Shot Learning Semantic Parsing

LEVEN: A Large-Scale Chinese Legal Event Detection Dataset

1 code implementation Findings (ACL) 2022 Feng Yao, Chaojun Xiao, Xiaozhi Wang, Zhiyuan Liu, Lei Hou, Cunchao Tu, Juanzi Li, Yun Liu, Weixing Shen, Maosong Sun

However, existing Legal Event Detection (LED) datasets only concern incomprehensive event types and have limited annotated data, which restricts the development of LED methods and their downstream applications.

Event Detection Retrieval

Schema-Free Dependency Parsing via Sequence Generation

no code implementations28 Jan 2022 Boda Lin, Zijun Yao, Jiaxin Shi, Shulin Cao, Binghao Tang, Si Li, Yong Luo, Juanzi Li, Lei Hou

To remedy these drawbacks, we propose to achieve universal and schema-free Dependency Parsing (DP) via Sequence Generation (SG) DPSG by utilizing only the pre-trained language model (PLM) without any auxiliary structures or parsing algorithms.

Decoder Dependency Parsing +1

SQUIRE: A Sequence-to-sequence Framework for Multi-hop Knowledge Graph Reasoning

1 code implementation17 Jan 2022 Yushi Bai, Xin Lv, Juanzi Li, Lei Hou, Yincen Qu, Zelin Dai, Feiyu Xiong

Multi-hop knowledge graph (KG) reasoning has been widely studied in recent years to provide interpretable predictions on missing links with evidential paths.

Decoder Navigate +1

Interactive Contrastive Learning for Self-supervised Entity Alignment

no code implementations17 Jan 2022 Kaisheng Zeng, Zhenhao Dong, Lei Hou, Yixin Cao, Minghao Hu, Jifan Yu, Xin Lv, Juanzi Li, Ling Feng

Self-supervised entity alignment (EA) aims to link equivalent entities across different knowledge graphs (KGs) without seed alignments.

Contrastive Learning Entity Alignment +1

On Transferability of Prompt Tuning for Natural Language Processing

1 code implementation NAACL 2022 Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, Jie zhou

To explore whether we can improve PT via prompt transfer, we empirically investigate the transferability of soft prompts across different downstream tasks and PLMs in this work.

Natural Language Understanding Transfer Learning

Exploring Universal Intrinsic Task Subspace via Prompt Tuning

1 code implementation15 Oct 2021 Yujia Qin, Xiaozhi Wang, Yusheng Su, Yankai Lin, Ning Ding, Jing Yi, Weize Chen, Zhiyuan Liu, Juanzi Li, Lei Hou, Peng Li, Maosong Sun, Jie zhou

In the experiments, we study diverse few-shot NLP tasks and surprisingly find that in a 250-dimensional subspace found with 100 tasks, by only tuning 250 free parameters, we can recover 97% and 83% of the full prompt tuning performance for 100 seen tasks (using different training data) and 20 unseen tasks, respectively, showing great generalization ability of the found intrinsic task subspace.

Program Transfer for Answering Complex Questions over Knowledge Bases

1 code implementation ACL 2022 Shulin Cao, Jiaxin Shi, Zijun Yao, Xin Lv, Jifan Yu, Lei Hou, Juanzi Li, Zhiyuan Liu, Jinghui Xiao

In this paper, we propose the approach of program transfer, which aims to leverage the valuable program annotations on the rich-resourced KBs as external supervision signals to aid program induction for the low-resourced KBs that lack program annotations.

Program induction Semantic Parsing

Information Cocoons in Online Navigation

no code implementations14 Sep 2021 Lei Hou, Xue Pan, Kecheng Liu, Zimo Yang, Jianguo Liu, Tao Zhou

Social media and online navigation bring us enjoyable experience in accessing information, and simultaneously create information cocoons (ICs) in which we are unconsciously trapped with limited and biased information.

Recommendation Systems Retrieval

Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation

no code implementations2 Sep 2021 Hongyin Zhu, Hao Peng, Zhiheng Lyu, Lei Hou, Juanzi Li, Jinghui Xiao

In this paper, we propose a heterogeneous knowledge language model (\textbf{HKLM}), a unified pre-trained language model (PLM) for all forms of text, including unstructured text, semi-structured text, and well-structured text.

Language Modeling Language Modelling +1

TWAG: A Topic-Guided Wikipedia Abstract Generator

1 code implementation ACL 2021 Fangwei Zhu, Shangqing Tu, Jiaxin Shi, Juanzi Li, Lei Hou, Tong Cui

Wikipedia abstract generation aims to distill a Wikipedia abstract from web sources and has met significant success by adopting multi-document summarization techniques.

Abstract generation Document Summarization +2

Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision Making

1 code implementation ACL 2021 Zijun Yao, Chengjiang Li, Tiansi Dong, Xin Lv, Jifan Yu, Lei Hou, Juanzi Li, Yichi Zhang, Zelin Dai

Using a set of comparison features and a limited amount of annotated data, KAT Induction learns an efficient decision tree that can be interpreted by generating entity matching rules whose structure is advocated by domain experts.

Attribute Decision Making +3

Is Multi-Hop Reasoning Really Explainable? Towards Benchmarking Reasoning Interpretability

1 code implementation EMNLP 2021 Xin Lv, Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Yichi Zhang, Zelin Dai

However, we find in experiments that many paths given by these models are actually unreasonable, while little works have been done on interpretability evaluation for them.

Benchmarking Link Prediction

Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph

1 code implementation EMNLP 2020 Xin Lv, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu, Wei zhang, Yichi Zhang, Hao Kong, Suhui Wu

On the one hand, sparse KGs contain less information, which makes it difficult for the model to choose correct paths.

MOOCCube: A Large-scale Data Repository for NLP Applications in MOOCs

no code implementations ACL 2020 Jifan Yu, Gan Luo, Tong Xiao, Qingyang Zhong, Yuquan Wang, Wenzheng Feng, Junyi Luo, Chenyu Wang, Lei Hou, Juanzi Li, Zhiyuan Liu, Jie Tang

The prosperity of Massive Open Online Courses (MOOCs) provides fodder for many NLP and AI research for education applications, e. g., course concept extraction, prerequisite relation discovery, etc.

Finding structural hole spanners based on community forest model and diminishing marginal utility in large scale social networks

no code implementations Knowledge-Based Systems, 105916. 2020 Yan Zhang, Hua Xu, Yunfeng Xu, Junhui Deng, Juan Gu, Rui Ma, Jie Lai, Jiangtao Hu, Xiaoshuai Yu, Lei Hou, Lidong Gu, Yanling Wei, Yichao Xiao, Junhao Lu

In this paper, we try to give a more visual and detailed definition of structural hole spanner based on the existing work, and propose a novel algorithm to identify structural hole spanner based on community forest model and diminishing marginal utility.

Community Detection Link Prediction +2

Semi-supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-graph Model

1 code implementation IJCNLP 2019 Chengjiang Li, Yixin Cao, Lei Hou, Jiaxin Shi, Juanzi Li, Tat-Seng Chua

Specifically, as for the knowledge embedding model, we utilize TransE to implicitly complete two KGs towards consistency and learn relational constraints between entities.

Entity Alignment Graph Attention +2

Course Concept Expansion in MOOCs with External Knowledge and Interactive Game

no code implementations ACL 2019 Jifan Yu, Chenyu Wang, Gan Luo, Lei Hou, Juanzi Li, Jie Tang, Zhiyuan Liu

As Massive Open Online Courses (MOOCs) become increasingly popular, it is promising to automatically provide extracurricular knowledge for MOOC users.

Adapting Meta Knowledge Graph Information for Multi-Hop Reasoning over Few-Shot Relations

1 code implementation IJCNLP 2019 Xin Lv, Yuxian Gu, Xu Han, Lei Hou, Juanzi Li, Zhiyuan Liu

Multi-hop knowledge graph (KG) reasoning is an effective and explainable method for predicting the target entity via reasoning paths in query answering (QA) task.

Link Prediction Meta-Learning

Neural Collective Entity Linking

1 code implementation COLING 2018 Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu

To address this issue, we propose a novel neural model for collective entity linking, named as NCEL.

Entity Linking

Differentiating Concepts and Instances for Knowledge Graph Embedding

1 code implementation EMNLP 2018 Xin Lv, Lei Hou, Juanzi Li, Zhiyuan Liu

Most conventional knowledge embedding methods encode both entities (concepts and instances) and relations as vectors in a low dimensional semantic space equally, ignoring the difference between concepts and instances.

Knowledge Graph Embedding Knowledge Graphs +2

Learning to Embed Sentences Using Attentive Recursive Trees

2 code implementations6 Nov 2018 Jiaxin Shi, Lei Hou, Juanzi Li, Zhiyuan Liu, Hanwang Zhang

Sentence embedding is an effective feature representation for most deep learning-based NLP tasks.

Sentence Sentence Embedding +1

Cannot find the paper you are looking for? You can Submit a new open access paper.