Search Results for author: Liangming Pan

Found 58 papers, 42 papers with code

Automatic True/False Question Generation for Educational Purpose

no code implementations NAACL (BEA) 2022 Bowei Zou, Pengfei Li, Liangming Pan, Ai Ti Aw

In field of teaching, true/false questioning is an important educational method for assessing students’ general understanding of learning materials.

Fact Verification Question Generation +2

Aristotle: Mastering Logical Reasoning with A Logic-Complete Decompose-Search-Resolve Framework

no code implementations22 Dec 2024 Jundong Xu, Hao Fei, Meng Luo, Qian Liu, Liangming Pan, William Yang Wang, Preslav Nakov, Mong-Li Lee, Wynne Hsu

In the context of large language models (LLMs), current advanced reasoning methods have made impressive strides in various reasoning tasks.

Logical Reasoning

Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning

no code implementations15 Dec 2024 Shengqiong Wu, Hao Fei, Liangming Pan, William Yang Wang, Shuicheng Yan, Tat-Seng Chua

Our framework systematically addresses potential issues in both visual and textual inputs by verifying and integrating perception-level information with cognition-level commonsense knowledge, ensuring more reliable outputs.

Hallucination

RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios

1 code implementation12 Dec 2024 Ruiwen Zhou, Wenyue Hua, Liangming Pan, Sitao Cheng, Xiaobao Wu, En Yu, William Yang Wang

This paper introduces RuleArena, a novel and challenging benchmark designed to evaluate the ability of large language models (LLMs) to follow complex, real-world rules in reasoning.

Logical Reasoning Long-Context Understanding

Improving Causal Reasoning in Large Language Models: A Survey

1 code implementation22 Oct 2024 Longxuan Yu, Delin Chen, Siheng Xiong, Qingyang Wu, Qingzhen Liu, Dawei Li, Zhikai Chen, Xiaoze Liu, Liangming Pan

In this survey, we provide a comprehensive review of research aimed at enhancing LLMs for causal reasoning.

Decision Making Survey

COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement

1 code implementation12 Oct 2024 Yuxi Xie, Anirudh Goyal, Xiaobao Wu, Xunjian Yin, Xiao Xu, Min-Yen Kan, Liangming Pan, William Yang Wang

Our approach models multiple token dependencies within manageable context windows, enabling the model to perform iterative refinement internally during the generation process.

Code Generation Computational Efficiency +3

Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models

1 code implementation10 Oct 2024 Sitao Cheng, Liangming Pan, Xunjian Yin, Xinyi Wang, William Yang Wang

To support this investigation, we introduce ECHOQA, a benchmark spanning scientific, factual, and commonsense knowledge.

Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement

2 code implementations6 Oct 2024 Xunjian Yin, Xinyi Wang, Liangming Pan, Xiaojun Wan, William Yang Wang

The rapid advancement of large language models (LLMs) has significantly enhanced the capabilities of AI-driven agents across various tasks.

Mathematical Reasoning Meta-Learning

TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoning

1 code implementation18 Sep 2024 Xinyuan Lu, Liangming Pan, Yubo Ma, Preslav Nakov, Min-Yen Kan

Current Large Language Models (LLMs) exhibit limited ability to understand table structures and to apply precise numerical reasoning, which is crucial for tasks such as table question answering (TQA) and table-based fact verification (TFV).

Fact Verification Question Answering +1

SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation

1 code implementation27 Jun 2024 Zijun Yao, Weijian Qi, Liangming Pan, Shulin Cao, Linmei Hu, Weichuan Liu, Lei Hou, Juanzi Li

This paper introduces Self-aware Knowledge Retrieval (SeaKR), a novel adaptive RAG model that extracts self-aware uncertainty of LLMs from their internal states.

Question Answering RAG +1

Investigating the Transferability of Code Repair for Low-Resource Programming Languages

no code implementations21 Jun 2024 Kyle Wong, Alfonso Amayuelas, Liangming Pan, William Yang Wang

To explain this behavior, we perform a further analysis and find that contrary to preexisting beliefs, the correlation between reasoning ability and code correction ability is weak.

Code Generation Code Repair

Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusion

2 code implementations28 May 2024 Xiaobao Wu, Xinshuai Dong, Liangming Pan, Thong Nguyen, Anh Tuan Luu

However, existing models suffer from repetitive topic and unassociated topic issues, failing to reveal the evolution and hindering further applications.

Contrastive Learning Diversity +2

Faithful Logical Reasoning via Symbolic Chain-of-Thought

1 code implementation28 May 2024 Jundong Xu, Hao Fei, Liangming Pan, Qian Liu, Mong-Li Lee, Wynne Hsu

Technically, building upon an LLM, SymbCoT 1) first translates the natural language context into the symbolic format, and then 2) derives a step-by-step plan to solve the problem with symbolic logical rules, 3) followed by a verifier to check the translation and reasoning chain.

Logical Reasoning

AKEW: Assessing Knowledge Editing in the Wild

1 code implementation29 Feb 2024 Xiaobao Wu, Liangming Pan, William Yang Wang, Anh Tuan Luu

Knowledge editing injects knowledge updates into language models to keep them correct and up-to-date.

counterfactual knowledge editing +1

SciAgent: Tool-augmented Language Models for Scientific Reasoning

no code implementations18 Feb 2024 Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, Aixin Sun, Hany Awadalla, Weizhu Chen

To make this task more practical and solvable for LLMs, we introduce a new task setting named tool-augmented scientific reasoning.

Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation

1 code implementation5 Feb 2024 Xinyi Wang, Alfonso Amayuelas, Kexun Zhang, Liangming Pan, Wenhu Chen, William Yang Wang

To understand how pre-training with a next-token prediction objective contributes to the emergence of such reasoning capability, we propose that we can view an LM as deriving new conclusions by aggregating indirect reasoning paths seen at pre-training time.

Knowledge Graphs Math

Position: AI/ML Influencers Have a Place in the Academic Process

no code implementations24 Jan 2024 Iain Xie Weissburg, Mehir Arora, Xinyi Wang, Liangming Pan, William Yang Wang

As the number of accepted papers at AI and ML conferences reaches into the thousands, it has become unclear how researchers access and read research publications.

Causal Inference Diversity +1

Efficient Online Data Mixing For Language Model Pre-Training

1 code implementation5 Dec 2023 Alon Albalak, Liangming Pan, Colin Raffel, William Yang Wang

The data used to pretrain large language models has a decisive impact on a model's downstream performance, which has led to a large body of work on data selection methods that aim to automatically determine the most suitable data to use for pretraining.

Language Modeling Language Modelling +1

A Survey on Detection of LLMs-Generated Content

1 code implementation24 Oct 2023 Xianjun Yang, Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Petzold, William Yang Wang, Wei Cheng

The burgeoning capabilities of advanced large language models (LLMs) such as ChatGPT have led to an increase in synthetic content generation with implications across a variety of sectors, including media, cybersecurity, public discourse, and education.

Survey

QACHECK: A Demonstration System for Question-Guided Multi-Hop Fact-Checking

1 code implementation11 Oct 2023 Liangming Pan, Xinyuan Lu, Min-Yen Kan, Preslav Nakov

Fact-checking real-world claims often requires complex, multi-step reasoning due to the absence of direct evidence to support or refute them.

Decision Making Fact Checking +1

Towards Verifiable Generation: A Benchmark for Knowledge-aware Language Model Attribution

1 code implementation9 Oct 2023 Xinze Li, Yixin Cao, Liangming Pan, Yubo Ma, Aixin Sun

Although achieving great success, Large Language Models (LLMs) usually suffer from unreliable hallucinations.

Attribute Language Modeling +4

Investigating Zero- and Few-shot Generalization in Fact Verification

1 code implementation18 Sep 2023 Liangming Pan, Yunxiang Zhang, Min-Yen Kan

In this paper, we explore zero- and few-shot generalization for fact verification (FV), which aims to generalize the FV model trained on well-resourced domains (e. g., Wikipedia) to low-resourced domains that lack human annotations.

Fact Verification

FOLLOWUPQG: Towards Information-Seeking Follow-up Question Generation

no code implementations10 Sep 2023 Yan Meng, Liangming Pan, Yixin Cao, Min-Yen Kan

We introduce the task of real-world information-seeking follow-up question generation (FQG), which aims to generate follow-up questions seeking a more in-depth understanding of an initial question and answer.

Informativeness Question Generation +1

Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models

1 code implementation23 May 2023 Alfonso Amayuelas, Kyle Wong, Liangming Pan, Wenhu Chen, William Wang

This paper investigates the capabilities of Large Language Models (LLMs) in the context of understanding their knowledge and uncertainty over questions.

Known Unknowns Open-Ended Question Answering

On the Risk of Misinformation Pollution with Large Language Models

1 code implementation23 May 2023 Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, William Yang Wang

In this paper, we comprehensively investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation and its subsequent impact on information-intensive applications, particularly Open-Domain Question Answering (ODQA) systems.

Misinformation Open-Domain Question Answering

INSTRUCTSCORE: Explainable Text Generation Evaluation with Finegrained Feedback

2 code implementations23 May 2023 Wenda Xu, Danqing Wang, Liangming Pan, Zhenqiao Song, Markus Freitag, William Yang Wang, Lei LI

By harnessing both explicit human instruction and the implicit knowledge of GPT-4, we fine-tune a text evaluation metric based on LLaMA, producing both a score for generated text and a human readable diagnostic report.

Text Generation

SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables

1 code implementation22 May 2023 Xinyuan Lu, Liangming Pan, Qian Liu, Preslav Nakov, Min-Yen Kan

Current scientific fact-checking benchmarks exhibit several shortcomings, such as biases arising from crowd-sourced claims and an over-reliance on text-based evidence.

Claim Verification Fact Checking

Fact-Checking Complex Claims with Program-Guided Reasoning

2 code implementations22 May 2023 Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, Preslav Nakov

Fact-checking real-world claims often requires collecting multiple pieces of evidence and applying complex multi-step reasoning.

Fact Checking In-Context Learning

Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning

1 code implementation20 May 2023 Liangming Pan, Alon Albalak, Xinyi Wang, William Yang Wang

We also introduce a self-refinement module, which utilizes the symbolic solver's error messages to revise symbolic formalizations.

Logical Reasoning

Modeling What-to-ask and How-to-ask for Answer-unaware Conversational Question Generation

1 code implementation4 May 2023 Xuan Long Do, Bowei Zou, Shafiq Joty, Anh Tai Tran, Liangming Pan, Nancy F. Chen, Ai Ti Aw

In addition, we propose Conv-Distinct, a novel evaluation metric for CQG, to evaluate the diversity of the generated conversation from a context.

Question Generation Question-Generation +1

InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling

2 code implementations7 Apr 2023 Xiaobao Wu, Xinshuai Dong, Thong Nguyen, Chaoqun Liu, Liangming Pan, Anh Tuan Luu

Instead of the direct alignment in previous work, we propose a topic alignment with mutual information method.

Topic Models

Hashtag-Guided Low-Resource Tweet Classification

1 code implementation20 Feb 2023 Shizhe Diao, Sedrick Scott Keh, Liangming Pan, Zhiliang Tian, Yan Song, Tong Zhang

Social media classification tasks (e. g., tweet sentiment analysis, tweet stance detection) are challenging because social media posts are typically short, informal, and ambiguous.

Classification Sentiment Analysis +1

Modeling and Leveraging Prerequisite Context in Recommendation

1 code implementation23 Sep 2022 Hengchang Hu, Liangming Pan, Yiding Ran, Min-Yen Kan

Prerequisites can play a crucial role in users' decision-making yet recommendation systems have not fully utilized such contextual background knowledge.

Decision Making Recommendation Systems

CoHS-CQG: Context and History Selection for Conversational Question Generation

1 code implementation COLING 2022 Xuan Long Do, Bowei Zou, Liangming Pan, Nancy F. Chen, Shafiq Joty, Ai Ti Aw

While previous studies mainly focus on how to model the flow and alignment of the conversation, there has been no thorough study to date on which parts of the context and history are necessary for the model.

Question Generation Question-Generation +1

Attacking Open-domain Question Answering by Injecting Misinformation

1 code implementation15 Oct 2021 Liangming Pan, Wenhu Chen, Min-Yen Kan, William Yang Wang

We curate both human-written and model-generated false documents that we inject into the evidence corpus of QA models and assess the impact on the performance of these systems.

Misinformation Open-Domain Question Answering

Interpreting the Robustness of Neural NLP Models to Textual Perturbations

no code implementations Findings (ACL) 2022 Yunxiang Zhang, Liangming Pan, Samson Tan, Min-Yen Kan

In this work, we test the hypothesis that the extent to which a model is affected by an unseen textual perturbation (robustness) can be explained by the learnability of the perturbation (defined as how well the model learns to identify the perturbation with a small amount of evidence).

Data Augmentation

Zero-shot Fact Verification by Claim Generation

1 code implementation ACL 2021 Liangming Pan, Wenhu Chen, Wenhan Xiong, Min-Yen Kan, William Yang Wang

However, for each new domain that requires fact verification, creating a dataset by manually writing claims and linking them to their supporting evidence is expensive.

2k Fact Verification

Exploring Question-Specific Rewards for Generating Deep Questions

1 code implementation COLING 2020 Yuxi Xie, Liangming Pan, Dongzhe Wang, Min-Yen Kan, Yansong Feng

Recent question generation (QG) approaches often utilize the sequence-to-sequence framework (Seq2Seq) to optimize the log-likelihood of ground-truth questions using teacher forcing.

Question Generation Question-Generation

Multi-modal Cooking Workflow Construction for Food Recipes

no code implementations20 Aug 2020 Liangming Pan, Jingjing Chen, Jianlong Wu, Shaoteng Liu, Chong-Wah Ngo, Min-Yen Kan, Yu-Gang Jiang, Tat-Seng Chua

Understanding food recipe requires anticipating the implicit causal effects of cooking actions, such that the recipe can be converted into a graph describing the temporal workflow of the recipe.

Common Sense Reasoning Decoder

Semantic Graphs for Generating Deep Questions

1 code implementation ACL 2020 Liangming Pan, Yuxi Xie, Yansong Feng, Tat-Seng Chua, Min-Yen Kan

This paper proposes the problem of Deep Question Generation (DQG), which aims to generate complex questions that require reasoning over multiple pieces of information of the input passage.

Question Generation Question-Generation

Recent Advances in Neural Question Generation

no code implementations22 May 2019 Liangming Pan, Wenqiang Lei, Tat-Seng Chua, Min-Yen Kan

Emerging research in Neural Question Generation (NQG) has started to integrate a larger variety of inputs, and generating questions requiring higher levels of cognition.

Question Generation Question-Generation

Resource Mention Extraction for MOOC Discussion Forums

no code implementations21 Nov 2018 Ya-Hui An, Liangming Pan, Min-Yen Kan, Qiang Dong, Yan Fu

We propose the novel problem of learning resource mention identification in MOOC forums.

Diversity

Course Concept Extraction in MOOCs via Embedding-Based Graph Propagation

no code implementations IJCNLP 2017 Liangming Pan, Xiaochen Wang, Chengjiang Li, Juanzi Li, Jie Tang

Massive Open Online Courses (MOOCs), offering a new way to study online, are revolutionizing education.

Prerequisite Relation Learning for Concepts in MOOCs

no code implementations ACL 2017 Liangming Pan, Chengjiang Li, Juanzi Li, Jie Tang

What prerequisite knowledge should students achieve a level of mastery before moving forward to learn subsequent coursewares?

Relation Representation Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.