Search Results for author: Hongru Wang

Found 41 papers, 18 papers with code

SMART: Self-Aware Agent for Tool Overuse Mitigation

no code implementations17 Feb 2025 Cheng Qian, Emre Can Acikgoz, Hongru Wang, Xiusi Chen, Avirup Sil, Dilek Hakkani-Tür, Gokhan Tur, Heng Ji

To support this paradigm, we introduce SMART-ER, a dataset spanning three domains, where reasoning alternates between parametric knowledge and tool-dependent steps, with each step enriched by rationales explaining when tools are necessary.

Large Language Model

NILE: Internal Consistency Alignment in Large Language Models

no code implementations21 Dec 2024 Minda Hu, Qiyuan Zhang, YuFei Wang, Bowei He, Hongru Wang, Jingyan Zhou, Liangyou Li, Yasheng Wang, Chen Ma, Irwin King

However, existing IFT datasets often contain knowledge that is inconsistent with LLMs' internal knowledge learned from the pre-training phase, which can greatly affect the efficacy of IFT.

UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models

1 code implementation16 Dec 2024 Boyang Xue, Fei Mi, Qi Zhu, Hongru Wang, Rui Wang, Sheng Wang, Erxin Yu, Xuming Hu, Kam-Fai Wong

Despite demonstrating impressive capabilities, Large Language Models (LLMs) still often struggle to accurately express the factual knowledge they possess, especially in cases where the LLMs' knowledge boundaries are ambiguous.

Question Answering

Compound-QA: A Benchmark for Evaluating LLMs on Compound Questions

no code implementations15 Nov 2024 Yutao Hou, Yajing Luo, Zhiwen Ruan, Hongru Wang, Weifeng Ge, Yun Chen, Guanhua Chen

In this paper, we introduce Compound Question Synthesis (CQ-Syn) to create the Compound-QA benchmark, focusing on compound questions with multiple sub-questions.

Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering

1 code implementation21 Oct 2024 Yu Zhao, Alessio Devoto, Giwon Hong, Xiaotang Du, Aryo Pradipta Gema, Hongru Wang, Xuanli He, Kam-Fai Wong, Pasquale Minervini

In this work, we propose \textsc{SpARE}, a \emph{training-free} representation engineering method that uses pre-trained sparse auto-encoders (SAEs) to control the knowledge selection behaviour of LLMs.

Open-Domain Question Answering

Analysing the Residual Stream of Language Models Under Knowledge Conflicts

1 code implementation21 Oct 2024 Yu Zhao, Xiaotang Du, Giwon Hong, Aryo Pradipta Gema, Alessio Devoto, Hongru Wang, Xuanli He, Kam-Fai Wong, Pasquale Minervini

Through probing tasks, we find that LLMs can internally register the signal of knowledge conflict in the residual stream, which can be accurately detected by probing the intermediate model activations.

MlingConf: A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models

1 code implementation16 Oct 2024 Boyang Xue, Hongru Wang, Rui Wang, Sheng Wang, Zezhong Wang, Yiming Du, Bin Liang, Kam-Fai Wong

This paper addresses this gap by introducing a comprehensive investigation of Multilingual Confidence estimation (MlingConf) on LLMs, focusing on both language-agnostic (LA) and language-specific (LS) tasks to explore the performance and language dominance effects of multilingual confidence estimations on different tasks.

AppBench: Planning of Multiple APIs from Various APPs for Complex User Instruction

1 code implementation10 Oct 2024 Hongru Wang, Rui Wang, Boyang Xue, Heming Xia, Jingtao Cao, Zeming Liu, Jeff Z. Pan, Kam-Fai Wong

In this paper, we introduce \texttt{AppBench}, the first benchmark to evaluate LLMs' ability to plan and execute multiple APIs from various sources in order to complete the user's task.

In-Context Learning

VLEU: a Method for Automatic Evaluation for Generalizability of Text-to-Image Models

1 code implementation23 Sep 2024 Jingtao Cao, Zheng Zhang, Hongru Wang, Kam-Fai Wong

Progress in Text-to-Image (T2I) models has significantly improved the generation of images from textual descriptions.

Image Generation

SoP: Unlock the Power of Social Facilitation for Automatic Jailbreak Attack

1 code implementation2 Jul 2024 Yan Yang, Zeguan Xiao, Xin Lu, Hongru Wang, Hailiang Huang, Guanhua Chen, Yun Chen

The widespread applications of large language models (LLMs) have brought about concerns regarding their potential misuse.

Red Teaming Safety Alignment

SeRTS: Self-Rewarding Tree Search for Biomedical Retrieval-Augmented Generation

no code implementations17 Jun 2024 Minda Hu, Licheng Zong, Hongru Wang, Jingyan Zhou, Jingjing Li, Yichen Gao, Kam-Fai Wong, Yu Li, Irwin King

By combining the reasoning capabilities of LLMs with the effectiveness of tree search, SeRTS boosts the zero-shot performance of retrieving high-quality and informative results for RAG.

Question Answering RAG +1

OSPC: Detecting Harmful Memes with Large Language Model as a Catalyst

no code implementations14 Jun 2024 Jingtao Cao, Zheng Zhang, Hongru Wang, Bin Liang, Hao Wang, Kam-Fai Wong

Utilizing the BLIP model for image captioning, PP-OCR and TrOCR for text recognition across multiple languages, and the Qwen LLM for nuanced language understanding, our system is capable of identifying harmful content in memes created in English, Chinese, Malay, and Tamil.

Image Captioning Language Modeling +4

AutoPSV: Automated Process-Supervised Verifier

2 code implementations27 May 2024 Jianqiao Lu, Zhiyang Dou, Hongru Wang, Zeyu Cao, Jianbo Dai, Yingjia Wan, Zhijiang Guo

\textsc{AutoPSV} begins by training a verification model on the correctness of final answers, enabling it to generate automatic process annotations.

Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challenges

no code implementations17 May 2024 Xiaoming Shi, Zeming Liu, Li Du, Yuxuan Wang, Hongru Wang, Yuhang Guo, Tong Ruan, Jie Xu, Shaoting Zhang

As a result, an overview of the categories, methods, and evaluation of medical dialogue systems remain limited and underspecified, hindering the further improvement of this area.

Survey

Knowledge Conflicts for LLMs: A Survey

1 code implementation13 Mar 2024 Rongwu Xu, Zehan Qi, Zhijiang Guo, Cunxiang Wang, Hongru Wang, Yue Zhang, Wei Xu

This survey provides an in-depth analysis of knowledge conflicts for large language models (LLMs), highlighting the complex challenges they encounter when blending contextual and parametric knowledge.

Misinformation Survey

Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Models

no code implementations5 Mar 2024 Rui Wang, Fei Mi, Yi Chen, Boyang Xue, Hongru Wang, Qi Zhu, Kam-Fai Wong, Ruifeng Xu

2) Role Prompting assigns a central prompt to the general domain and a unique role prompt to each specific domain to minimize inter-domain confusion during training.

Domain Adaptation

UniRetriever: Multi-task Candidates Selection for Various Context-Adaptive Conversational Retrieval

no code implementations26 Feb 2024 Hongru Wang, Boyang Xue, Baohang Zhou, Rui Wang, Fei Mi, Weichao Wang, Yasheng Wang, Kam-Fai Wong

Conversational retrieval refers to an information retrieval system that operates in an iterative and interactive manner, requiring the retrieval of various external resources, such as persona, knowledge, and even response, to effectively engage with the user and successfully complete the dialogue.

Information Retrieval Retrieval

Self-DC: When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questions

no code implementations21 Feb 2024 Hongru Wang, Boyang Xue, Baohang Zhou, Tianhua Zhang, Cunxiang Wang, Huimin Wang, Guanhua Chen, Kam-Fai Wong

Previous research has typically concentrated on leveraging the internal knowledge of Large Language Models (LLMs) to answer known questions (i. e., \textit{internal reasoning such as generate-then-read}).

Binary Classification Open-Domain Question Answering +1

A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models

1 code implementation21 Feb 2024 Boyang Xue, Hongru Wang, Rui Wang, Sheng Wang, Zezhong Wang, Yiming Du, Bin Liang, Kam-Fai Wong

This paper addresses this gap by introducing a comprehensive investigation of Multilingual Confidence estimation (MlingConf) on LLMs, focusing on both language-agnostic (LA) and language-specific (LS) tasks to explore the performance and language dominance effects of multilingual confidence estimations on different tasks.

UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems

no code implementations24 Jan 2024 Hongru Wang, WenYu Huang, Yang Deng, Rui Wang, Zezhong Wang, YuFei Wang, Fei Mi, Jeff Z. Pan, Kam-Fai Wong

To better plan and incorporate the use of multiple sources in generating personalized response, we firstly decompose it into three sub-tasks: Knowledge Source Selection, Knowledge Retrieval, and Response Generation.

RAG Response Generation +1

A Survey of the Evolution of Language Model-Based Dialogue Systems

no code implementations28 Nov 2023 Hongru Wang, Lingzhi Wang, Yiming Du, Liang Chen, Jingyan Zhou, YuFei Wang, Kam-Fai Wong

This survey delves into the historical trajectory of dialogue systems, elucidating their intricate relationship with advancements in language models by categorizing this evolution into four distinct stages, each marked by pivotal LM breakthroughs: 1) Early_Stage: characterized by statistical LMs, resulting in rule-based or machine-learning-driven dialogue_systems; 2) Independent development of TOD and ODD based on neural_language_models (NLM; e. g., LSTM and GRU), since NLMs lack intrinsic knowledge in their parameters; 3) fusion between different types of dialogue systems with the advert of pre-trained_language_models (PLMs), starting from the fusion between four_sub-tasks_within_TOD, and then TOD_with_ODD; and 4) current LLM-based_dialogue_system, wherein LLMs can be used to conduct TOD and ODD seamlessly.

Language Modeling Language Modelling +1

Large Language Models as Source Planner for Personalized Knowledge-grounded Dialogue

no code implementations13 Oct 2023 Hongru Wang, Minda Hu, Yang Deng, Rui Wang, Fei Mi, Weichao Wang, Yasheng Wang, Wai-Chung Kwan, Irwin King, Kam-Fai Wong

Open-domain dialogue system usually requires different sources of knowledge to generate more informative and evidential responses.

Response Generation

Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignment

1 code implementation12 Oct 2023 Boyang Xue, Weichao Wang, Hongru Wang, Fei Mi, Rui Wang, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu, Kam-Fai Wong

Inspired by previous work which identified that feed-forward networks (FFNs) within Transformers are responsible for factual knowledge expressions, we investigate two methods to efficiently improve the factual expression capability {of FFNs} by knowledge enhancement and alignment respectively.

TPE: Towards Better Compositional Reasoning over Conceptual Tools with Multi-persona Collaboration

no code implementations28 Sep 2023 Hongru Wang, Huimin Wang, Lingzhi Wang, Minda Hu, Rui Wang, Boyang Xue, Hongyuan Lu, Fei Mi, Kam-Fai Wong

Large language models (LLMs) have demonstrated exceptional performance in planning the use of various functional tools, such as calculators and retrievers, particularly in question-answering tasks.

Question Answering Response Generation

JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialog Policy Learning

1 code implementation1 Sep 2023 Wai-Chung Kwan, Huimin Wang, Hongru Wang, Zezhong Wang, Xian Wu, Yefeng Zheng, Kam-Fai Wong

In addition, JoTR employs reinforcement learning with a reward-shaping mechanism to efficiently finetune the word-level dialogue policy, which allows the model to learn from its interactions, improving its performance over time.

Action Generation Diversity

Enhancing Large Language Models Against Inductive Instructions with Dual-critique Prompting

1 code implementation23 May 2023 Rui Wang, Hongru Wang, Fei Mi, Yi Chen, Boyang Xue, Kam-Fai Wong, Ruifeng Xu

Numerous works are proposed to align large language models (LLMs) with human intents to better fulfill instructions, ensuring they are trustful and helpful.

counterfactual Fact Checking

Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration

1 code implementation23 May 2023 Yang Deng, Lizi Liao, Liang Chen, Hongru Wang, Wenqiang Lei, Tat-Seng Chua

Conversational systems based on Large Language Models (LLMs), such as ChatGPT, show exceptional proficiency in context understanding and response generation.

Descriptive Response Generation

Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs

2 code implementations19 May 2023 Hongru Wang, Rui Wang, Fei Mi, Yang Deng, Zezhong Wang, Bin Liang, Ruifeng Xu, Kam-Fai Wong

Large Language Models (LLMs), such as \texttt{ChatGPT}, greatly empower dialogue systems with strong language understanding and generation capabilities.

Question Answering Semantic Similarity +1

DIGAT: Modeling News Recommendation with Dual-Graph Interaction

1 code implementation11 Oct 2022 Zhiming Mao, Jian Li, Hongru Wang, Xingshan Zeng, Kam-Fai Wong

Second, existing graph-based NR methods are promising but lack effective news-user feature interaction, rendering the graph-based recommendation suboptimal.

Graph Attention News Recommendation +1

Integrating Pretrained Language Model for Dialogue Policy Learning

no code implementations2 Nov 2021 Hongru Wang, Huimin Wang, Zezhong Wang, Kam-Fai Wong

Reinforcement Learning (RL) has been witnessed its potential for training a dialogue policy agent towards maximizing the accumulated rewards given from users.

Language Modeling Language Modelling +2

Inconsistent Few-Shot Relation Classification via Cross-Attentional Prototype Networks with Contrastive Learning

no code implementations13 Oct 2021 Hongru Wang, Zhijing Jin, Jiarun Cao, Gabriel Pui Cheong Fung, Kam-Fai Wong

However, previous works rarely investigate the effects of a different number of classes (i. e., $N$-way) and number of labeled data per class (i. e., $K$-shot) during training vs. testing.

Contrastive Learning Few-Shot Relation Classification +1

TopicRefine: Joint Topic Prediction and Dialogue Response Generation for Multi-turn End-to-End Dialogue System

no code implementations11 Sep 2021 Hongru Wang, Mingyu Cui, Zimo Zhou, Gabriel Pui Cheong Fung, Kam-Fai Wong

A multi-turn dialogue always follows a specific topic thread, and topic shift at the discourse level occurs naturally as the conversation progresses, necessitating the model's ability to capture different topics and generate topic-aware responses.

Response Generation

KddRES: A Multi-level Knowledge-driven Dialogue Dataset for Restaurant Towards Customized Dialogue System

no code implementations17 Nov 2020 Hongru Wang, Min Li, Zimo Zhou, Gabriel Pui Cheong Fung, Kam-Fai Wong

In this paper, we publish a first Cantonese knowledge-driven Dialogue Dataset for REStaurant (KddRES) in Hong Kong, which grounds the information in multi-turn conversations to one specific restaurant.

Diversity

Cannot find the paper you are looking for? You can Submit a new open access paper.