Search Results for author: Muhao Chen

Found 110 papers, 66 papers with code

Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment

2 code implementations12 Nov 2016 Muhao Chen, Yingtao Tian, Mohan Yang, Carlo Zaniolo

Many recent works have demonstrated the benefits of knowledge graph embeddings in completing monolingual knowledge graphs.

Entity Alignment Knowledge Graph Embeddings +2

Attention-based Natural Language Person Retrieval

no code implementations24 May 2017 Tao Zhou, Muhao Chen, Jie Yu, Demetri Terzopoulos

Following the recent progress in image classification and captioning using deep learning, we develop a novel natural language person retrieval system based on an attention mechanism.

Image Classification Person Retrieval +2

Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment

no code implementations18 Jun 2018 Muhao Chen, Yingtao Tian, Kai-Wei Chang, Steven Skiena, Carlo Zaniolo

Since many multilingual KGs also provide literal descriptions of entities, in this paper, we introduce an embedding-based approach which leverages a weakly aligned multilingual KG for semi-supervised cross-lingual learning using entity descriptions.

Entity Alignment Knowledge Graphs

Neural Article Pair Modeling for Wikipedia Sub-article Matching

1 code implementation31 Jul 2018 Muhao Chen, Changping Meng, Gang Huang, Carlo Zaniolo

Nowadays, editors tend to separate different subtopics of a long Wiki-pedia article into multiple sub-articles.

On2Vec: Embedding-based Relation Prediction for Ontology Population

no code implementations7 Sep 2018 Muhao Chen, Yingtao Tian, Xuelu Chen, Zijun Xue, Carlo Zaniolo

Recent advances in translation-based graph embedding methods for populating instance-level knowledge graphs lead to promising new approaching for the ontology population problem.

Graph Embedding Knowledge Graphs +2

Enhanced Network Embeddings via Exploiting Edge Labels

1 code implementation13 Sep 2018 Haochen Chen, Xiaofei Sun, Yingtao Tian, Bryan Perozzi, Muhao Chen, Steven Skiena

Network embedding methods aim at learning low-dimensional latent representation of nodes in a network.

Social and Information Networks Physics and Society

Embedding Uncertain Knowledge Graphs

1 code implementation26 Nov 2018 Xuelu Chen, Muhao Chen, Weijia Shi, Yizhou Sun, Carlo Zaniolo

However, there are many KGs that model uncertain knowledge, which typically model the inherent uncertainty of relations facts with a confidence score, and embedding such uncertain knowledge represents an unresolved challenge.

Binary Classification General Classification +3

Quantification and Analysis of Scientific Language Variation Across Research Fields

no code implementations4 Dec 2018 Pei Zhou, Muhao Chen, Kai-Wei Chang, Carlo Zaniolo

Quantifying differences in terminologies from various academic domains has been a longstanding problem yet to be solved.

Language Modelling

Learning to Identify High Betweenness Centrality Nodes from Scratch: A Novel Graph Neural Network Approach

1 code implementation24 May 2019 Changjun Fan, Li Zeng, Yuhui Ding, Muhao Chen, Yizhou Sun, Zhong Liu

By training on small-scale networks, the learned model is capable of assigning relative BC scores to nodes for any unseen networks, and thus identifying the highly-ranked nodes.

Community Detection

Learning Bilingual Word Embeddings Using Lexical Definitions

no code implementations WS 2019 Weijia Shi, Muhao Chen, Yingtao Tian, Kai-Wei Chang

Bilingual word embeddings, which representlexicons of different languages in a shared em-bedding space, are essential for supporting se-mantic and knowledge transfers in a variety ofcross-lingual NLP tasks.

Translation Word Alignment +1

Fast and Accurate Network Embeddings via Very Sparse Random Projection

2 code implementations30 Aug 2019 Haochen Chen, Syed Fahad Sultan, Yingtao Tian, Muhao Chen, Steven Skiena

Two key features of FastRP are: 1) it explicitly constructs a node similarity matrix that captures transitive relationships in a graph and normalizes matrix entries based on node degrees; 2) it utilizes very sparse random projection, which is a scalable optimization-free method for dimension reduction.

Dimensionality Reduction Network Embedding

Retrofitting Contextualized Word Embeddings with Paraphrases

no code implementations IJCNLP 2019 Weijia Shi, Muhao Chen, Pei Zhou, Kai-Wei Chang

Contextualized word embedding models, such as ELMo, generate meaningful representations of words and their context.

Sentence Sentence Classification +1

Knowledge Graph Alignment Network with Gated Multi-hop Neighborhood Aggregation

1 code implementation20 Nov 2019 Zequn Sun, Chengming Wang, Wei Hu, Muhao Chen, Jian Dai, Wei zhang, Yuzhong Qu

As the direct neighbors of counterpart entities are usually dissimilar due to the schema heterogeneity, AliNet introduces distant neighbors to expand the overlap between their neighborhood structures.

Entity Alignment Knowledge Graphs

A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs

1 code implementation10 Mar 2020 Zequn Sun, Qingheng Zhang, Wei Hu, Chengming Wang, Muhao Chen, Farahnaz Akrami, Chengkai Li

Recent advancement in KG embedding impels the advent of embedding-based entity alignment, which encodes entities in a continuous embedding space and measures entity similarities based on the learned embeddings.

Benchmarking Entity Alignment +1

Cross-lingual Entity Alignment with Incidental Supervision

1 code implementation EACL 2021 Muhao Chen, Weijia Shi, Ben Zhou, Dan Roth

Much research effort has been put to multilingual knowledge graph (KG) embedding methods to address the entity alignment task, which seeks to match entities in different languagespecific KGs that refer to the same real-world object.

Entity Alignment Knowledge Graphs

Visual Pivoting for (Unsupervised) Entity Alignment

2 code implementations28 Sep 2020 Fangyu Liu, Muhao Chen, Dan Roth, Nigel Collier

This work studies the use of visual semantic representations to align entities in heterogeneous knowledge graphs (KGs).

Ranked #3 on Multi-modal Entity Alignment on UMVM-oea-d-w-v1 (using extra training data)

Knowledge Graphs Multi-modal Entity Alignment

Knowledge Association with Hyperbolic Knowledge Graph Embeddings

1 code implementation EMNLP 2020 Zequn Sun, Muhao Chen, Wei Hu, Chengming Wang, Jian Dai, Wei zhang

Capturing associations for knowledge graphs (KGs) through entity alignment, entity type inference and other related tasks benefits NLP applications with comprehensive knowledge representations.

Entity Alignment Knowledge Graph Embeddings +1

Multilingual Knowledge Graph Completion via Ensemble Knowledge Transfer

1 code implementation Findings of the Association for Computational Linguistics 2020 Xuelu Chen, Muhao Chen, Changjun Fan, Ankith Uppunda, Yizhou Sun, Carlo Zaniolo

Predicting missing facts in a knowledge graph (KG) is a crucial task in knowledge base construction and reasoning, and it has been the subject of much research in recent works using KG embeddings.

Knowledge Graph Completion Self-Learning +1

Joint Constrained Learning for Event-Event Relation Extraction

no code implementations EMNLP 2020 Haoyu Wang, Muhao Chen, Hongming Zhang, Dan Roth

Understanding natural language involves recognizing how multiple event mentions structurally and temporally interact with each other.

Event Relation Extraction Relation +1

"What Are You Trying to Do?" Semantic Typing of Event Processes

no code implementations13 Oct 2020 Muhao Chen, Hongming Zhang, Haoyu Wang, Dan Roth

This paper studies a new cognitively motivated semantic typing task, multi-axis event process typing, that, given an event process, attempts to infer free-form type labels describing (i) the type of action made by the process and (ii) the type of object the process seeks to affect.

Learning-To-Rank Object +1

Analogous Process Structure Induction for Sub-event Sequence Prediction

no code implementations EMNLP 2020 Hongming Zhang, Muhao Chen, Haoyu Wang, Yangqiu Song, Dan Roth

Computational and cognitive studies of event understanding suggest that identifying, comprehending, and predicting events depend on having structured representations of a sequence of events and on conceptualizing (abstracting) its components into (soft) event categories.

What Are You Trying to Do? Semantic Typing of Event Processes

no code implementations CONLL 2020 Muhao Chen, Hongming Zhang, Haoyu Wang, Dan Roth

This paper studies a new cognitively motivated semantic typing task, multi-axis event process typing, that, given anevent process, attempts to infer free-form typelabels describing (i) the type of action made bythe process and (ii) the type of object the pro-cess seeks to affect.

Learning-To-Rank Vocal Bursts Type Prediction

Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks

1 code implementation15 Dec 2020 Cunchao Zhu, Muhao Chen, Changjun Fan, Guangquan Cheng, Yan Zhan

Since such temporal knowledge graphs often suffer from incompleteness, it is important to develop time-aware representation learning models that help to infer the missing temporal facts.

Representation Learning

An Improved Baseline for Sentence-level Relation Extraction

1 code implementation2 Feb 2021 Wenxuan Zhou, Muhao Chen

Sentence-level relation extraction (RE) aims at identifying the relationship between two entities in a sentence.

Relation Relation Extraction +1

Bio-JOIE: Joint Representation Learning of Biological Knowledge Bases

1 code implementation7 Mar 2021 Junheng Hao, Chelsea Ju, Muhao Chen, Yizhou Sun, Carlo Zaniolo, Wei Wang

Leveraging a wide-range of biological knowledge, such as gene ontology and protein-protein interaction (PPI) networks from other closely related species presents a vital approach to infer the molecular impact of a new species.

Representation Learning Type prediction

Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts

1 code implementation15 Mar 2021 Junheng Hao, Muhao Chen, Wenchao Yu, Yizhou Sun, Wei Wang

The cross-view association model is learned to bridge the embeddings of ontological concepts and their corresponding instance-view entities.

Entity Typing Knowledge Graphs +1

Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning

1 code implementation NAACL 2021 Xuelu Chen, Michael Boratko, Muhao Chen, Shib Sankar Dasgupta, Xiang Lorraine Li, Andrew McCallum

Knowledge bases often consist of facts which are harvested from a variety of sources, many of which are noisy and some of which conflict, resulting in a level of uncertainty for each triple.

Knowledge Graph Embedding

PaCo: Preconditions Attributed to Commonsense Knowledge

1 code implementation18 Apr 2021 Ehsan Qasemi, Filip Ilievski, Muhao Chen, Pedro Szekely

To address this gap, we propose a novel challenge of reasoning with circumstantial preconditions.

Common Sense Reasoning

Retrieving Complex Tables with Multi-Granular Graph Representation Learning

1 code implementation4 May 2021 Fei Wang, Kexuan Sun, Muhao Chen, Jay Pujara, Pedro Szekely

The task of natural language table retrieval (NLTR) seeks to retrieve semantically relevant tables based on natural language queries.

Graph Representation Learning Natural Language Queries +2

Knowing the No-match: Entity Alignment with Dangling Cases

1 code implementation ACL 2021 Zequn Sun, Muhao Chen, Wei Hu

Since KGs possess different sets of entities, there could be entities that cannot find alignment across them, leading to the problem of dangling entities.

Abstention Prediction Entity Alignment +1

Design and analysis of deployable clustered tensegrity cable domes

no code implementations15 Jun 2021 Shuo Ma, Muhao Chen, Xingfei Yuan, Robert E. Skelton

Results show that the proposed CTS cable dome always has one prestress mode and is globally stable in its deployment trajectory.

Do Language Models Perform Generalizable Commonsense Inference?

1 code implementation Findings (ACL) 2021 Peifeng Wang, Filip Ilievski, Muhao Chen, Xiang Ren

Inspired by evidence that pretrained language models (LMs) encode commonsense knowledge, recent work has applied LMs to automatically populate commonsense knowledge graphs (CKGs).

Knowledge Graphs

Event-Centric Natural Language Processing

no code implementations ACL 2021 Muhao Chen, Hongming Zhang, Qiang Ning, Manling Li, Heng Ji, Kathleen McKeown, Dan Roth

This tutorial targets researchers and practitioners who are interested in AI technologies that help machines understand natural language text, particularly real-world events described in the text.

Table-based Fact Verification with Salience-aware Learning

1 code implementation Findings (EMNLP) 2021 Fei Wang, Kexuan Sun, Jay Pujara, Pedro Szekely, Muhao Chen

From one perspective, our system conducts masked salient token prediction to enhance the model for alignment and reasoning between the table and the statement.

counterfactual Data Augmentation +2

Learning Constraints and Descriptive Segmentation for Subevent Detection

no code implementations EMNLP 2021 Haoyu Wang, Hongming Zhang, Muhao Chen, Dan Roth

The task of subevent detection aims to resolve this granularity issue, recognizing the membership of multi-granular events in event complexes.

Descriptive Text Segmentation

HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning

no code implementations Findings (EMNLP) 2021 Mingyu Derek Ma, Muhao Chen, Te-Lin Wu, Nanyun Peng

Taxonomies are valuable resources for many applications, but the limited coverage due to the expensive manual curation process hinders their general applicability.

Representation Learning Taxonomy Expansion

Salience-Aware Event Chain Modeling for Narrative Understanding

no code implementations EMNLP 2021 Xiyang Zhang, Muhao Chen, Jonathan May

Storytelling, whether via fables, news reports, documentaries, or memoirs, can be thought of as the communication of interesting and related events that, taken together, form a concrete process.

Question Answering

Dynamics and control of clustered tensegrity systems

no code implementations17 Oct 2021 Shuo Ma, Muhao Chen, Robert E. Skelton

This paper presents the formulations of nonlinear and linearized statics, dynamics, and control for any clustered tensegrity system (CTS).

Structure-Aware Label Smoothing for Graph Neural Networks

no code implementations1 Dec 2021 Yiwei Wang, Yujun Cai, Yuxuan Liang, Wei Wang, Henghui Ding, Muhao Chen, Jing Tang, Bryan Hooi

Representing a label distribution as a one-hot vector is a common practice in training node classification models.

Classification Node Classification

Contextualized Scene Imagination for Generative Commonsense Reasoning

1 code implementation ICLR 2022 Peifeng Wang, Jonathan Zamora, Junfeng Liu, Filip Ilievski, Muhao Chen, Xiang Ren

In this paper, we propose an Imagine-and-Verbalize (I&V) method, which learns to imagine a relational scene knowledge graph (SKG) with relations between the input concepts, and leverage the SKG as a constraint when generating a plausible scene description.

Common Sense Reasoning Descriptive +2

Sharpness-Aware Minimization with Dynamic Reweighting

no code implementations16 Dec 2021 Wenxuan Zhou, Fangyu Liu, huan zhang, Muhao Chen

Deep neural networks are often overparameterized and may not easily achieve model generalization.

Natural Language Understanding

Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference

1 code implementation12 Feb 2022 Bangzheng Li, Wenpeng Yin, Muhao Chen

The task of ultra-fine entity typing (UFET) seeks to predict diverse and free-form words or phrases that describe the appropriate types of entities mentioned in sentences.

Entity Typing Learning-To-Rank +2

Answer Consolidation: Formulation and Benchmarking

1 code implementation NAACL 2022 Wenxuan Zhou, Qiang Ning, Heba Elfardy, Kevin Small, Muhao Chen

Current question answering (QA) systems primarily consider the single-answer scenario, where each question is assumed to be paired with one correct answer.

Benchmarking Question Answering

Unified Semantic Typing with Meaningful Label Inference

1 code implementation NAACL 2022 James Y. Huang, Bangzheng Li, Jiashu Xu, Muhao Chen

Semantic typing aims at classifying tokens or spans of interest in a textual context into semantic categories such as relations, entity types, and event types.

Entity Typing Relation Classification

GRAPHCACHE: Message Passing as Caching for Sentence-Level Relation Extraction

no code implementations Findings (NAACL) 2022 Yiwei Wang, Muhao Chen, Wenxuan Zhou, Yujun Cai, Yuxuan Liang, Bryan Hooi

GRAPHCACHE aggregates the features from sentences in the whole dataset to learn global representations of properties, and use them to augment the local features within individual sentences.

Relation Relation Extraction +1

Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learning

1 code implementation NAACL 2022 Fei Wang, Zhewei Xu, Pedro Szekely, Muhao Chen

This prunes the full self-attention structure into an order-invariant graph attention that captures the connected graph structure of cells belonging to the same row or column, and it differentiates between relevant cells and irrelevant cells from the structural perspective.

Data Augmentation Data-to-Text Generation +3

Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis

1 code implementation NAACL 2022 Yiwei Wang, Muhao Chen, Wenxuan Zhou, Yujun Cai, Yuxuan Liang, Dayiheng Liu, Baosong Yang, Juncheng Liu, Bryan Hooi

In this paper, we propose the CORE (Counterfactual Analysis based Relation Extraction) debiasing method that guides the RE models to focus on the main effects of textual context without losing the entity information.

counterfactual Relation +2

Summarization as Indirect Supervision for Relation Extraction

1 code implementation19 May 2022 Keming Lu, I-Hung Hsu, Wenxuan Zhou, Mingyu Derek Ma, Muhao Chen

Considering that summarization tasks aim at acquiring concise expressions of synoptical information from the longer context, these tasks naturally align with the objective of RE, i. e., extracting a kind of synoptical information that describes the relation of entity mentions.

Relation Relation Extraction +1

Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing

1 code implementation25 May 2022 Nan Xu, Fei Wang, Bangzheng Li, Mingtao Dong, Muhao Chen

Due to shortcuts from surface patterns to annotated entity labels and biased training, existing entity typing models are subject to the problem of spurious correlations.

counterfactual Data Augmentation +2

Equilibrium and stiffness study of clustered tensegrity structures with the consideration of pulley sizes

no code implementations8 Jun 2022 Shuo Ma, Yiqian Chen, Muhao Chen, Robert E. Skelton

This paper presents the equilibrium and stiffness study of clustered tensegrity structures (CTS) considering pulley sizes.

PInKS: Preconditioned Commonsense Inference with Minimal Supervision

1 code implementation16 Jun 2022 Ehsan Qasemi, Piyush Khanna, Qiang Ning, Muhao Chen

Reasoning with preconditions such as "glass can be used for drinking water unless the glass is shattered" remains an open problem for language models.

Informativeness

Design and control analysis of a deployable clustered hyperbolic paraboloid cable net

no code implementations26 Jun 2022 Shuo Ma, Kai Lu, Muhao Chen, Robert E. Skelton

This paper presents an analytical and experimental design and deployment control analysis of a hyperbolic paraboloid cable net based on clustering actuation strategies.

Experimental Design

Bending the Future: Autoregressive Modeling of Temporal Knowledge Graphs in Curvature-Variable Hyperbolic Spaces

1 code implementation12 Sep 2022 Jihoon Sohn, Mingyu Derek Ma, Muhao Chen

The chronological hierarchies between knowledge graphs at different timestamps are represented by embedding the knowledge graphs as vectors in a common hyperbolic space.

Knowledge Graphs

VIPHY: Probing "Visible" Physical Commonsense Knowledge

1 code implementation15 Sep 2022 Shikhar Singh, Ehsan Qasemi, Muhao Chen

While such tasks measure the requisite knowledge to ground and reason over a given visual instance, they do not, however, measure the ability of VLMs to retain and generalize such knowledge.

Visual Reasoning

Are All Steps Equally Important? Benchmarking Essentiality Detection of Events

no code implementations8 Oct 2022 Haoyu Wang, Hongming Zhang, Yueguan Wang, Yuqian Deng, Muhao Chen, Dan Roth

In this paper, we address this gap by examining the extent to which current models comprehend the essentiality of step events in relation to a goal event.

Benchmarking

Parameter-Efficient Tuning with Special Token Adaptation

1 code implementation10 Oct 2022 Xiaocong Yang, James Y. Huang, Wenxuan Zhou, Muhao Chen

Parameter-efficient tuning aims at updating only a small subset of parameters when adapting a pretrained model to downstream tasks.

Natural Language Understanding NER +2

SpaBERT: A Pretrained Language Model from Geographic Data for Geo-Entity Representation

no code implementations21 Oct 2022 Zekun Li, Jina Kim, Yao-Yi Chiang, Muhao Chen

Characterizing geo-entities is integral to various application domains, such as geo-intelligence and map comprehension, while a key challenge is to capture the spatial-varying context of an entity.

Entity Linking Entity Typing +2

Salience Allocation as Guidance for Abstractive Summarization

1 code implementation22 Oct 2022 Fei Wang, Kaiqiang Song, Hongming Zhang, Lifeng Jin, Sangwoo Cho, Wenlin Yao, Xiaoyang Wang, Muhao Chen, Dong Yu

Recent literature adds extractive summaries as guidance for abstractive summarization models to provide hints of salient content and achieves better performance.

Abstractive Text Summarization

PINTO: Faithful Language Reasoning Using Prompt-Generated Rationales

1 code implementation3 Nov 2022 Peifeng Wang, Aaron Chan, Filip Ilievski, Muhao Chen, Xiang Ren

Neural language models (LMs) have achieved impressive results on various language-based reasoning tasks by utilizing latent knowledge encoded in their own pretrained parameters.

counterfactual Decision Making

Can NLI Provide Proper Indirect Supervision for Low-resource Biomedical Relation Extraction?

1 code implementation21 Dec 2022 Jiashu Xu, Mingyu Derek Ma, Muhao Chen

Two key obstacles in biomedical relation extraction (RE) are the scarcity of annotations and the prevalence of instances without explicitly pre-defined labels due to low annotation coverage.

Multi-class Classification Natural Language Inference +2

Multi-hop Evidence Retrieval for Cross-document Relation Extraction

1 code implementation21 Dec 2022 Keming Lu, I-Hung Hsu, Wenxuan Zhou, Mingyu Derek Ma, Muhao Chen

Relation Extraction (RE) has been extended to cross-document scenarios because many relations are not simply described in a single document.

Relation Relation Extraction +1

Continual Contrastive Finetuning Improves Low-Resource Relation Extraction

no code implementations21 Dec 2022 Wenxuan Zhou, Sheng Zhang, Tristan Naumann, Muhao Chen, Hoifung Poon

In this paper, we aim at bridging the gap and propose to pretrain and finetune the RE model using consistent objectives of contrastive learning.

Contrastive Learning Relation +3

Context-faithful Prompting for Large Language Models

1 code implementation20 Mar 2023 Wenxuan Zhou, Sheng Zhang, Hoifung Poon, Muhao Chen

However, their reliance on parametric knowledge may cause them to overlook contextual cues, leading to incorrect predictions in context-sensitive NLP tasks (e. g., knowledge acquisition tasks).

counterfactual Machine Reading Comprehension +1

Take a Break in the Middle: Investigating Subgoals towards Hierarchical Script Generation

1 code implementation18 May 2023 Xinze Li, Yixin Cao, Muhao Chen, Aixin Sun

Goal-oriented Script Generation is a new task of generating a list of steps that can fulfill the given goal.

How Fragile is Relation Extraction under Entity Replacements?

1 code implementation22 May 2023 Yiwei Wang, Bryan Hooi, Fei Wang, Yujun Cai, Yuxuan Liang, Wenxuan Zhou, Jing Tang, Manjuan Duan, Muhao Chen

In principle, textual context determines the ground-truth relation and the RE models should be able to correctly identify the relations reflected by the textual context.

Benchmarking Causal Inference +2

Preconditioned Visual Language Inference with Weak Supervision

no code implementations22 May 2023 Ehsan Qasemi, Amani R. Maina-Kilaas, Devadutta Dash, Khalid Alsaggaf, Muhao Chen

However, it is unclear if SOTA visual language models (VLMs) can extract such preconditions and infer the affordance of objects with them.

Improving Factuality of Abstractive Summarization without Sacrificing Summary Quality

1 code implementation24 May 2023 Tanay Dixit, Fei Wang, Muhao Chen

However, most of the prior works on training factuality-aware models have ignored the negative effect it has on summary quality.

Abstractive Text Summarization Contrastive Learning

A Causal View of Entity Bias in (Large) Language Models

1 code implementation24 May 2023 Fei Wang, Wenjie Mo, Yiwei Wang, Wenxuan Zhou, Muhao Chen

Building upon this SCM, we propose causal intervention techniques to mitigate entity bias for both white-box and black-box settings.

Machine Reading Comprehension Memorization +1

Adversarial Demonstration Attacks on Large Language Models

no code implementations24 May 2023 Jiongxiao Wang, Zichen Liu, Keun Hee Park, Zhuojun Jiang, Zhaoheng Zheng, Zhuofeng Wu, Muhao Chen, Chaowei Xiao

We propose a novel attack method named advICL, which aims to manipulate only the demonstration without changing the input to mislead the models.

In-Context Learning

Robust Natural Language Understanding with Residual Attention Debiasing

1 code implementation28 May 2023 Fei Wang, James Y. Huang, Tianyi Yan, Wenxuan Zhou, Muhao Chen

However, previous ensemble-based debiasing methods typically apply debiasing on top-level logits without directly addressing biased attention patterns.

Natural Language Understanding

Contrastive Bootstrapping for Label Refinement

1 code implementation7 Jun 2023 Shudi Hou, Yu Xia, Muhao Chen, Sujian Li

Traditional text classification typically categorizes texts into pre-defined coarse-grained classes, from which the produced models cannot handle the real-world scenario where finer categories emerge periodically for accurate services.

Clustering text-classification +1

Software Entity Recognition with Noise-Robust Learning

1 code implementation21 Aug 2023 Tai Nguyen, Yifeng Di, Joohan Lee, Muhao Chen, Tianyi Zhang

Recognizing software entities such as library names from free-form text is essential to enable many software engineering (SE) technologies, such as traceability link recovery, automated documentation, and API recommendation.

Self-Augmentation Improves Zero-Shot Cross-Lingual Transfer

no code implementations19 Sep 2023 Fei Wang, Kuan-Hao Huang, Kai-Wei Chang, Muhao Chen

In this paper, we propose a simple yet effective method, SALT, to improve the zero-shot cross-lingual transfer of the multilingual pretrained language models without the help of such external data.

Multilingual NLP Zero-Shot Cross-Lingual Transfer

AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models

2 code implementations3 Oct 2023 Xiaogeng Liu, Nan Xu, Muhao Chen, Chaowei Xiao

In light of these challenges, we intend to answer this question: Can we develop an approach that can automatically generate stealthy jailbreak prompts?

Decision Making

DOMINO: A Dual-System for Multi-step Visual Language Reasoning

1 code implementation4 Oct 2023 Peifang Wang, Olga Golovneva, Armen Aghajanyan, Xiang Ren, Muhao Chen, Asli Celikyilmaz, Maryam Fazel-Zarandi

By fine-tuning the System-2 module (LLaMA-2 70B) on only a small amount of data on multi-step reasoning, the accuracy of our method is further improved and surpasses the best fully-supervised end-to-end approach by 5. 7% and a pipeline approach with FlanPaLM (540B) by 7. 5% on a challenging dataset with human-authored questions.

Arithmetic Reasoning Language Modelling +2

Primacy Effect of ChatGPT

1 code implementation20 Oct 2023 Yiwei Wang, Yujun Cai, Muhao Chen, Yuxuan Liang, Bryan Hooi

We have two main findings: i) ChatGPT's decision is sensitive to the order of labels in the prompt; ii) ChatGPT has a clearly higher chance to select the labels at earlier positions as the answer.

Natural Language Understanding Question Answering

Affective and Dynamic Beam Search for Story Generation

1 code implementation23 Oct 2023 Tenghao Huang, Ehsan Qasemi, Bangzheng Li, He Wang, Faeze Brahman, Muhao Chen, Snigdha Chaturvedi

Storytelling's captivating potential makes it a fascinating research area, with implications for entertainment, education, therapy, and cognitive studies.

Sentence Story Generation

GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding

1 code implementation23 Oct 2023 Zekun Li, Wenxuan Zhou, Yao-Yi Chiang, Muhao Chen

This paper introduces GeoLM, a geospatially grounded language model that enhances the understanding of geo-entities in natural language.

Contrastive Learning Entity Typing +4

How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities

1 code implementation15 Nov 2023 Lingbo Mo, Boshi Wang, Muhao Chen, Huan Sun

The rapid progress in open-source Large Language Models (LLMs) is significantly driving AI development forward.

Ethics Fairness +2

Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?

no code implementations16 Nov 2023 Bangzheng Li, Ben Zhou, Fei Wang, Xingyu Fu, Dan Roth, Muhao Chen

During the construction of the evidence, we purposefully replace semantic clues (entities) that may lead to the correct answer with distractor clues (evidence) that will not directly lead to the correct answer but require a chain-like reasoning process.

Hallucination Sentence

Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking

no code implementations16 Nov 2023 Nan Xu, Fei Wang, Ben Zhou, Bang Zheng Li, Chaowei Xiao, Muhao Chen

While large language models (LLMs) have demonstrated increasing power, they have also given rise to a wide range of harmful behaviors.

Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations

no code implementations16 Nov 2023 Wenjie Mo, Jiashu Xu, Qin Liu, Jiongxiao Wang, Jun Yan, Chaowei Xiao, Muhao Chen

Existing studies in backdoor defense have predominantly focused on the training phase, overlooking the critical aspect of testing time defense.

backdoor defense

On the Exploitability of Reinforcement Learning with Human Feedback for Large Language Models

no code implementations16 Nov 2023 Jiongxiao Wang, Junlin Wu, Muhao Chen, Yevgeniy Vorobeychik, Chaowei Xiao

Reinforcement Learning with Human Feedback (RLHF) is a methodology designed to align Large Language Models (LLMs) with human preferences, playing an important role in LLMs alignment.

Backdoor Attack Data Poisoning

Rethinking Tabular Data Understanding with Large Language Models

1 code implementation27 Dec 2023 Tianyang Liu, Fei Wang, Muhao Chen

Large Language Models (LLMs) have shown to be capable of various tasks, yet their capability in interpreting and reasoning over tabular data remains an underexplored area.

Semantic Parsing

DeepEdit: Knowledge Editing as Decoding with Constraints

1 code implementation19 Jan 2024 Yiwei Wang, Muhao Chen, Nanyun Peng, Kai-Wei Chang

We propose DeepEdit (Depth-first Search based Progressive Decoding for Knowledge Editing), a neuro-symbolic method that improves knowledge editing with better coherence of reasoning, relevance to the question, and awareness of updated knowledge.

Informativeness knowledge editing +2

Instructional Fingerprinting of Large Language Models

1 code implementation21 Jan 2024 Jiashu Xu, Fei Wang, Mingyu Derek Ma, Pang Wei Koh, Chaowei Xiao, Muhao Chen

The exorbitant cost of training Large language models (LLMs) from scratch makes it essential to fingerprint the models to protect intellectual property via ownership authentication and to ensure downstream users and developers comply with their license terms (e. g. restricting commercial use).

Privacy-Preserving Language Model Inference with Instance Obfuscation

no code implementations13 Feb 2024 Yixiang Yao, Fei Wang, Srivatsan Ravi, Muhao Chen

Language Models as a Service (LMaaS) offers convenient access for developers and researchers to perform inference using pre-trained language models.

Benchmarking Language Modelling +2

Contrastive Instruction Tuning

no code implementations17 Feb 2024 Tianyi Yan, Fei Wang, James Y. Huang, Wenxuan Zhou, Fan Yin, Aram Galstyan, Wenpeng Yin, Muhao Chen

Instruction tuning has been used as a promising approach to improve the performance of large language models (LLMs) on unseen tasks.

Sentence

Mitigating Fine-tuning Jailbreak Attack with Backdoor Enhanced Alignment

no code implementations22 Feb 2024 Jiongxiao Wang, Jiazhao Li, Yiquan Li, Xiangyu Qi, Junjie Hu, Yixuan Li, Patrick McDaniel, Muhao Chen, Bo Li, Chaowei Xiao

Despite the general capabilities of Large Language Models (LLMs) like GPT-4 and Llama-2, these models still request fine-tuning or adaptation with customized data when it comes to meeting the specific business demands and intricacies of tailored use cases.

X-Shot: A Unified System to Handle Frequent, Few-shot and Zero-shot Learning Simultaneously in Classification

1 code implementation6 Mar 2024 Hanzi Xu, Muhao Chen, Lifu Huang, Slobodan Vucetic, Wenpeng Yin

In recent years, few-shot and zero-shot learning, which learn to predict labels with limited annotated instances, have garnered significant attention.

Domain Generalization Instruction Following +1

AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting

1 code implementation14 Mar 2024 Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, Chaowei Xiao

However, with the integration of additional modalities, MLLMs are exposed to new vulnerabilities, rendering them prone to structured-based jailbreak attacks, where semantic content (e. g., "harmful text") has been injected into the images to mislead MLLMs.

Monotonic Paraphrasing Improves Generalization of Language Model Prompting

no code implementations24 Mar 2024 Qin Liu, Fei Wang, Nan Xu, Tianyi Yan, Tao Meng, Muhao Chen

In this paper, we propose monotonic paraphrasing (MonoPara), an end-to-end decoding strategy that paraphrases given prompts or instructions into their lower perplexity counterparts based on an ensemble of a paraphrase LM for prompt (or instruction) rewriting, and a target LM (i. e. the prompt or instruction executor) that constrains the generation for lower perplexity.

Language Modelling

New Frontiers of Information Extraction

no code implementations NAACL (ACL) 2022 Muhao Chen, Lifu Huang, Manling Li, Ben Zhou, Heng Ji, Dan Roth

This tutorial targets researchers and practitioners who are interested in AI and ML technologies for structural information extraction (IE) from unstructured textual sources.

Cannot find the paper you are looking for? You can Submit a new open access paper.