Search Results for author: Jiawei Han

Found 219 papers, 126 papers with code

Phrase-aware Unsupervised Constituency Parsing

no code implementations ACL 2022 Xiaotao Gu, Yikang Shen, Jiaming Shen, Jingbo Shang, Jiawei Han

Recent studies have achieved inspiring success in unsupervised grammar induction using masked language modeling (MLM) as the proxy task.

Constituency Parsing Language Modelling +1

ChemNER: Fine-Grained Chemistry Named Entity Recognition with Ontology-Guided Distant Supervision

no code implementations EMNLP 2021 Xuan Wang, Vivian Hu, Xiangchen Song, Shweta Garg, Jinfeng Xiao, Jiawei Han

For example, chemistry research needs to study dozens to hundreds of distinct, fine-grained entity types, making consistent and accurate annotation difficult even for crowds of domain experts.

named-entity-recognition Named Entity Recognition +1

Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction

no code implementations17 Feb 2024 Sizhe Zhou, Yu Meng, Bowen Jin, Jiawei Han

(2) We fine-tune a bidirectional Small Language Model (SLM) using these initial seeds to learn the relations for the target domain.

Few-Shot Learning Language Modelling +3

GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Models

no code implementations16 Feb 2024 Pengcheng Jiang, Jiacheng Lin, Zifeng Wang, Jimeng Sun, Jiawei Han

The field of relation extraction (RE) is experiencing a notable shift towards generative relation extraction (GRE), leveraging the capabilities of large language models (LLMs).

Relation Relation Extraction +1

Similarity-based Neighbor Selection for Graph LLMs

1 code implementation6 Feb 2024 Rui Li, Jiwei Li, Jiawei Han, Guoyin Wang

Our research further underscores the significance of graph structure integration in LLM applications and identifies key factors for their success in node classification.

Node Classification

Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains

1 code implementation23 Jan 2024 Yu Zhang, Yunyi Zhang, Yanzhen Shen, Yu Deng, Lucian Popa, Larisa Shwartz, ChengXiang Zhai, Jiawei Han

In this paper, we study the task of seed-guided fine-grained entity typing in science and engineering domains, which takes the name and a few seed entities for each entity type as the only supervision and aims to classify new entity mentions into both seen and unseen types (i. e., those without seed entities).

Entity Typing Natural Language Inference

Investigating Data Contamination for Pre-training Language Models

no code implementations11 Jan 2024 Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, Sanmi Koyejo

Language models pre-trained on web-scale corpora demonstrate impressive capabilities on diverse downstream tasks.

Language Modelling

Large Language Models on Graphs: A Comprehensive Survey

1 code implementation5 Dec 2023 Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, Jiawei Han

Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i. e., graph-based reasoning).

Language Modelling

RESIN-EDITOR: A Schema-guided Hierarchical Event Graph Visualizer and Editor

1 code implementation5 Dec 2023 Khanh Duy Nguyen, Zixuan Zhang, Reece Suchocki, Sha Li, Martha Palmer, Susan Brown, Jiawei Han, Heng Ji

In this paper, we present RESIN-EDITOR, an interactive event graph visualizer and editor designed for analyzing complex events.

SCStory: Self-supervised and Continual Online Story Discovery

1 code implementation27 Nov 2023 Susik Yoon, Yu Meng, Dongha Lee, Jiawei Han

With a lightweight hierarchical embedding module that first learns sentence representations and then article representations, SCStory identifies story-relevant information of news articles and uses them to discover stories.

Continual Learning Contrastive Learning +1

Structured Chemistry Reasoning with Large Language Models

no code implementations16 Nov 2023 Siru Ouyang, Zhuosheng Zhang, Bing Yan, Xuan Liu, Yejin Choi, Jiawei Han, Lianhui Qin

Large Language Models (LLMs) excel in diverse areas, yet struggle with complex scientific reasoning, especially in the field of chemistry.

General Knowledge

MART: Improving LLM Safety with Multi-round Automatic Red-Teaming

no code implementations13 Nov 2023 Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, Yuning Mao

Specifically, an adversarial LLM and a target LLM interplay with each other in an iterative manner, where the adversarial LLM aims to generate challenging prompts that elicit unsafe responses from the target LLM, while the target LLM is fine-tuned with safety aligned data on these adversarial prompts.

Instruction Following Response Generation

Don't Make Your LLM an Evaluation Benchmark Cheater

no code implementations3 Nov 2023 Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin, Ji-Rong Wen, Jiawei Han

Large language models~(LLMs) have greatly advanced the frontiers of artificial intelligence, attaining remarkable improvement in model capacity.

Instruct and Extract: Instruction Tuning for On-Demand Information Extraction

1 code implementation24 Oct 2023 Yizhu Jiao, Ming Zhong, Sha Li, Ruining Zhao, Siru Ouyang, Heng Ji, Jiawei Han

However, when it comes to information extraction - a classic task in natural language processing - most task-specific systems cannot align well with long-tail ad hoc extraction use cases for non-expert users.

Instruction Following

"Why Should I Review This Paper?" Unifying Semantic, Topic, and Citation Factors for Paper-Reviewer Matching

no code implementations23 Oct 2023 Yu Zhang, Yanzhen Shen, Xiusi Chen, Bowen Jin, Jiawei Han

As many academic conferences are overwhelmed by a rapidly increasing number of paper submissions, automatically finding appropriate reviewers for each submission becomes a more urgent need than ever.

Information Retrieval Language Modelling +1

The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions

1 code implementation19 Oct 2023 Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Iter, Reid Pryzant, Chenguang Zhu, Heng Ji, Jiawei Han

Recent progress in Large Language Models (LLMs) has produced models that exhibit remarkable performance across a variety of NLP tasks.

Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective

no code implementations17 Oct 2023 Ming Zhong, Chenxin An, Weizhu Chen, Jiawei Han, Pengcheng He

In this paper, we seek to empirically investigate knowledge transfer from larger to smaller models through a parametric perspective.

Transfer Learning

Ontology Enrichment for Effective Fine-grained Entity Typing

no code implementations11 Oct 2023 Siru Ouyang, Jiaxin Huang, Pranav Pillai, Yunyi Zhang, Yu Zhang, Jiawei Han

In this study, we propose OnEFET, where we (1) enrich each node in the ontology structure with two types of extra information: instance information for training sample augmentation and topic information to relate types to contexts, and (2) develop a coarse-to-fine typing algorithm that exploits the enriched information by training an entailment model with contrasting topics and instance-based augmented training samples.

Entity Typing

Language Models As Semantic Indexers

no code implementations11 Oct 2023 Bowen Jin, Hansi Zeng, Guoyin Wang, Xiusi Chen, Tianxin Wei, Ruirui Li, Zhengyang Wang, Zheng Li, Yang Li, Hanqing Lu, Suhang Wang, Jiawei Han, Xianfeng Tang

Semantic identifier (ID) is an important concept in information retrieval that aims to preserve the semantics of objects such as documents and items inside their IDs.

Contrastive Learning Information Retrieval +2

Learning Multiplex Embeddings on Text-rich Networks with One Text Encoder

no code implementations10 Oct 2023 Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Han Zhao, Jiawei Han

Mainstream text representation learning methods use pretrained language models (PLMs) to generate one embedding for each text unit, expecting that all types of relations between texts can be captured by these single-view embeddings.

Representation Learning

Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs

no code implementations3 Oct 2023 Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, Jianfeng Gao

In this study, we introduce adaptive KV cache compression, a plug-and-play method that reduces the memory footprint of generative inference for Large Language Models (LLMs).

ReactIE: Enhancing Chemical Reaction Extraction with Weak Supervision

no code implementations4 Jul 2023 Ming Zhong, Siru Ouyang, Minhao Jiang, Vivian Hu, Yizhu Jiao, Xuan Wang, Jiawei Han

Structured chemical reaction information plays a vital role for chemists engaged in laboratory work and advanced endeavors such as computer-aided drug design.

Weakly Supervised Multi-Label Classification of Full-Text Scientific Papers

1 code implementation24 Jun 2023 Yu Zhang, Bowen Jin, Xiusi Chen, Yanzhen Shen, Yunyi Zhang, Yu Meng, Jiawei Han

Instead of relying on human-annotated training samples to build a classifier, weakly supervised scientific paper classification aims to classify papers only using category descriptions (e. g., category names, category-indicative keywords).

Multi-Label Classification

Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond

1 code implementation16 Jun 2023 Fangzhi Xu, Qika Lin, Jiawei Han, Tianzhe Zhao, Jun Liu, Erik Cambria

Firstly, to offer systematic evaluations, we select fifteen typical logical reasoning datasets and organize them into deductive, inductive, abductive and mixed-form reasoning settings.

Benchmarking Evidence Selection +2

Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation

1 code implementation23 May 2023 Da Yin, Xiao Liu, Fan Yin, Ming Zhong, Hritik Bansal, Jiawei Han, Kai-Wei Chang

Instruction tuning has emerged to enhance the capabilities of large language models (LLMs) to comprehend instructions and generate appropriate responses.

Continual Learning

PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training

1 code implementation23 May 2023 Yunyi Zhang, Minhao Jiang, Yu Meng, Yu Zhang, Jiawei Han

Weakly-supervised text classification trains a classifier using the label name of each target class as the only supervision, which largely reduces human annotation efforts.

Pseudo Label Sentiment Analysis +3

OntoType: Ontology-Guided Zero-Shot Fine-Grained Entity Typing with Weak Supervision from Pre-Trained Language Models

no code implementations21 May 2023 Tanay Komarlu, Minhao Jiang, Xuan Wang, Jiawei Han

In this study, we vision that an ontology provides a semantics-rich, hierarchical structure, which will help select the best results generated by multiple PLM models and head words.

Entity Typing Natural Language Inference +1

Patton: Language Model Pretraining on Text-Rich Networks

no code implementations20 May 2023 Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, Jiawei Han

A real-world text corpus sometimes comprises not only text documents but also semantic links between them (e. g., academic papers in a bibliographic network are linked by citations and co-authorships).

Language Modelling Masked Language Modeling +1

Unsupervised Story Discovery from Continuous News Streams via Scalable Thematic Embedding

1 code implementation8 Apr 2023 Susik Yoon, Dongha Lee, Yunyi Zhang, Jiawei Han

Unsupervised discovery of stories with correlated news articles in real-time helps people digest massive news streams without expensive human annotations.

Sentence

MEGClass: Extremely Weakly Supervised Text Classification via Mutually-Enhancing Text Granularities

1 code implementation4 Apr 2023 Priyanka Kargupta, Tanay Komarlu, Susik Yoon, Xuan Wang, Jiawei Han

By preserving the heterogeneity of potential classes, MEGClass can select the most informative class-indicative documents as iterative feedback to enhance the initial word-based class representations and ultimately fine-tune a pre-trained text classifier.

text-classification Text Classification

GLEN: General-Purpose Event Detection for Thousands of Types

1 code implementation16 Mar 2023 Qiusi Zhan, Sha Li, Kathryn Conger, Martha Palmer, Heng Ji, Jiawei Han

Finally, we perform error analysis and show that label noise is still the largest challenge for improving performance for this new dataset.

Event Detection Event Extraction

Edgeformers: Graph-Empowered Transformers for Representation Learning on Textual-Edge Networks

1 code implementation21 Feb 2023 Bowen Jin, Yu Zhang, Yu Meng, Jiawei Han

Edges in many real-world social/information networks are associated with rich text information (e. g., user-user communications or user-product reviews).

Edge Classification Link Prediction +1

PDSum: Prototype-driven Continuous Summarization of Evolving Multi-document Sets Stream

1 code implementation10 Feb 2023 Susik Yoon, Hou Pong Chan, Jiawei Han

Summarizing text-rich documents has been long studied in the literature, but most of the existing efforts have been made to summarize a static and predefined multi-document set.

Document Summarization Multi-Document Summarization

The Effect of Metadata on Scientific Literature Tagging: A Cross-Field Cross-Model Study

1 code implementation7 Feb 2023 Yu Zhang, Bowen Jin, Qi Zhu, Yu Meng, Jiawei Han

Due to the exponential growth of scientific publications on the Web, there is a pressing need to tag each paper with fine-grained topics so that researchers can track their interested fields of study rather than drowning in the whole literature.

Language Modelling Multi Label Text Classification +3

Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories

no code implementations7 Feb 2023 Suyu Ge, Chenyan Xiong, Corby Rosset, Arnold Overwijk, Jiawei Han, Paul Bennett

In this paper we improve the zero-shot generalization ability of language models via Mixture-Of-Memory Augmentation (MoMA), a mechanism that retrieves augmentation documents from multiple information corpora ("external memories"), with the option to "plug in" new memory at inference time.

Retrieval Zero-shot Generalization

Representation Deficiency in Masked Language Modeling

no code implementations4 Feb 2023 Yu Meng, Jitin Krishnan, Sinong Wang, Qifan Wang, Yuning Mao, Han Fang, Marjan Ghazvininejad, Jiawei Han, Luke Zettlemoyer

In this work, we offer a new perspective on the consequence of such a discrepancy: We demonstrate empirically and theoretically that MLM pretraining allocates some model dimensions exclusively for representing $\texttt{[MASK]}$ tokens, resulting in a representation deficiency for real tokens and limiting the pretrained model's expressiveness when it is adapted to downstream data without $\texttt{[MASK]}$ tokens.

Language Modelling Masked Language Modeling

Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts

1 code implementation12 Dec 2022 Yu Zhang, Yunyi Zhang, Martin Michalski, Yucheng Jiang, Yu Meng, Jiawei Han

Instead of mining coherent topics from a given text corpus in a completely unsupervised manner, seed-guided topic discovery methods leverage user-provided seed words to extract distinctive and coherent topics so that the mined topics can better cater to the user's interest.

Language Modelling Word Embeddings

Entity Set Co-Expansion in StackOverflow

no code implementations5 Dec 2022 Yu Zhang, Yunyi Zhang, Yucheng Jiang, Martin Michalski, Yu Deng, Lucian Popa, ChengXiang Zhai, Jiawei Han

Given a few seed entities of a certain type (e. g., Software or Programming Language), entity set expansion aims to discover an extensive set of entities that share the same type as the seeds.

graph construction Management

Open Relation and Event Type Discovery with Type Abstraction

1 code implementation30 Nov 2022 Sha Li, Heng Ji, Jiawei Han

To tackle this problem, we introduce the idea of type abstraction, where the model is prompted to generalize and name the type.

Event Extraction Relation +2

Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning

1 code implementation6 Nov 2022 Yu Meng, Martin Michalski, Jiaxin Huang, Yu Zhang, Tarek Abdelzaher, Jiawei Han

In this work, we study few-shot learning with PLMs from a different perspective: We first tune an autoregressive PLM on the few-shot samples and then use it as a generator to synthesize a large amount of novel training samples which augment the original training set.

Few-Shot Learning

Open-Vocabulary Argument Role Prediction for Event Extraction

1 code implementation3 Nov 2022 Yizhu Jiao, Sha Li, Yiqing Xie, Ming Zhong, Heng Ji, Jiawei Han

Specifically, we formulate the role prediction problem as an in-filling task and construct prompts for a pre-trained language model to generate candidate roles.

Event Extraction Language Modelling

Large Language Models Can Self-Improve

no code implementations20 Oct 2022 Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, Jiawei Han

We show that our approach improves the general reasoning ability of a 540B-parameter LLM (74. 4%->82. 1% on GSM8K, 78. 2%->83. 0% on DROP, 90. 0%->94. 4% on OpenBookQA, and 63. 4%->67. 9% on ANLI-A3) and achieves state-of-the-art-level performance, without any ground truth label.

Arithmetic Reasoning Common Sense Reasoning +3

Topic Taxonomy Expansion via Hierarchy-Aware Topic Phrase Generation

no code implementations18 Oct 2022 Dongha Lee, Jiaming Shen, Seonghyeon Lee, Susik Yoon, Hwanjo Yu, Jiawei Han

Topic taxonomies display hierarchical topic structures of a text corpus and provide topical knowledge to enhance various NLP applications.

Relation Taxonomy Expansion

Towards a Unified Multi-Dimensional Evaluator for Text Generation

2 code implementations13 Oct 2022 Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, PengFei Liu, Chenguang Zhu, Heng Ji, Jiawei Han

We re-frame NLG evaluation as a Boolean Question Answering (QA) task, and by guiding the model with different questions, we can use one evaluator to evaluate from multiple dimensions.

nlg evaluation Question Answering +4

Few-shot Text Classification with Dual Contrastive Consistency

no code implementations29 Sep 2022 Liwen Sun, Jiawei Han

In this paper, we explore how to utilize pre-trained language model to perform few-shot text classification where only a few annotated examples are given for each class.

Contrastive Learning Few-Shot Text Classification +4

TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter

1 code implementation15 Sep 2022 Xinyang Zhang, Yury Malkov, Omar Florez, Serim Park, Brian McWilliams, Jiawei Han, Ahmed El-Kishky

Most existing PLMs are not tailored to the noisy user-generated text on social media, and the pre-training does not factor in the valuable social engagement logs available in a social network.

Language Modelling

MentorGNN: Deriving Curriculum for Pre-Training GNNs

1 code implementation21 Aug 2022 Dawei Zhou, Lecheng Zheng, Dongqi Fu, Jiawei Han, Jingrui He

To comprehend heterogeneous graph signals at different granularities, we propose a curriculum learning paradigm that automatically re-weighs graph signals in order to ensure a good generalization in the target domain.

Domain Adaptation Graph Mining

Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation and Instance Generation

1 code implementation28 Jun 2022 Jiaxin Huang, Yu Meng, Jiawei Han

We study the problem of few-shot Fine-grained Entity Typing (FET), where only a few annotated entity mentions with contexts are given for each entity type.

Entity Typing Language Modelling +1

TeKo: Text-Rich Graph Neural Networks with External Knowledge

no code implementations15 Jun 2022 Zhizhi Yu, Di Jin, Jianguo Wei, Ziyang Liu, Yue Shang, Yun Xiao, Jiawei Han, Lingfei Wu

Graph Neural Networks (GNNs) have gained great popularity in tackling various analytical tasks on graph-structured data (i. e., networks).

Unsupervised Key Event Detection from Massive Text Corpora

1 code implementation8 Jun 2022 Yunyi Zhang, Fang Guo, Jiaming Shen, Jiawei Han

Automated event detection from news corpora is a crucial task towards mining fast-evolving structured knowledge.

Event Detection

Schema-Guided Event Graph Completion

no code implementations6 Jun 2022 Hongwei Wang, Zixuan Zhang, Sha Li, Jiawei Han, Yizhou Sun, Hanghang Tong, Joseph P. Olive, Heng Ji

Existing link prediction or graph completion methods have difficulty dealing with event graphs because they are usually designed for a single large graph such as a social network or a knowledge graph, rather than multiple small dynamic event graphs.

Link Prediction

All Birds with One Stone: Multi-task Text Classification for Efficient Inference with One Forward Pass

no code implementations22 May 2022 Jiaxin Huang, Tianqi Liu, Jialu Liu, Adam D. Lelkes, Cong Yu, Jiawei Han

Multi-Task Learning (MTL) models have shown their robustness, effectiveness, and efficiency for transferring learned knowledge across tasks.

Multi-Task Learning text-classification +1

Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networks

1 code implementation20 May 2022 Bowen Jin, Yu Zhang, Qi Zhu, Jiawei Han

In heterogeneous text-rich networks, this task is more challenging due to (1) presence or absence of text: Some nodes are associated with rich textual information, while others are not; (2) diversity of types: Nodes and edges of multiple types form a heterogeneous network structure.

Clustering Graph Attention +5

OA-Mine: Open-World Attribute Mining for E-Commerce Products with Weak Supervision

1 code implementation29 Apr 2022 Xinyang Zhang, Chenwei Zhang, Xian Li, Xin Luna Dong, Jingbo Shang, Christos Faloutsos, Jiawei Han

Most prior works on this matter mine new values for a set of known attributes but cannot handle new attributes that arose from constantly changing data.

Attribute Language Modelling

Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators

1 code implementation ICLR 2022 Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett, Jiawei Han, Xia Song

We present a new framework AMOS that pretrains text encoders with an Adversarial learning curriculum via a Mixture Of Signals from multiple auxiliary generators.

Shift-Robust Node Classification via Graph Adversarial Clustering

no code implementations7 Mar 2022 Qi Zhu, Chao Zhang, Chanyoung Park, Carl Yang, Jiawei Han

Then a shift-robust classifier is optimized on training graph and adversarial samples on target graph, which are generated by cluster GNN.

Classification Clustering +2

TaxoEnrich: Self-Supervised Taxonomy Completion via Structure-Semantic Representations

no code implementations10 Feb 2022 Minhao Jiang, Xiangchen Song, Jieyu Zhang, Jiawei Han

Taxonomies are fundamental to many real-world applications in various domains, serving as structural representations of knowledge.

Position

Generating Training Data with Language Models: Towards Zero-Shot Language Understanding

1 code implementation9 Feb 2022 Yu Meng, Jiaxin Huang, Yu Zhang, Jiawei Han

Pretrained language models (PLMs) have demonstrated remarkable performance in various natural language processing tasks: Unidirectional PLMs (e. g., GPT) are well known for their superior text generation capabilities; bidirectional PLMs (e. g., BERT) have been the prominent choice for natural language understanding (NLU) tasks.

Few-Shot Learning MNLI-m +5

Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations

1 code implementation9 Feb 2022 Yu Meng, Yunyi Zhang, Jiaxin Huang, Yu Zhang, Jiawei Han

Interestingly, there have not been standard approaches to deploy PLMs for topic discovery as better alternatives to topic models.

Clustering Language Modelling +1

TaxoCom: Topic Taxonomy Completion with Hierarchical Discovery of Novel Topic Clusters

no code implementations18 Jan 2022 Dongha Lee, Jiaming Shen, SeongKu Kang, Susik Yoon, Jiawei Han, Hwanjo Yu

Topic taxonomies, which represent the latent topic (or category) structure of document collections, provide valuable knowledge of contents in many applications such as web search and information filtering.

Clustering Topic coverage

Universal Graph Convolutional Networks

1 code implementation NeurIPS 2021 Di Jin, Zhizhi Yu, Cuiying Huo, Rui Wang, Xiao Wang, Dongxiao He, Jiawei Han

So can we reasonably utilize these segmentation rules to design a universal propagation mechanism independent of the network structural assumption?

Out-of-Category Document Identification Using Target-Category Names as Weak Supervision

no code implementations24 Nov 2021 Dongha Lee, Dongmin Hyun, Jiawei Han, Hwanjo Yu

To address this challenge, we introduce a new task referred to as out-of-category detection, which aims to distinguish the documents according to their semantic relevance to the inlier (or target) categories by using the category names as weak supervision.

MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information

1 code implementation7 Nov 2021 Yu Zhang, Shweta Garg, Yu Meng, Xiusi Chen, Jiawei Han

We study the problem of weakly supervised text classification, which aims to classify text documents into a set of pre-defined categories with category surface names only and without any annotated training document provided.

text-classification Text Classification

Fine-Grained Opinion Summarization with Minimal Supervision

no code implementations17 Oct 2021 Suyu Ge, Jiaxin Huang, Yu Meng, Sharon Wang, Jiawei Han

Opinion summarization aims to profile a target by extracting opinions from multiple documents.

Fine-Grained Opinion Analysis

UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning

1 code implementation ACL 2022 Yuning Mao, Lambert Mathias, Rui Hou, Amjad Almahairi, Hao Ma, Jiawei Han, Wen-tau Yih, Madian Khabsa

Recent parameter-efficient language model tuning (PELT) methods manage to match the performance of fine-tuning with much fewer trainable parameters and perform especially well when training data is limited.

Language Modelling Model Selection

Entity Linking Meets Deep Learning: Techniques and Solutions

no code implementations26 Sep 2021 Wei Shen, Yuhan Li, Yinan Liu, Jiawei Han, Jianyong Wang, Xiaojie Yuan

Entity linking (EL) is the process of linking entity mentions appearing in web text with their corresponding entities in a knowledge base.

Entity Linking Knowledge Base Population +2

Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training

1 code implementation EMNLP 2021 Yu Meng, Yunyi Zhang, Jiaxin Huang, Xuan Wang, Yu Zhang, Heng Ji, Jiawei Han

We study the problem of training named entity recognition (NER) models using only distantly-labeled data, which can be automatically obtained by matching entity mentions in the raw text with entity types in a knowledge base.

Language Modelling named-entity-recognition +2

Corpus-based Open-Domain Event Type Induction

1 code implementation EMNLP 2021 Jiaming Shen, Yunyi Zhang, Heng Ji, Jiawei Han

As events of the same type could be expressed in multiple ways, we propose to represent each event type as a cluster of <predicate sense, object head> pairs.

Event Extraction Object +1

Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data

1 code implementation NeurIPS 2021 Qi Zhu, Natalia Ponomareva, Jiawei Han, Bryan Perozzi

In this work we present a method, Shift-Robust GNN (SR-GNN), designed to account for distributional differences between biased training data and the graph's true inference distribution.

Multi-head or Single-head? An Empirical Comparison for Transformer Training

1 code implementation17 Jun 2021 Liyuan Liu, Jialu Liu, Jiawei Han

Multi-head attention plays a crucial role in the recent success of Transformer models, which leads to consistent performance improvements over conventional attention in various applications.

Eider: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusion

1 code implementation Findings (ACL) 2022 Yiqing Xie, Jiaming Shen, Sha Li, Yuning Mao, Jiawei Han

Typical DocRE methods blindly take the full document as input, while a subset of the sentences in the document, noted as the evidence, are often sufficient for humans to predict the relation of an entity pair.

Document-level Relation Extraction Relation

Event Time Extraction and Propagation via Graph Attention Networks

1 code implementation NAACL 2021 Haoyang Wen, Yanru Qu, Heng Ji, Qiang Ning, Jiawei Han, Avi Sil, Hanghang Tong, Dan Roth

Grounding events into a precise timeline is important for natural language understanding but has received limited attention in recent work.

Graph Attention Natural Language Understanding +3

UCPhrase: Unsupervised Context-aware Quality Phrase Tagging

2 code implementations28 May 2021 Xiaotao Gu, Zihan Wang, Zhenyu Bi, Yu Meng, Liyuan Liu, Jiawei Han, Jingbo Shang

Training a conventional neural tagger based on silver labels usually faces the risk of overfitting phrase surface names.

Keyphrase Extraction Language Modelling +3

Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation

2 code implementations EMNLP 2021 Yuning Mao, Wenchang Ma, Deren Lei, Jiawei Han, Xiang Ren

In this paper, we present a systematic analysis that studies whether current seq2seq models, especially pre-trained language models, are good enough for preserving important input concepts and to what extent explicitly guiding generation with the concepts as lexical constraints is beneficial.

Conditional Text Generation Denoising

Document-Level Event Argument Extraction by Conditional Generation

1 code implementation NAACL 2021 Sha Li, Heng Ji, Jiawei Han

On the task of argument extraction, we achieve an absolute gain of 7. 6% F1 and 5. 7% F1 over the next best model on the RAMS and WikiEvents datasets respectively.

Document-level Event Extraction Event Argument Extraction +2

The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction

1 code implementation EMNLP 2021 Manling Li, Sha Li, Zhenhailong Wang, Lifu Huang, Kyunghyun Cho, Heng Ji, Jiawei Han, Clare Voss

We introduce a new concept of Temporal Complex Event Schema: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations.

Who Should Go First? A Self-Supervised Concept Sorting Model for Improving Taxonomy Expansion

no code implementations8 Apr 2021 Xiangchen Song, Jiaming Shen, Jieyu Zhang, Jiawei Han

Taxonomies have been widely used in various machine learning and text mining systems to organize knowledge and facilitate downstream tasks.

Taxonomy Expansion

Toward Tweet Entity Linking with Heterogeneous Information Networks

1 code implementation IEEE Transactions on Knowledge and Data Engineering 2021 Wei Shen, Yuwei Yin, Yang Yang, Jiawei Han, Jianyong Wang, Xiaojie Yuan

The task of linking an entity mention in a tweet with its corresponding entity in a heterogeneous information network is of great importance, for the purpose of enriching heterogeneous information networks with the abundant and fresh knowledge embedded in tweets.

Entity Linking Metric Learning

Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks

no code implementations23 Feb 2021 Xinyang Zhang, Chenwei Zhang, Luna Xin Dong, Jingbo Shang, Jiawei Han

Specifically, we jointly train two modules with different inductive biases -- a text analysis module for text understanding and a network learning module for class-discriminative, scalable network learning.

Product Categorization Text Categorization

COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

2 code implementations NeurIPS 2021 Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett, Jiawei Han, Xia Song

The first token-level task, Corrective Language Modeling, is to detect and correct tokens replaced by the auxiliary model, in order to better capture token-level semantics.

Contrastive Learning Language Modelling +1

MATCH: Metadata-Aware Text Classification in A Large Hierarchy

1 code implementation15 Feb 2021 Yu Zhang, Zhihong Shen, Yuxiao Dong, Kuansan Wang, Jiawei Han

Multi-label text classification refers to the problem of assigning each given document its most relevant labels from the label set.

General Classification Multi Label Text Classification +2

Rider: Reader-Guided Passage Reranking for Open-Domain Question Answering

1 code implementation1 Jan 2021 Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Weizhu Chen

Current open-domain question answering systems often follow a Retriever-Reader architecture, where the retriever first retrieves relevant passages and the reader then reads the retrieved passages to form an answer.

Natural Questions Open-Domain Question Answering +2

Few-Shot Named Entity Recognition: A Comprehensive Study

2 code implementations29 Dec 2020 Jiaxin Huang, Chunyuan Li, Krishan Subudhi, Damien Jose, Shobana Balakrishnan, Weizhu Chen, Baolin Peng, Jianfeng Gao, Jiawei Han

This paper presents a comprehensive study to efficiently build named entity recognition (NER) systems when a small number of in-domain labeled data is available.

Few-Shot Learning named-entity-recognition +2

Hierarchical Metadata-Aware Document Categorization under Weak Supervision

1 code implementation26 Oct 2020 Yu Zhang, Xiusi Chen, Yu Meng, Jiawei Han

Our experiments demonstrate a consistent improvement of HiMeCat over competitive baselines and validate the contribution of our representation learning and data augmentation modules.

Data Augmentation Document Classification +1

Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation

2 code implementations24 Oct 2020 Yuning Mao, Xiang Ren, Heng Ji, Jiawei Han

Despite significant progress, state-of-the-art abstractive summarization methods are still prone to hallucinate content inconsistent with the source document.

Abstractive Text Summarization Keyphrase Extraction

On the Transformer Growth for Progressive BERT Training

no code implementations NAACL 2021 Xiaotao Gu, Liyuan Liu, Hongkun Yu, Jing Li, Chen Chen, Jiawei Han

Due to the excessive cost of large-scale language model pre-training, considerable efforts have been made to train BERT progressively -- start from an inferior but low-cost model and gradually grow the model to increase the computational complexity.

Language Modelling

BiTe-GCN: A New GCN Architecture via BidirectionalConvolution of Topology and Features on Text-Rich Networks

no code implementations23 Oct 2020 Di Jin, Xiangchen Song, Zhizhi Yu, Ziyang Liu, Heling Zhang, Zhaomeng Cheng, Jiawei Han

We propose BiTe-GCN, a novel GCN architecture with bidirectional convolution of both topology and features on text-rich networks to solve these limitations.

Text Classification Using Label Names Only: A Language Model Self-Training Approach

2 code implementations EMNLP 2020 Yu Meng, Yunyi Zhang, Jiaxin Huang, Chenyan Xiong, Heng Ji, Chao Zhang, Jiawei Han

In this paper, we explore the potential of only using the label name of each class to train classification models on unlabeled data, without using any labeled documents.

Document Classification General Classification +6

CoRel: Seed-Guided Topical Taxonomy Construction by Concept Learning and Relation Transferring

1 code implementation13 Oct 2020 Jiaxin Huang, Yiqing Xie, Yu Meng, Yunyi Zhang, Jiawei Han

Taxonomy is not only a fundamental form of knowledge representation, but also crucial to vast knowledge-rich applications, such as question answering and web search.

Question Answering Relation

A Spherical Hidden Markov Model for Semantics-Rich Human Mobility Modeling

1 code implementation5 Oct 2020 Wanzheng Zhu, Chao Zhang, Shuochao Yao, Xiaobin Gao, Jiawei Han

We propose SHMM, a multi-modal spherical hidden Markov model for semantics-rich human mobility modeling.

SynSetExpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discovery

no code implementations EMNLP 2020 Jiaming Shen, Wenda Qiu, Jingbo Shang, Michelle Vanni, Xiang Ren, Jiawei Han

To facilitate the research on studying the interplays of these two tasks, we create the first large-scale Synonym-Enhanced Set Expansion (SE2) dataset via crowdsourcing.

Generation-Augmented Retrieval for Open-domain Question Answering

1 code implementation ACL 2021 Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, Weizhu Chen

We demonstrate that the generated contexts substantially enrich the semantics of the queries and GAR with sparse representations (BM25) achieves comparable or better performance than state-of-the-art dense retrieval methods such as DPR.

Natural Questions Open-Domain Question Answering +4

Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization

1 code implementation NeurIPS 2021 Qi Zhu, Carl Yang, Yidan Xu, Haonan Wang, Chao Zhang, Jiawei Han

Graph neural networks (GNNs) have achieved superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs.

Knowledge Graphs Transfer Learning

Hierarchical Topic Mining via Joint Spherical Tree and Text Embedding

1 code implementation18 Jul 2020 Yu Meng, Yunyi Zhang, Jiaxin Huang, Yu Zhang, Chao Zhang, Jiawei Han

Mining a set of meaningful topics organized into a hierarchy is intuitively appealing since topic correlations are ubiquitous in massive text corpora.

text-classification Topic Models

GCN for HIN via Implicit Utilization of Attention and Meta-paths

no code implementations6 Jul 2020 Di Jin, Zhizhi Yu, Dongxiao He, Carl Yang, Philip S. Yu, Jiawei Han

Graph neural networks for HIN embeddings typically adopt a hierarchical attention (including node-level and meta-path-level attentions) to capture the information from meta-path-based neighbors.

Octet: Online Catalog Taxonomy Enrichment with Self-Supervision

no code implementations18 Jun 2020 Yuning Mao, Tong Zhao, Andrey Kan, Chenwei Zhang, Xin Luna Dong, Christos Faloutsos, Jiawei Han

We propose to distantly train a sequence labeling model for term extraction and employ graph neural networks (GNNs) to capture the taxonomy structure as well as the query-item-taxonomy interactions for term attachment.

Term Extraction

Unsupervised Differentiable Multi-aspect Network Embedding

1 code implementation7 Jun 2020 Chanyoung Park, Carl Yang, Qi Zhu, Donghyun Kim, Hwanjo Yu, Jiawei Han

To capture the multiple aspects of each node, existing studies mainly rely on offline graph clustering performed prior to the actual embedding, which results in the cluster membership of each node (i. e., node aspect distribution) fixed throughout training of the embedding model.

Clustering Graph Clustering +2

Open-Domain Question Answering with Pre-Constructed Question Spaces

no code implementations NAACL 2021 Jinfeng Xiao, Lidan Wang, Franck Dernoncourt, Trung Bui, Tong Sun, Jiawei Han

Our reader-retriever first uses an offline reader to read the corpus and generate collections of all answerable questions associated with their answers, and then uses an online retriever to respond to user queries by searching the pre-constructed question spaces for answers that are most likely to be asked in the given way.

Information Retrieval Knowledge Graphs +2

Partially-Typed NER Datasets Integration: Connecting Practice to Theory

no code implementations1 May 2020 Shi Zhi, Liyuan Liu, Yu Zhang, Shiyin Wang, Qi Li, Chao Zhang, Jiawei Han

While typical named entity recognition (NER) models require the training set to be annotated with all target types, each available datasets may only cover a part of them.

named-entity-recognition Named Entity Recognition +1

Minimally Supervised Categorization of Text with Metadata

1 code implementation1 May 2020 Yu Zhang, Yu Meng, Jiaxin Huang, Frank F. Xu, Xuan Wang, Jiawei Han

Then, based on the same generative process, we synthesize training samples to address the bottleneck of label scarcity.

Document Classification

Empower Entity Set Expansion via Language Model Probing

1 code implementation ACL 2020 Yunyi Zhang, Jiaming Shen, Jingbo Shang, Jiawei Han

Existing set expansion methods bootstrap the seed entity set by adaptively selecting context features and extracting new entities.

Language Modelling Question Answering

Heterogeneous Network Representation Learning: A Unified Framework with Survey and Benchmark

1 code implementation1 Apr 2020 Carl Yang, Yuxin Xiao, Yu Zhang, Yizhou Sun, Jiawei Han

Since there has already been a broad body of HNE algorithms, as the first contribution of this work, we provide a generic paradigm for the systematic categorization and analysis over the merits of various existing HNE algorithms.

Attribute Network Embedding

Comprehensive Named Entity Recognition on CORD-19 with Distant or Weak Supervision

no code implementations27 Mar 2020 Xuan Wang, Xiangchen Song, Bangzheng Li, Yingjun Guan, Jiawei Han

We created this CORD-NER dataset with comprehensive named entity recognition (NER) on the COVID-19 Open Research Dataset Challenge (CORD-19) corpus (2020-03-13).

named-entity-recognition Named Entity Recognition +1

Guiding Corpus-based Set Expansion by Auxiliary Sets Generation and Co-Expansion

1 code implementation27 Jan 2020 Jiaxin Huang, Yiqing Xie, Yu Meng, Jiaming Shen, Yunyi Zhang, Jiawei Han

Given a small set of seed entities (e. g., ``USA'', ``Russia''), corpus-based set expansion is to induce an extensive set of entities which share the same semantic class (Country in this example) from a given corpus.

Generating Representative Headlines for News Stories

2 code implementations26 Jan 2020 Xiaotao Gu, Yuning Mao, Jiawei Han, Jialu Liu, Hongkun Yu, You Wu, Cong Yu, Daniel Finnie, Jiaqi Zhai, Nicholas Zukoski

In this work, we study the problem of generating representative headlines for news stories.

cube2net: Efficient Query-Specific Network Construction with Data Cube Organization

no code implementations18 Jan 2020 Carl Yang, Mengxiong Liu, Frank He, Jian Peng, Jiawei Han

With extensive experiments of two classic network mining tasks on different real-world large datasets, we show that our proposed cube2net pipeline is general, and much more effective and efficient in query-specific network construction, compared with other methods without the leverage of data cube or reinforcement learning.

reinforcement-learning Reinforcement Learning (RL)

Inf-VAE: A Variational Autoencoder Framework to Integrate Homophily and Influence in Diffusion Prediction

2 code implementations1 Jan 2020 Aravind Sankar, Xinyang Zhang, Adit Krishnan, Jiawei Han

Recent years have witnessed tremendous interest in understanding and predicting information spread on social media platforms such as Twitter, Facebook, etc.

Unsupervised Attributed Multiplex Network Embedding

2 code implementations15 Nov 2019 Chanyoung Park, Donghyun Kim, Jiawei Han, Hwanjo Yu

Even for those that consider the multiplexity of a network, they overlook node attributes, resort to node labels for training, and fail to model the global properties of a graph.

Network Embedding Relation

Mining News Events from Comparable News Corpora: A Multi-Attribute Proximity Network Modeling Approach

no code implementations14 Nov 2019 Hyungsul Kim, Ahmed El-Kishky, Xiang Ren, Jiawei Han

This proximity network captures the corpus-level co-occurence statistics for candidate event descriptors, event attributes, as well as their connections.

Attribute News Summarization

Relation Learning on Social Networks with Multi-Modal Graph Edge Variational Autoencoders

no code implementations4 Nov 2019 Carl Yang, Jieyu Zhang, Haonan Wang, Sha Li, Myungwan Kim, Matt Walker, Yiou Xiao, Jiawei Han

While node semantics have been extensively explored in social networks, little research attention has been paid to profile edge semantics, i. e., social relations.

Relation

Spherical Text Embedding

1 code implementation NeurIPS 2019 Yu Meng, Jiaxin Huang, Guangyuan Wang, Chao Zhang, Honglei Zhuang, Lance Kaplan, Jiawei Han

While text embeddings are typically learned in the Euclidean space, directional similarity is often more effective in tasks such as word similarity and document clustering, which creates a gap between the training stage and usage stage of text embedding.

Clustering Riemannian optimization +1

SetExpan: Corpus-Based Set Expansion via Context Feature Selection and Rank Ensemble

1 code implementation17 Oct 2019 Jiaming Shen, Zeqiu Wu, Dongming Lei, Jingbo Shang, Xiang Ren, Jiawei Han

In this study, we propose a novel framework, SetExpan, which tackles this problem, with two techniques: (1) a context feature selection method that selects clean context features for calculating entity-entity distributional similarity, and (2) a ranking-based unsupervised ensemble method for expanding entity set based on denoised context features.

feature selection Question Answering

FUSE: Multi-Faceted Set Expansion by Coherent Clustering of Skip-grams

1 code implementation10 Oct 2019 Wanzheng Zhu, Hongyu Gong, Jiaming Shen, Chao Zhang, Jingbo Shang, Suma Bhat, Jiawei Han

In this paper, we study the task of multi-faceted set expansion, which aims to capture all semantic facets in the seed set and return multiple sets of entities, one for each semantic facet.

Clustering Language Modelling

Place Deduplication with Embeddings

no code implementations29 Sep 2019 Carl Yang, Do Huy Hoang, Tomas Mikolov, Jiawei Han

Thanks to the advancing mobile location services, people nowadays can post about places to share visiting experience on-the-go.

I Know You'll Be Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social Application

no code implementations29 Sep 2019 Carl Yang, Xiaolin Shi, Jie Luo, Jiawei Han

Then we design a novel deep learning pipeline based on LSTM and attention to accurately predict user churn with very limited initial behavior data, by leveraging the correlations among users' multi-dimensional activities and the underlying user types.

Clustering

Query-Specific Knowledge Summarization with Entity Evolutionary Networks

no code implementations29 Sep 2019 Carl Yang, Lingrui Gan, Zongyi Wang, Jiaming Shen, Jinfeng Xiao, Jiawei Han

Given a query, unlike traditional IR that finds relevant documents or entities, in this work, we focus on retrieving both entities and their connections for insightful knowledge summarization.

Neural Embedding Propagation on Heterogeneous Networks

1 code implementation29 Sep 2019 Carl Yang, Jieyu Zhang, Jiawei Han

While generalizing LP as a simple instance, NEP is far more powerful in its natural awareness of different types of objects and links, and the ability to automatically capture their important interaction patterns.

Network Embedding

Meta-Graph Based HIN Spectral Embedding: Methods, Analyses, and Insights

no code implementations29 Sep 2019 Carl Yang, Yichen Feng, Pan Li, Yu Shi, Jiawei Han

In this work, we propose to study the utility of different meta-graphs, as well as how to simultaneously leverage multiple meta-graphs for HIN embedding in an unsupervised manner.

Discovering Hypernymy in Text-Rich Heterogeneous Information Network by Exploiting Context Granularity

1 code implementation4 Sep 2019 Yu Shi, Jiaming Shen, Yuchen Li, Naijing Zhang, Xinwei He, Zhengzhi Lou, Qi Zhu, Matthew Walker, Myunghwan Kim, Jiawei Han

Extensive experiments on two large real-world datasets demonstrate the effectiveness of HyperMine and the utility of modeling context granularity.

Knowledge Graphs