Search Results for author:
Found 261 papers, 156 papers with code
Phrase-aware Unsupervised Constituency Parsing
Recent studies have achieved inspiring success in unsupervised grammar induction using masked language modeling (MLM) as the proxy task.
Few-Shot Named Entity Recognition: An Empirical Baseline Study
This paper presents an empirical study to efficiently build named entity recognition (NER) systems when a small amount of in-domain labeled data is available.
RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios
We introduce RESIN-11, a new schema-guided event extraction&prediction framework that can be applied to a large variety of newsworthy scenarios.
ChemNER: Fine-Grained Chemistry Named Entity Recognition with Ontology-Guided Distant Supervision
For example, chemistry research needs to study dozens to hundreds of distinct, fine-grained entity types, making consistent and accurate annotation difficult even for crowds of domain experts.
Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
Efficiently acquiring external knowledge and up-to-date information is essential for effective reasoning and text generation in large language models (LLMs).
Multimodal Search in Chemical Documents and Reactions
We present a multimodal search tool that facilitates retrieval of chemical reactions, molecular structures, and associated text from scientific literature.
Tree-of-Debate: Multi-Persona Debate Trees Elicit Critical Thinking for Scientific Comparative Analysis
With the exponential growth of research facilitated by modern technology and improved accessibility, scientific discoveries have become increasingly fragmented within and across fields.
GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphs
The rapid development of Multimodal Large Language Models (MLLMs) has enabled the integration of multiple modalities, including texts and images, within the large language model (LLM) framework.
Improving Scientific Document Retrieval with Concept Coverage-based Query Set Generation
In specialized fields like the scientific domain, constructing large-scale human-annotated datasets poses a significant challenge due to the need for domain expertise.
RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generation
Retrieval-augmented language models often struggle with knowledge-intensive tasks due to inefficient retrieval, unstructured knowledge integration, and single-pass architectures.
LLM Alignment as Retriever Optimization: An Information Retrieval Perspective
In this work, we introduce a novel direct optimization approach for LLM alignment by drawing on established Information Retrieval (IR) principles.
Preference Leakage: A Contamination Problem in LLM-as-a-judge
All of these findings imply that preference leakage is a widespread and challenging problem in the area of LLM-as-a-judge.
Deep Learning and Foundation Models for Weather Prediction: A Survey
Physics-based numerical models have been the bedrock of atmospheric sciences for decades, offering robust solutions but often at the cost of significant computational resources.
Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?
In this work, to address these issues, we leverage multi-modal prompt learning to effectively adapt pre-trained GNN to downstream tasks and data, given only a few semantically labeled samples, each with extremely weak text supervision.
Taxonomy-guided Semantic Indexing for Academic Paper Search
Academic paper search is an essential task for efficient literature discovery and scientific advancement.
Schema-Guided Culture-Aware Complex Event Simulation with Multi-Agent Role-Play
Complex news events, such as natural disasters and socio-political conflicts, require swift responses from the government and society.
Scaling Diffusion Language Models via Adaptation from Autoregressive Models
Diffusion Language Models (DLMs) have emerged as a promising new paradigm for text generative modeling, potentially addressing limitations of autoregressive (AR) models.
Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation
This paper delves into the effects of decoding temperatures on speculative decoding's efficacy.
InstructG2I: Synthesizing Images from Multimodal Attributed Graphs
In this paper, we approach an overlooked yet critical task Graph2Image: generating images from multimodal attributed graphs (MMAGs).
Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
Retrieval-augmented generation (RAG) empowers large language models (LLMs) to utilize external knowledge sources.
Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval
KARE constructs a comprehensive multi-source KG by integrating biomedical databases, clinical literature, and LLM-generated insights, and organizes it using hierarchical graph community detection and summarization for precise and contextually relevant information retrieval.
RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph
Unlike traditional function-level or file-level coding tasks, AI software engineering requires not only basic coding proficiency but also advanced skills in managing and interacting with code repositories.
A Little Goes a Long Way: Efficient Long Context Training and Inference with Partial Contexts
To address this, two critical steps are often required: a pretrained LLM typically undergoes a separate stage for context length extension by training on long-context data, followed by architectural modifications to reduce the overhead of KV cache during serving.
Law of the Weakest Link: Cross Capabilities of Large Language Models
The development and evaluation of Large Language Models (LLMs) have largely focused on individual capabilities.
CoDiCast: Conditional Diffusion Model for Global Weather Prediction with Uncertainty Quantification
Our conditional diffusion model, CoDiCast, can generate 6-day global weather forecasts, at 6-hour steps and $5. 625^\circ$ latitude-longitude resolution, for over 5 variables, in about 12 minutes on a commodity A100 GPU machine with 80GB memory.
Subspace Prototype Guidance for Mitigating Class Imbalance in Point Cloud Semantic Segmentation
Specifically, the point cloud is initially separated into independent point sets by category to provide initial conditions for the generation of feature subspaces.
Ranked #10 on
Semantic Segmentation
on S3DIS Area5
Investigating Instruction Tuning Large Language Models on Graphs
Inspired by the recent advancements of Large Language Models (LLMs) in NLP tasks, there's growing interest in applying LLMs to graph-related tasks.
Unsupervised Episode Detection for Large-Scale News Events
Episodic structures are inherently interpretable and adaptable to evolving large-scale key events.
Establishing Knowledge Preference in Language Models
Language models are known to encode a great amount of factual knowledge through pretraining.
F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data
In this paper, we propose a novel approach that leverages strategically chosen proxy data reflective of potential sales patterns from similar entities during non-peak periods, enriched by features learned from a graph neural networks (GNNs)-based forecasting model, to predict demand during peak events.
Instruct, Not Assist: LLM-based Multi-Turn Planning and Hierarchical Questioning for Socratic Code Debugging
We tackle this issue in the code debugging domain with TreeInstruct, an Instructor agent guided by a novel state space-based planning algorithm.
A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery
In many scientific fields, large language models (LLMs) have revolutionized the way text and other modalities of data (e. g., molecules and proteins) are handled, achieving superior performance in various applications and augmenting the scientific discovery process.
When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs
Our critical survey based on the newly categorized research questions shows that (1) no prior work demonstrates successful self-correction with feedback from prompted LLMs, except for studies in tasks that are exceptionally suited for self-correction, (2) self-correction works well in tasks that can use reliable external feedback, and (3) large-scale fine-tuning enables self-correction.
PathReasoner: Modeling Reasoning Path with Equivalent Extension for Logical Question Answering
In particular, we propose a path-attention module to joint model in-atom and cross-atom relations with the high-order diffusion strategy.
KG-FIT: Knowledge Graph Fine-Tuning Upon Open-World Knowledge
Knowledge Graph Embedding (KGE) techniques are crucial in learning compact representations of entities and relations within a knowledge graph, facilitating efficient reasoning and knowledge discovery.
Automated Construction of Theme-specific Knowledge Graphs
Specifically, we start with an entity ontology of the theme from Wikipedia, based on which we then generate candidate relations by Large Language Models (LLMs) to construct a relation ontology.
MolCRAFT: Structure-Based Drug Design in Continuous Parameter Space
Generative models for structure-based drug design (SBDD) have shown promising results in recent years.
Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
Then, we propose a simple and effective framework called Graph Chain-of-thought (Graph-CoT) to augment LLMs with graphs by encouraging LLMs to reason on the graph iteratively.
TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale
To overcome this, we introduce TriSum, a framework for distilling LLMs' text summarization abilities into a compact, local model.
ILCiteR: Evidence-grounded Interpretable Local Citation Recommendation
We contribute a novel dataset for the evidence-grounded local citation recommendation task and demonstrate the efficacy of our proposed conditional neural rank-ensembling approach for re-ranking evidence spans.
Improving Retrieval in Theme-specific Applications using a Corpus Topical Taxonomy
Document retrieval has greatly benefited from the advancements of large-scale pre-trained language models (PLMs).
TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervision
On the other hand, previous weakly-supervised hierarchical text classification methods only utilize the raw taxonomy skeleton and ignore the rich information hidden in the text corpus that can serve as additional class-indicative features.
Multi-LoRA Composition for Image Generation
Low-Rank Adaptation (LoRA) is extensively utilized in text-to-image models for the accurate rendition of specific elements like distinct characters or unique styles in generated images.
A Unified Taxonomy-Guided Instruction Tuning Framework for Entity Set Expansion and Taxonomy Expansion
To be specific, we identify two common skills needed for entity set expansion, taxonomy expansion, and seed-guided taxonomy construction: finding "siblings" and finding "parents".
Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction
(3) We expand pattern coverage and mitigate bias from initial seeds by integrating feedback from the SLM's predictions on the unlabeled corpus and the synthesis history.
GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Models
The field of relation extraction (RE) is experiencing a notable shift towards generative relation extraction (GRE), leveraging the capabilities of large language models (LLMs).
Similarity-based Neighbor Selection for Graph LLMs
Our research further underscores the significance of graph structure integration in LLM applications and identifies key factors for their success in node classification.
Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains
In this paper, we study the task of seed-guided fine-grained entity typing in science and engineering domains, which takes the name and a few seed entities for each entity type as the only supervision and aims to classify new entity mentions into both seen and unseen types (i. e., those without seed entities).
Chem-FINESE: Validating Fine-Grained Few-shot Entity Extraction through Text Reconstruction
Fine-grained few-shot entity extraction in the chemical domain faces two unique challenges.
Investigating Data Contamination for Pre-training Language Models
Language models pre-trained on web-scale corpora demonstrate impressive capabilities on diverse downstream tasks.
TrustLLM: Trustworthiness in Large Language Models
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
RESIN-EDITOR: A Schema-guided Hierarchical Event Graph Visualizer and Editor
In this paper, we present RESIN-EDITOR, an interactive event graph visualizer and editor designed for analyzing complex events.
Large Language Models on Graphs: A Comprehensive Survey
Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i. e., graph-based reasoning).
SCStory: Self-supervised and Continual Online Story Discovery
With a lightweight hierarchical embedding module that first learns sentence representations and then article representations, SCStory identifies story-relevant information of news articles and uses them to discover stories.
Structured Chemistry Reasoning with Large Language Models
Large Language Models (LLMs) excel in diverse areas, yet struggle with complex scientific reasoning, especially in the field of chemistry.
MART: Improving LLM Safety with Multi-round Automatic Red-Teaming
Specifically, an adversarial LLM and a target LLM interplay with each other in an iterative manner, where the adversarial LLM aims to generate challenging prompts that elicit unsafe responses from the target LLM, while the target LLM is fine-tuned with safety aligned data on these adversarial prompts.
Don't Make Your LLM an Evaluation Benchmark Cheater
Large language models~(LLMs) have greatly advanced the frontiers of artificial intelligence, attaining remarkable improvement in model capacity.
Instruct and Extract: Instruction Tuning for On-Demand Information Extraction
However, when it comes to information extraction - a classic task in natural language processing - most task-specific systems cannot align well with long-tail ad hoc extraction use cases for non-expert users.
Chain-of-Factors Paper-Reviewer Matching
To address this issue, we propose a unified model for paper-reviewer matching that jointly considers semantic, topic, and citation factors.
The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions
Recent progress in Large Language Models (LLMs) has produced models that exhibit remarkable performance across a variety of NLP tasks.
Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
In this paper, we seek to empirically investigate knowledge transfer from larger to smaller models through a parametric perspective.
Ontology Enrichment for Effective Fine-grained Entity Typing
In this study, we propose OnEFET, where we (1) enrich each node in the ontology structure with two types of extra information: instance information for training sample augmentation and topic information to relate types to contexts, and (2) develop a coarse-to-fine typing algorithm that exploits the enriched information by training an entailment model with contrasting topics and instance-based augmented training samples.
Language Models As Semantic Indexers
Semantic identifier (ID) is an important concept in information retrieval that aims to preserve the semantics of objects such as documents and items inside their IDs.
Learning Multiplex Representations on Text-Attributed Graphs with One Language Model Encoder
Mainstream text representation learning methods use pretrained language models (PLMs) to generate one embedding for each text unit, expecting that all types of relations between texts can be captured by these single-view embeddings.
Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
In this study, we introduce adaptive KV cache compression, a plug-and-play method that reduces the memory footprint of generative inference for Large Language Models (LLMs).
Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification
Event schemas are a form of world knowledge about the typical progression of events.
ReactIE: Enhancing Chemical Reaction Extraction with Weak Supervision
Structured chemical reaction information plays a vital role for chemists engaged in laboratory work and advanced endeavors such as computer-aided drug design.
Weakly Supervised Multi-Label Classification of Full-Text Scientific Papers
Instead of relying on human-annotated training samples to build a classifier, weakly supervised scientific paper classification aims to classify papers only using category descriptions (e. g., category names, category-indicative keywords).
Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond
Logical reasoning consistently plays a fundamental and significant role in the domains of knowledge engineering and artificial intelligence.
Explaining and Adapting Graph Conditional Shift
Graph Neural Networks (GNNs) have shown remarkable performance on graph-structured data.
Bi-level Contrastive Learning for Knowledge-Enhanced Molecule Representations
In this paper, we introduce a novel method called GODE, which accounts for the dual-level structure inherent in molecules.
Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models
The mission of open knowledge graph (KG) completion is to draw new findings from known facts.
Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation
Instruction tuning has emerged to enhance the capabilities of large language models (LLMs) to comprehend instructions and generate appropriate responses.
PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training
Weakly-supervised text classification trains a classifier using the label name of each target class as the only supervision, which largely reduces human annotation efforts.
OntoType: Ontology-Guided and Pre-Trained Language Model Assisted Fine-Grained Entity Typing
Fine-grained entity typing (FET), which assigns entities in text with context-sensitive, fine-grained semantic types, is a basic but important task for knowledge extraction from unstructured text.
Patton: Language Model Pretraining on Text-Rich Networks
A real-world text corpus sometimes comprises not only text documents but also semantic links between them (e. g., academic papers in a bibliographic network are linked by citations and co-authorships).
Unsupervised Story Discovery from Continuous News Streams via Scalable Thematic Embedding
Unsupervised discovery of stories with correlated news articles in real-time helps people digest massive news streams without expensive human annotations.
MEGClass: Extremely Weakly Supervised Text Classification via Mutually-Enhancing Text Granularities
By preserving the heterogeneity of potential classes, MEGClass can select the most informative class-indicative documents as iterative feedback to enhance the initial word-based class representations and ultimately fine-tune a pre-trained text classifier.
GLEN: General-Purpose Event Detection for Thousands of Types
Finally, we perform error analysis and show that label noise is still the largest challenge for improving performance for this new dataset.
Edgeformers: Graph-Empowered Transformers for Representation Learning on Textual-Edge Networks
Edges in many real-world social/information networks are associated with rich text information (e. g., user-user communications or user-product reviews).
PDSum: Prototype-driven Continuous Summarization of Evolving Multi-document Sets Stream
Summarizing text-rich documents has been long studied in the literature, but most of the existing efforts have been made to summarize a static and predefined multi-document set.
The Effect of Metadata on Scientific Literature Tagging: A Cross-Field Cross-Model Study
Due to the exponential growth of scientific publications on the Web, there is a pressing need to tag each paper with fine-grained topics so that researchers can track their interested fields of study rather than drowning in the whole literature.
Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories
In this paper we improve the zero-shot generalization ability of language models via Mixture-Of-Memory Augmentation (MoMA), a mechanism that retrieves augmentation documents from multiple information corpora ("external memories"), with the option to "plug in" new memory at inference time.
Representation Deficiency in Masked Language Modeling
In this work, we offer a new perspective on the consequence of such a discrepancy: We demonstrate empirically and theoretically that MLM pretraining allocates some model dimensions exclusively for representing $\texttt{[MASK]}$ tokens, resulting in a representation deficiency for real tokens and limiting the pretrained model's expressiveness when it is adapted to downstream data without $\texttt{[MASK]}$ tokens.
Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts
Instead of mining coherent topics from a given text corpus in a completely unsupervised manner, seed-guided topic discovery methods leverage user-provided seed words to extract distinctive and coherent topics so that the mined topics can better cater to the user's interest.
Entity Set Co-Expansion in StackOverflow
Given a few seed entities of a certain type (e. g., Software or Programming Language), entity set expansion aims to discover an extensive set of entities that share the same type as the seeds.
Open Relation and Event Type Discovery with Type Abstraction
To tackle this problem, we introduce the idea of type abstraction, where the model is prompted to generalize and name the type.
Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning
In this work, we study few-shot learning with PLMs from a different perspective: We first tune an autoregressive PLM on the few-shot samples and then use it as a generator to synthesize a large amount of novel training samples which augment the original training set.
Open-Vocabulary Argument Role Prediction for Event Extraction
Specifically, we formulate the role prediction problem as an in-filling task and construct prompts for a pre-trained language model to generate candidate roles.
PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion
For instance, we outperform the fully finetuning approaches on a KG completion benchmark by tuning only 1% of the parameters.
Ranked #5 on
Link Prediction
on UMLS
Large Language Models Can Self-Improve
We show that our approach improves the general reasoning ability of a 540B-parameter LLM (74. 4%->82. 1% on GSM8K, 78. 2%->83. 0% on DROP, 90. 0%->94. 4% on OpenBookQA, and 63. 4%->67. 9% on ANLI-A3) and achieves state-of-the-art-level performance, without any ground truth label.
Ranked #1 on
Question Answering
on DROP
Topic Taxonomy Expansion via Hierarchy-Aware Topic Phrase Generation
Topic taxonomies display hierarchical topic structures of a text corpus and provide topical knowledge to enhance various NLP applications.
Towards a Unified Multi-Dimensional Evaluator for Text Generation
We re-frame NLG evaluation as a Boolean Question Answering (QA) task, and by guiding the model with different questions, we can use one evaluator to evaluate from multiple dimensions.
Few-shot Text Classification with Dual Contrastive Consistency
In this paper, we explore how to utilize pre-trained language model to perform few-shot text classification where only a few annotated examples are given for each class.
Survey of Query-based Text Summarization
Query-based text summarization is an important real world problem that requires to condense the prolix text data into a summary under the guidance of the query information provided by users.
TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter
Most existing PLMs are not tailored to the noisy user-generated text on social media, and the pre-training does not factor in the valuable social engagement logs available in a social network.
MentorGNN: Deriving Curriculum for Pre-Training GNNs
To comprehend heterogeneous graph signals at different granularities, we propose a curriculum learning paradigm that automatically re-weighs graph signals in order to ensure a good generalization in the target domain.
Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation and Instance Generation
We study the problem of few-shot Fine-grained Entity Typing (FET), where only a few annotated entity mentions with contexts are given for each entity type.
TeKo: Text-Rich Graph Neural Networks with External Knowledge
Graph Neural Networks (GNNs) have gained great popularity in tackling various analytical tasks on graph-structured data (i. e., networks).
Unsupervised Key Event Detection from Massive Text Corpora
Automated event detection from news corpora is a crucial task towards mining fast-evolving structured knowledge.
Schema-Guided Event Graph Completion
Existing link prediction or graph completion methods have difficulty dealing with event graphs because they are usually designed for a single large graph such as a social network or a knowledge graph, rather than multiple small dynamic event graphs.
All Birds with One Stone: Multi-task Text Classification for Efficient Inference with One Forward Pass
Multi-Task Learning (MTL) models have shown their robustness, effectiveness, and efficiency for transferring learned knowledge across tasks.
Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networks
In heterogeneous text-rich networks, this task is more challenging due to (1) presence or absence of text: Some nodes are associated with rich textual information, while others are not; (2) diversity of types: Nodes and edges of multiple types form a heterogeneous network structure.
CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision
Scientific extreme summarization (TLDR) aims to form ultra-short summaries of scientific papers.
Ranked #1 on
Extreme Summarization
on CiteSum
Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds
Discovering latent topics from text corpora has been studied for decades.
OA-Mine: Open-World Attribute Mining for E-Commerce Products with Weak Supervision
Most prior works on this matter mine new values for a set of known attributes but cannot handle new attributes that arose from constantly changing data.
Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators
We present a new framework AMOS that pretrains text encoders with an Adversarial learning curriculum via a Mixture Of Signals from multiple auxiliary generators.
Shift-Robust Node Classification via Graph Adversarial Clustering
Then a shift-robust classifier is optimized on training graph and adversarial samples on target graph, which are generated by cluster GNN.
P4E: Few-Shot Event Detection as Prompt-Guided Identification and Localization
Our framework decomposes event detection into an identification task and a localization task.
Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification
Large-scale multi-label text classification (LMTC) aims to associate a document with its relevant labels from a large candidate set.
TaxoEnrich: Self-Supervised Taxonomy Completion via Structure-Semantic Representations
Taxonomies are fundamental to many real-world applications in various domains, serving as structural representations of knowledge.
Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations
Interestingly, there have not been standard approaches to deploy PLMs for topic discovery as better alternatives to topic models.
Generating Training Data with Language Models: Towards Zero-Shot Language Understanding
Pretrained language models (PLMs) have demonstrated remarkable performance in various natural language processing tasks: Unidirectional PLMs (e. g., GPT) are well known for their superior text generation capabilities; bidirectional PLMs (e. g., BERT) have been the prominent choice for natural language understanding (NLU) tasks.
Ranked #5 on
Zero-Shot Text Classification
on AG News
Unsupervised Multi-Granularity Summarization
In this paper, we propose the first unsupervised multi-granularity summarization framework, GranuSum.
TaxoCom: Topic Taxonomy Completion with Hierarchical Discovery of Novel Topic Clusters
Topic taxonomies, which represent the latent topic (or category) structure of document collections, provide valuable knowledge of contents in many applications such as web search and information filtering.
Universal Graph Convolutional Networks
So can we reasonably utilize these segmentation rules to design a universal propagation mechanism independent of the network structural assumption?
Out-of-Category Document Identification Using Target-Category Names as Weak Supervision
To address this challenge, we introduce a new task referred to as out-of-category detection, which aims to distinguish the documents according to their semantic relevance to the inlier (or target) categories by using the category names as weak supervision.
MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information
We study the problem of weakly supervised text classification, which aims to classify text documents into a set of pre-defined categories with category surface names only and without any annotated training document provided.
Fine-Grained Opinion Summarization with Minimal Supervision
Opinion summarization aims to profile a target by extracting opinions from multiple documents.
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning
Recent parameter-efficient language model tuning (PELT) methods manage to match the performance of fine-tuning with much fewer trainable parameters and perform especially well when training data is limited.
Entity Linking Meets Deep Learning: Techniques and Solutions
Entity linking (EL) is the process of linking entity mentions appearing in web text with their corresponding entities in a knowledge base.
SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction
Consequently, it is more challenging to encode the key information sources--relevant contexts and entity types.
Ranked #1 on
Relation Extraction
on CDR
Chemical-Reaction-Aware Molecule Representation Learning
Molecule representation learning (MRL) methods aim to embed molecules into a real vector space.
Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training
We study the problem of training named entity recognition (NER) models using only distantly-labeled data, which can be automatically obtained by matching entity mentions in the raw text with entity types in a knowledge base.
Corpus-based Open-Domain Event Type Induction
As events of the same type could be expressed in multiple ways, we propose to represent each event type as a cluster of <predicate sense, object head> pairs.
Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data
In this work we present a method, Shift-Robust GNN (SR-GNN), designed to account for distributional differences between biased training data and the graph's true inference distribution.
Multi-head or Single-head? An Empirical Comparison for Transformer Training
Multi-head attention plays a crucial role in the recent success of Transformer models, which leads to consistent performance improvements over conventional attention in various applications.
Eider: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusion
Typical DocRE methods blindly take the full document as input, while a subset of the sentences in the document, noted as the evidence, are often sufficient for humans to predict the relation of an entity pair.
Ranked #5 on
Relation Extraction
on DocRED
TaxoClass: Hierarchical Multi-Label Text Classification Using Only Class Names
Hierarchical multi-label text classification (HMTC) aims to tag each document with a set of classes from a taxonomic class hierarchy.
Multi Label Text Classification
Multi-Label Text Classification
+3
Event Time Extraction and Propagation via Graph Attention Networks
Grounding events into a precise timeline is important for natural language understanding but has received limited attention in recent work.
RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System
We present a new information extraction system that can automatically construct temporal event graphs from a collection of news documents from multiple sources, multiple languages (English and Spanish for our experiment), and multiple data modalities (speech, text, image and video).
Training ELECTRA Augmented with Multi-word Selection
In this study, we present a new text encoder pre-training method that improves ELECTRA based on multi-task learning.
UCPhrase: Unsupervised Context-aware Quality Phrase Tagging
Training a conventional neural tagger based on silver labels usually faces the risk of overfitting phrase surface names.
Ranked #1 on
Phrase Tagging
on KPTimes
Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation
In this paper, we present a systematic analysis that studies whether current seq2seq models, especially pre-trained language models, are good enough for preserving important input concepts and to what extent explicitly guiding generation with the concepts as lexical constraints is beneficial.
The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction
We introduce a new concept of Temporal Complex Event Schema: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations.
Document-Level Event Argument Extraction by Conditional Generation
On the task of argument extraction, we achieve an absolute gain of 7. 6% F1 and 5. 7% F1 over the next best model on the RAMS and WikiEvents datasets respectively.
Document-level Event Extraction
Event Argument Extraction
+2
Who Should Go First? A Self-Supervised Concept Sorting Model for Improving Taxonomy Expansion
Taxonomies have been widely used in various machine learning and text mining systems to organize knowledge and facilitate downstream tasks.
Toward Tweet Entity Linking with Heterogeneous Information Networks
The task of linking an entity mention in a tweet with its corresponding entity in a heterogeneous information network is of great importance, for the purpose of enriching heterogeneous information networks with the abundant and fresh knowledge embedded in tweets.
Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks
Specifically, we jointly train two modules with different inductive biases -- a text analysis module for text understanding and a network learning module for class-discriminative, scalable network learning.
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining
The first token-level task, Corrective Language Modeling, is to detect and correct tokens replaced by the auxiliary model, in order to better capture token-level semantics.
MATCH: Metadata-Aware Text Classification in A Large Hierarchy
Multi-label text classification refers to the problem of assigning each given document its most relevant labels from the label set.
Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup
Contrastive learning has been applied successfully to learn vector representations of text.
Rider: Reader-Guided Passage Reranking for Open-Domain Question Answering
Current open-domain question answering systems often follow a Retriever-Reader architecture, where the retriever first retrieves relevant passages and the reader then reads the retrieved passages to form an answer.
Few-Shot Named Entity Recognition: A Comprehensive Study
This paper presents a comprehensive study to efficiently build named entity recognition (NER) systems when a small number of in-domain labeled data is available.
Hierarchical Metadata-Aware Document Categorization under Weak Supervision
Our experiments demonstrate a consistent improvement of HiMeCat over competitive baselines and validate the contribution of our representation learning and data augmentation modules.
Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation
Despite significant progress, state-of-the-art abstractive summarization methods are still prone to hallucinate content inconsistent with the source document.
On the Transformer Growth for Progressive BERT Training
Due to the excessive cost of large-scale language model pre-training, considerable efforts have been made to train BERT progressively -- start from an inferior but low-cost model and gradually grow the model to increase the computational complexity.
BiTe-GCN: A New GCN Architecture via BidirectionalConvolution of Topology and Features on Text-Rich Networks
We propose BiTe-GCN, a novel GCN architecture with bidirectional convolution of both topology and features on text-rich networks to solve these limitations.
CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding
To verify the effectiveness of the proposed framework, we apply CoDA to Transformer-based models on a wide range of natural language understanding tasks.
Text Classification Using Label Names Only: A Language Model Self-Training Approach
In this paper, we explore the potential of only using the label name of each class to train classification models on unlabeled data, without using any labeled documents.
Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding
Aspect-based sentiment analysis of review texts is of great value for understanding user feedback in a fine-grained manner.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+2
CoRel: Seed-Guided Topical Taxonomy Construction by Concept Learning and Relation Transferring
Taxonomy is not only a fundamental form of knowledge representation, but also crucial to vast knowledge-rich applications, such as question answering and web search.
A Spherical Hidden Markov Model for Semantics-Rich Human Mobility Modeling
We propose SHMM, a multi-modal spherical hidden Markov model for semantics-rich human mobility modeling.
Near-imperceptible Neural Linguistic Steganography via Self-Adjusting Arithmetic Coding
Linguistic steganography studies how to hide secret messages in natural language cover texts.
Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning
Additionally, the explicit redundancy measure in MMR helps the neural representation of the summary to better capture redundancy.
SynSetExpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discovery
To facilitate the research on studying the interplays of these two tasks, we create the first large-scale Synonym-Enhanced Set Expansion (SE2) dataset via crowdsourcing.
Generation-Augmented Retrieval for Open-domain Question Answering
We demonstrate that the generated contexts substantially enrich the semantics of the queries and GAR with sparse representations (BM25) achieves comparable or better performance than state-of-the-art dense retrieval methods such as DPR.
Ranked #9 on
Passage Retrieval
on Natural Questions
Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization
Graph neural networks (GNNs) have achieved superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs.
Hierarchical Topic Mining via Joint Spherical Tree and Text Embedding
Mining a set of meaningful topics organized into a hierarchy is intuitively appealing since topic correlations are ubiquitous in massive text corpora.
Ranked #1 on
Topic Models
on Arxiv HEP-TH citation graph
GCN for HIN via Implicit Utilization of Attention and Meta-paths
Graph neural networks for HIN embeddings typically adopt a hierarchical attention (including node-level and meta-path-level attentions) to capture the information from meta-path-based neighbors.
COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation
To combat COVID-19, both clinicians and scientists need to digest vast amounts of relevant biomedical knowledge in scientific literature to understand the disease mechanism and related biological functions.
EVIDENCEMINER: Textual Evidence Discovery for Life Sciences
EVIDENCEMINER is constructed in a completely automated way without any human effort for training data annotation.
AutoKnow: Self-Driving Knowledge Collection for Products of Thousands of Types
Can one build a knowledge graph (KG) for all products in the world?
Octet: Online Catalog Taxonomy Enrichment with Self-Supervision
We propose to distantly train a sequence labeling model for term extraction and employ graph neural networks (GNNs) to capture the taxonomy structure as well as the query-item-taxonomy interactions for term attachment.
Unsupervised Differentiable Multi-aspect Network Embedding
To capture the multiple aspects of each node, existing studies mainly rely on offline graph clustering performed prior to the actual embedding, which results in the cluster membership of each node (i. e., node aspect distribution) fixed throughout training of the embedding model.
Open-Domain Question Answering with Pre-Constructed Question Spaces
Our reader-retriever first uses an offline reader to read the corpus and generate collections of all answerable questions associated with their answers, and then uses an online retriever to respond to user queries by searching the pre-constructed question spaces for answers that are most likely to be asked in the given way.
Partially-Typed NER Datasets Integration: Connecting Practice to Theory
While typical named entity recognition (NER) models require the training set to be annotated with all target types, each available datasets may only cover a part of them.
Minimally Supervised Categorization of Text with Metadata
Then, based on the same generative process, we synthesize training samples to address the bottleneck of label scarcity.
Empower Entity Set Expansion via Language Model Probing
Existing set expansion methods bootstrap the seed entity set by adaptively selecting context features and extracting new entities.
Automatic Textual Evidence Mining in COVID-19 Literature
We created this EVIDENCEMINER system for automatic textual evidence mining in COVID-19 literature.
Understanding the Difficulty of Training Transformers
Transformers have proved effective in many NLP tasks.
Ranked #5 on
Machine Translation
on WMT2014 English-French
Heterogeneous Network Representation Learning: A Unified Framework with Survey and Benchmark
Since there has already been a broad body of HNE algorithms, as the first contribution of this work, we provide a generic paradigm for the systematic categorization and analysis over the merits of various existing HNE algorithms.
Comprehensive Named Entity Recognition on CORD-19 with Distant or Weak Supervision
We created this CORD-NER dataset with comprehensive named entity recognition (NER) on the COVID-19 Open Research Dataset Challenge (CORD-19) corpus (2020-03-13).
Guiding Corpus-based Set Expansion by Auxiliary Sets Generation and Co-Expansion
Given a small set of seed entities (e. g., ``USA'', ``Russia''), corpus-based set expansion is to induce an extensive set of entities which share the same semantic class (Country in this example) from a given corpus.
Generating Representative Headlines for News Stories
In this work, we study the problem of generating representative headlines for news stories.
TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network
Taxonomies consist of machine-interpretable semantics and provide valuable knowledge for many web applications.
cube2net: Efficient Query-Specific Network Construction with Data Cube Organization
With extensive experiments of two classic network mining tasks on different real-world large datasets, we show that our proposed cube2net pipeline is general, and much more effective and efficient in query-specific network construction, compared with other methods without the leverage of data cube or reinforcement learning.
Inf-VAE: A Variational Autoencoder Framework to Integrate Homophily and Influence in Diffusion Prediction
Recent years have witnessed tremendous interest in understanding and predicting information spread on social media platforms such as Twitter, Facebook, etc.
Unsupervised Attributed Multiplex Network Embedding
Even for those that consider the multiplexity of a network, they overlook node attributes, resort to node labels for training, and fail to model the global properties of a graph.
Mining News Events from Comparable News Corpora: A Multi-Attribute Proximity Network Modeling Approach
This proximity network captures the corpus-level co-occurence statistics for candidate event descriptors, event attributes, as well as their connections.
Relation Learning on Social Networks with Multi-Modal Graph Edge Variational Autoencoders
While node semantics have been extensively explored in social networks, little research attention has been paid to profile edge semantics, i. e., social relations.
Spherical Text Embedding
While text embeddings are typically learned in the Euclidean space, directional similarity is often more effective in tasks such as word similarity and document clustering, which creates a gap between the training stage and usage stage of text embedding.
HiExpan: Task-Guided Taxonomy Construction by Hierarchical Tree Expansion
Taxonomies are of great value to many knowledge-rich applications.
SetExpan: Corpus-Based Set Expansion via Context Feature Selection and Rank Ensemble
In this study, we propose a novel framework, SetExpan, which tackles this problem, with two techniques: (1) a context feature selection method that selects clean context features for calculating entity-entity distributional similarity, and (2) a ranking-based unsupervised ensemble method for expanding entity set based on denoised context features.
HiGitClass: Keyword-Driven Hierarchical Classification of GitHub Repositories
With the massive number of repositories available, there is a pressing need for topic-based search.
FUSE: Multi-Faceted Set Expansion by Coherent Clustering of Skip-grams
In this paper, we study the task of multi-faceted set expansion, which aims to capture all semantic facets in the seed set and return multiple sets of entities, one for each semantic facet.
Meta-Graph Based HIN Spectral Embedding: Methods, Analyses, and Insights
In this work, we propose to study the utility of different meta-graphs, as well as how to simultaneously leverage multiple meta-graphs for HIN embedding in an unsupervised manner.
Query-Specific Knowledge Summarization with Entity Evolutionary Networks
Given a query, unlike traditional IR that finds relevant documents or entities, in this work, we focus on retrieving both entities and their connections for insightful knowledge summarization.
Place Deduplication with Embeddings
Thanks to the advancing mobile location services, people nowadays can post about places to share visiting experience on-the-go.
I Know You'll Be Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social Application
Then we design a novel deep learning pipeline based on LSTM and attention to accurately predict user churn with very limited initial behavior data, by leveraging the correlations among users' multi-dimensional activities and the underlying user types.
Neural Embedding Propagation on Heterogeneous Networks
While generalizing LP as a simple instance, NEP is far more powerful in its natural awareness of different types of objects and links, and the ability to automatically capture their important interaction patterns.
Discovering Hypernymy in Text-Rich Heterogeneous Information Network by Exploiting Context Granularity
Extensive experiments on two large real-world datasets demonstrate the effectiveness of HyperMine and the utility of modeling context granularity.
CrossWeigh: Training Named Entity Tagger from Imperfect Annotations
Therefore, we manually correct these label mistakes and form a cleaner test set.
Ranked #6 on
Named Entity Recognition (NER)
on CoNLL++
(using extra training data)
Hierarchical Text Classification with Reinforced Label Assignment
While existing hierarchical text classification (HTC) methods attempt to capture label hierarchies for model training, they either make local decisions regarding each label or completely ignore the hierarchy information during inference.
Ranked #1 on
Text Classification
on RCV1
(Macro F1 metric)
Facet-Aware Evaluation for Extractive Summarization
In this paper, we present a facet-aware evaluation setup for better assessment of the information coverage in extracted summaries.
Discriminative Topic Mining via Category-Name Guided Text Embedding
We propose a new task, discriminative topic mining, which leverages a set of user-provided category names to mine discriminative topics from text corpora.
Parsimonious Morpheme Segmentation with an Application to Enriching Word Embeddings
However, in many languages and specialized corpora, words are composed by concatenating semantically meaningful subword structures.
Raw-to-End Name Entity Recognition in Social Media
Our model neither requires the conversion from character sequences to word sequences, nor assumes tokenizer can correctly detect all word boundaries.
On the Variance of the Adaptive Learning Rate and Beyond
The learning rate warmup heuristic achieves remarkable success in stabilizing training, accelerating convergence and improving generalization for adaptive stochastic optimization algorithms like RMSprop and Adam.
