Search Results for author:
Found 227 papers, 134 papers with code
Graph-based Consensus Maximization among Multiple Supervised and Unsupervised Models
First, we can boost the diversity of classification ensemble by incorporating multiple clustering outputs, each of which provides grouping constraints for the joint label predictions of a set of related objects.
Graph Regularized Nonnegative Matrix Factorization for Data Representation
In GNMF, an affinity graph is constructed to encode the geometrical information and we seek a matrix factorization, which respects the graph structure.
Generalized Fisher Score for Feature Selection
Fisher score is one of the most widely used supervised feature selection methods.
A Simple Algorithm for Semi-supervised Learning with Improved Generalization Error Bound
In this work, we develop a simple algorithm for semi-supervised regression.
Selective Labeling via Error Bound Minimization
In particular, we derive a deterministic generalization error bound for LapRLS trained on subsampled data, and propose to select a subset of data points to label by minimizing this upper bound.
KERT: Automatic Extraction and Ranking of Topical Keyphrases from Content-Representative Document Titles
We introduce KERT (Keyphrase Extraction and Ranking by Topic), a framework for topical keyphrase generation and ranking.
Scalable and Robust Construction of Topical Hierarchies
Automated generation of high-quality topical hierarchies for a text collection is a dream problem in knowledge engineering with many valuable applications.
Scalable Topical Phrase Mining from Text Corpora
Our solution combines a novel phrase mining framework to segment a document into single and multi-word phrases, and a new topic model that operates on the induced document partition.
Robust Tensor Decomposition with Gross Corruption
In this paper, we study the statistical performance of robust tensor decomposition with gross corruption.
Label Noise Reduction in Entity Typing by Heterogeneous Partial-Label Embedding
Current systems of fine-grained entity typing use distant supervision in conjunction with existing knowledge bases to assign categories (type labels) to entity mentions.
World Knowledge as Indirect Supervision for Document Clustering
We provide three ways to specify the world knowledge to domains by resolving the ambiguity of the entities and their types, and represent the data with world knowledge as a heterogeneous information network.
pg-Causality: Identifying Spatiotemporal Causal Pathways for Air Pollutants with Urban Big Data
Therefore, we present \emph{p-Causality}, a novel pattern-aided causality analysis approach that combines the strengths of \emph{pattern mining} and \emph{Bayesian learning} to efficiently and faithfully identify the \emph{ST causal pathways}.
CoType: Joint Extraction of Typed Entities and Relations with Knowledge Bases
We propose a novel domain-independent framework, called CoType, that runs a data-driven text segmentation algorithm to extract entity mentions, and jointly embeds entity mentions, relation mentions, text features and type labels into two low-dimensional spaces (for entity and relation mentions respectively), where, in each space, objects whose types are close will also have similar representations.
Ranked #11 on
Relation Extraction
on NYT11-HRL
Meta-Path Guided Embedding for Similarity Search in Large-Scale Heterogeneous Information Networks
It models vertices as low-dimensional vectors to explore network structure-embedded similarity.
DPPred: An Effective Prediction Framework with Concise Discriminative Patterns
In the literature, two series of models have been proposed to address prediction problems including classification and regression.
FastHybrid: A Hybrid Model for Efficient Answer Selection
In this paper, we propose an extremely efficient hybrid model (FastHybrid) that tackles the problem from both an accuracy and scalability point of view.
Automated Phrase Mining from Massive Text Corpora
As one of the fundamental tasks in text analysis, phrase mining aims at extracting quality phrases from a text corpus.
MetaPAD: Meta Pattern Discovery from Massive Text Corpora
We propose an efficient framework, called MetaPAD, which discovers meta patterns from massive corpora with three techniques: (1) it develops a context-aware segmentation method to carefully determine the boundaries of patterns with a learnt pattern quality assessment function, which avoids costly dependency parsing and generates high-quality patterns; (2) it identifies and groups synonymous meta patterns from multiple facets---their types, contexts, and extractions; and (3) it examines type distributions of entities in the instances extracted by each group of patterns, and looks for appropriate type levels to make discovered patterns precise.
PReP: Path-Based Relevance from a Probabilistic Perspective in Heterogeneous Information Networks
We also identify, from real-world data, and propose to model cross-meta-path synergy, which is a characteristic important for defining path-based HIN relevance and has not been modeled by existing methods.
Automatic Synonym Discovery with Knowledge Bases
In this paper, we study the problem of automatic synonym discovery with knowledge bases, that is, identifying synonyms for knowledge base entities in a given domain-specific corpus.
Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach
These annotations, referred as heterogeneous supervision, often conflict with each other, which brings a new challenge to the original relation extraction task: how to infer the true label from noisy labels for a given instance.
Identifying Semantically Deviating Outlier Documents
A document outlier is a document that substantially deviates in semantics from the majority ones in a corpus.
Empower Sequence Labeling with Task-Aware Neural Language Model
In this study, we develop a novel neural framework to extract abundant knowledge hidden in raw texts to empower the sequence labeling task.
Ranked #13 on
Part-Of-Speech Tagging
on Penn Treebank
An Attention-based Collaboration Framework for Multi-View Network Representation Learning
Existing approaches usually study networks with a single type of proximity between nodes, which defines a single view of a network.
Indirect Supervision for Relation Extraction using Question-Answer Pairs
However, due to the incompleteness of knowledge bases and the context-agnostic labeling, the training data collected via distant supervision (DS) can be very noisy.
Weakly-supervised Relation Extraction by Pattern-enhanced Embedding Learning
We propose a novel co-training framework with a distributional module and a pattern module.
Graph Clustering with Dynamic Embedding
Unlike most existing embedding methods that are task-agnostic, we simultaneously solve for the underlying node representations and the optimal clustering assignments in an end-to-end manner.
Social and Information Networks Physics and Society
mvn2vec: Preservation and Collaboration in Multi-View Network Embedding
With experiments on a series of synthetic datasets, a large-scale internal Snapchat dataset, and two public datasets, we confirm the validity and importance of preservation and collaboration as two objectives for multi-view network embedding.
Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning
Motivation: State-of-the-art biomedical named entity recognition (BioNER) systems often require handcrafted features specific to each entity type, such as genes, chemicals and diseases.
Investigating Rumor News Using Agreement-Aware Search
We build Maester based on the following two key observations: (1) relatedness can commonly be determined by keywords and entities occurring in both questions and articles, and (2) the level of agreement between the investigative question and the related news article can often be decided by a few key sentences.
AspEm: Embedding Learning by Aspects in Heterogeneous Information Networks
Therefore, we are motivated to propose a novel embedding learning framework---AspEm---to preserve the semantic information in HINs based on multiple aspects.
Expert Finding in Heterogeneous Bibliographic Networks with Locally-trained Embeddings
We study the task of expert finding in heterogeneous bibliographical networks based on two aspects: textual content analysis and authority ranking.
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling
Many efforts have been made to facilitate natural language processing tasks with pre-trained language models (LMs), and brought significant improvements to various applications.
Ranked #47 on
Named Entity Recognition (NER)
on CoNLL 2003 (English)
Integrating Local Context and Global Cohesiveness for Open Information Extraction
However, current Open IE systems focus on modeling local context information in a sentence to extract relation tuples, while ignoring the fact that global statistics in a large corpus can be collectively leveraged to identify high-quality sentence-level extractions.
Entity Set Search of Scientific Literature: An Unsupervised Ranking Approach
Different from Web or general domain search, a large portion of queries in scientific literature search are entity-set queries, that is, multiple entities of possibly different types.
End-to-End Reinforcement Learning for Automatic Taxonomy Induction
We present a novel end-to-end reinforcement learning approach to automatic taxonomy induction from a set of terms.
Entropy-Based Subword Mining with an Application to Word Embeddings
Recent literature has shown a wide variety of benefits to mapping traditional one-hot representations of words and phrases to lower-dimensional real-valued vectors known as word embeddings.
Easing Embedding Learning by Comprehensive Transcription of Heterogeneous Information Networks
To cope with the challenges in the comprehensive transcription of HINs, we propose the HEER algorithm, which embeds HINs via edge representations that are further coupled with properly-learned heterogeneous metrics.
Weakly-Supervised Neural Text Classification
Although many semi-supervised and weakly-supervised text classification models exist, they cannot be easily applied to deep neural models and meanwhile support limited supervision types.
Learning Named Entity Tagger using Domain-Specific Dictionary
Recent advances in deep neural models allow us to build reliable named entity recognition (NER) systems without handcrafting features.
End-to-End Hierarchical Text Classification with Label Assignment Policy
We present an end-to-end reinforcement learning approach to hierarchical text classification where documents are labeled by placing them at the right positions in a given hierarchy.
Mining Entity Synonyms with Efficient Neural Set Generation
Mining entity synonym sets (i. e., sets of terms referring to the same entity) is an important task for many entity-leveraging applications.
User-Guided Clustering in Heterogeneous Information Networks via Motif-Based Comprehensive Transcription
We therefore approach the problem of user-guided clustering in HINs with network motifs.
TaxoGen: Unsupervised Topic Taxonomy Construction by Adaptive Term Embedding and Clustering
Our method, TaxoGen, uses term embeddings and hierarchical clustering to construct a topic taxonomy in a recursive fashion.
Databases
Weakly-Supervised Hierarchical Text Classification
During the training process, our model features a hierarchical neural structure, which mimics the given hierarchy and is capable of determining the proper levels for documents with a blocking mechanism.
STFNets: Learning Sensing Signals from the Time-Frequency Perspective with Short-Time Fourier Neural Networks
IoT applications, however, often measure physical phenomena, where the underlying physics (such as inertia, wireless signal propagation, or the natural frequency of oscillation) are fundamentally a function of signal frequencies, offering better features in the frequency domain.
Biomedical Event Extraction based on Knowledge-driven Tree-LSTM
Event extraction for the biomedical domain is more challenging than that in the general news domain since it requires broader acquisition of domain-specific knowledge and deeper understanding of complex contexts.
Task-Guided Pair Embedding in Heterogeneous Network
In this paper, we propose a novel task-guided pair embedding framework in heterogeneous network, called TaPEm, that directly models the relationship between a pair of nodes that are related to a specific task (e. g., paper-author relationship in author identification).
Reliability-aware Dynamic Feature Composition for Name Tagging
We design a set of word frequency-based reliability signals to indicate the quality of each word embedding.
Constrained Sequence-to-sequence Semitic Root Extraction for Enriching Word Embeddings
In this paper, we tackle the problem of {``}root extraction{''} from words in the Semitic language family.
Arabic Named Entity Recognition: What Works and What's Next
This paper presents the winning solution to the Arabic Named Entity Recognition challenge run by Topcoder. com.
On the Variance of the Adaptive Learning Rate and Beyond
The learning rate warmup heuristic achieves remarkable success in stabilizing training, accelerating convergence and improving generalization for adaptive stochastic optimization algorithms like RMSprop and Adam.
Raw-to-End Name Entity Recognition in Social Media
Our model neither requires the conversion from character sequences to word sequences, nor assumes tokenizer can correctly detect all word boundaries.
Parsimonious Morpheme Segmentation with an Application to Enriching Word Embeddings
However, in many languages and specialized corpora, words are composed by concatenating semantically meaningful subword structures.
Discriminative Topic Mining via Category-Name Guided Text Embedding
We propose a new task, discriminative topic mining, which leverages a set of user-provided category names to mine discriminative topics from text corpora.
Hierarchical Text Classification with Reinforced Label Assignment
While existing hierarchical text classification (HTC) methods attempt to capture label hierarchies for model training, they either make local decisions regarding each label or completely ignore the hierarchy information during inference.
Ranked #1 on
Text Classification
on RCV1
(Macro F1 metric)
Facet-Aware Evaluation for Extractive Summarization
In this paper, we present a facet-aware evaluation setup for better assessment of the information coverage in extracted summaries.
CrossWeigh: Training Named Entity Tagger from Imperfect Annotations
Therefore, we manually correct these label mistakes and form a cleaner test set.
Ranked #3 on
Named Entity Recognition (NER)
on CoNLL++
(using extra training data)
Discovering Hypernymy in Text-Rich Heterogeneous Information Network by Exploiting Context Granularity
Extensive experiments on two large real-world datasets demonstrate the effectiveness of HyperMine and the utility of modeling context granularity.
I Know You'll Be Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social Application
Then we design a novel deep learning pipeline based on LSTM and attention to accurately predict user churn with very limited initial behavior data, by leveraging the correlations among users' multi-dimensional activities and the underlying user types.
Neural Embedding Propagation on Heterogeneous Networks
While generalizing LP as a simple instance, NEP is far more powerful in its natural awareness of different types of objects and links, and the ability to automatically capture their important interaction patterns.
Query-Specific Knowledge Summarization with Entity Evolutionary Networks
Given a query, unlike traditional IR that finds relevant documents or entities, in this work, we focus on retrieving both entities and their connections for insightful knowledge summarization.
Place Deduplication with Embeddings
Thanks to the advancing mobile location services, people nowadays can post about places to share visiting experience on-the-go.
Meta-Graph Based HIN Spectral Embedding: Methods, Analyses, and Insights
In this work, we propose to study the utility of different meta-graphs, as well as how to simultaneously leverage multiple meta-graphs for HIN embedding in an unsupervised manner.
FUSE: Multi-Faceted Set Expansion by Coherent Clustering of Skip-grams
In this paper, we study the task of multi-faceted set expansion, which aims to capture all semantic facets in the seed set and return multiple sets of entities, one for each semantic facet.
HiGitClass: Keyword-Driven Hierarchical Classification of GitHub Repositories
With the massive number of repositories available, there is a pressing need for topic-based search.
SetExpan: Corpus-Based Set Expansion via Context Feature Selection and Rank Ensemble
In this study, we propose a novel framework, SetExpan, which tackles this problem, with two techniques: (1) a context feature selection method that selects clean context features for calculating entity-entity distributional similarity, and (2) a ranking-based unsupervised ensemble method for expanding entity set based on denoised context features.
HiExpan: Task-Guided Taxonomy Construction by Hierarchical Tree Expansion
Taxonomies are of great value to many knowledge-rich applications.
Spherical Text Embedding
While text embeddings are typically learned in the Euclidean space, directional similarity is often more effective in tasks such as word similarity and document clustering, which creates a gap between the training stage and usage stage of text embedding.
Relation Learning on Social Networks with Multi-Modal Graph Edge Variational Autoencoders
While node semantics have been extensively explored in social networks, little research attention has been paid to profile edge semantics, i. e., social relations.
Mining News Events from Comparable News Corpora: A Multi-Attribute Proximity Network Modeling Approach
This proximity network captures the corpus-level co-occurence statistics for candidate event descriptors, event attributes, as well as their connections.
Unsupervised Attributed Multiplex Network Embedding
Even for those that consider the multiplexity of a network, they overlook node attributes, resort to node labels for training, and fail to model the global properties of a graph.
Inf-VAE: A Variational Autoencoder Framework to Integrate Homophily and Influence in Diffusion Prediction
Recent years have witnessed tremendous interest in understanding and predicting information spread on social media platforms such as Twitter, Facebook, etc.
cube2net: Efficient Query-Specific Network Construction with Data Cube Organization
With extensive experiments of two classic network mining tasks on different real-world large datasets, we show that our proposed cube2net pipeline is general, and much more effective and efficient in query-specific network construction, compared with other methods without the leverage of data cube or reinforcement learning.
TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network
Taxonomies consist of machine-interpretable semantics and provide valuable knowledge for many web applications.
Generating Representative Headlines for News Stories
In this work, we study the problem of generating representative headlines for news stories.
Guiding Corpus-based Set Expansion by Auxiliary Sets Generation and Co-Expansion
Given a small set of seed entities (e. g., ``USA'', ``Russia''), corpus-based set expansion is to induce an extensive set of entities which share the same semantic class (Country in this example) from a given corpus.
Comprehensive Named Entity Recognition on CORD-19 with Distant or Weak Supervision
We created this CORD-NER dataset with comprehensive named entity recognition (NER) on the COVID-19 Open Research Dataset Challenge (CORD-19) corpus (2020-03-13).
Heterogeneous Network Representation Learning: A Unified Framework with Survey and Benchmark
Since there has already been a broad body of HNE algorithms, as the first contribution of this work, we provide a generic paradigm for the systematic categorization and analysis over the merits of various existing HNE algorithms.
Understanding the Difficulty of Training Transformers
Transformers have proved effective in many NLP tasks.
Ranked #5 on
Machine Translation
on WMT2014 English-French
Automatic Textual Evidence Mining in COVID-19 Literature
We created this EVIDENCEMINER system for automatic textual evidence mining in COVID-19 literature.
Empower Entity Set Expansion via Language Model Probing
Existing set expansion methods bootstrap the seed entity set by adaptively selecting context features and extracting new entities.
Partially-Typed NER Datasets Integration: Connecting Practice to Theory
While typical named entity recognition (NER) models require the training set to be annotated with all target types, each available datasets may only cover a part of them.
Minimally Supervised Categorization of Text with Metadata
Then, based on the same generative process, we synthesize training samples to address the bottleneck of label scarcity.
Open-Domain Question Answering with Pre-Constructed Question Spaces
Our reader-retriever first uses an offline reader to read the corpus and generate collections of all answerable questions associated with their answers, and then uses an online retriever to respond to user queries by searching the pre-constructed question spaces for answers that are most likely to be asked in the given way.
Unsupervised Differentiable Multi-aspect Network Embedding
To capture the multiple aspects of each node, existing studies mainly rely on offline graph clustering performed prior to the actual embedding, which results in the cluster membership of each node (i. e., node aspect distribution) fixed throughout training of the embedding model.
Octet: Online Catalog Taxonomy Enrichment with Self-Supervision
We propose to distantly train a sequence labeling model for term extraction and employ graph neural networks (GNNs) to capture the taxonomy structure as well as the query-item-taxonomy interactions for term attachment.
AutoKnow: Self-Driving Knowledge Collection for Products of Thousands of Types
Can one build a knowledge graph (KG) for all products in the world?
COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation
To combat COVID-19, both clinicians and scientists need to digest vast amounts of relevant biomedical knowledge in scientific literature to understand the disease mechanism and related biological functions.
EVIDENCEMINER: Textual Evidence Discovery for Life Sciences
EVIDENCEMINER is constructed in a completely automated way without any human effort for training data annotation.
GCN for HIN via Implicit Utilization of Attention and Meta-paths
Graph neural networks for HIN embeddings typically adopt a hierarchical attention (including node-level and meta-path-level attentions) to capture the information from meta-path-based neighbors.
Hierarchical Topic Mining via Joint Spherical Tree and Text Embedding
Mining a set of meaningful topics organized into a hierarchy is intuitively appealing since topic correlations are ubiquitous in massive text corpora.
Ranked #1 on
Topic Models
on Arxiv HEP-TH citation graph
Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization
Graph neural networks (GNNs) have achieved superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs.
Generation-Augmented Retrieval for Open-domain Question Answering
We demonstrate that the generated contexts substantially enrich the semantics of the queries and GAR with sparse representations (BM25) achieves comparable or better performance than state-of-the-art dense retrieval methods such as DPR.
Ranked #9 on
Passage Retrieval
on Natural Questions
SynSetExpan: An Iterative Framework for Joint Entity Set Expansion and Synonym Discovery
To facilitate the research on studying the interplays of these two tasks, we create the first large-scale Synonym-Enhanced Set Expansion (SE2) dataset via crowdsourcing.
Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning
Additionally, the explicit redundancy measure in MMR helps the neural representation of the summary to better capture redundancy.
Near-imperceptible Neural Linguistic Steganography via Self-Adjusting Arithmetic Coding
Linguistic steganography studies how to hide secret messages in natural language cover texts.
A Spherical Hidden Markov Model for Semantics-Rich Human Mobility Modeling
We propose SHMM, a multi-modal spherical hidden Markov model for semantics-rich human mobility modeling.
Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding
Aspect-based sentiment analysis of review texts is of great value for understanding user feedback in a fine-grained manner.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+2
CoRel: Seed-Guided Topical Taxonomy Construction by Concept Learning and Relation Transferring
Taxonomy is not only a fundamental form of knowledge representation, but also crucial to vast knowledge-rich applications, such as question answering and web search.
Text Classification Using Label Names Only: A Language Model Self-Training Approach
In this paper, we explore the potential of only using the label name of each class to train classification models on unlabeled data, without using any labeled documents.
CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding
To verify the effectiveness of the proposed framework, we apply CoDA to Transformer-based models on a wide range of natural language understanding tasks.
On the Transformer Growth for Progressive BERT Training
Due to the excessive cost of large-scale language model pre-training, considerable efforts have been made to train BERT progressively -- start from an inferior but low-cost model and gradually grow the model to increase the computational complexity.
BiTe-GCN: A New GCN Architecture via BidirectionalConvolution of Topology and Features on Text-Rich Networks
We propose BiTe-GCN, a novel GCN architecture with bidirectional convolution of both topology and features on text-rich networks to solve these limitations.
Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation
Despite significant progress, state-of-the-art abstractive summarization methods are still prone to hallucinate content inconsistent with the source document.
Hierarchical Metadata-Aware Document Categorization under Weak Supervision
Our experiments demonstrate a consistent improvement of HiMeCat over competitive baselines and validate the contribution of our representation learning and data augmentation modules.
Few-Shot Named Entity Recognition: A Comprehensive Study
This paper presents a comprehensive study to efficiently build named entity recognition (NER) systems when a small number of in-domain labeled data is available.
Rider: Reader-Guided Passage Reranking for Open-Domain Question Answering
Current open-domain question answering systems often follow a Retriever-Reader architecture, where the retriever first retrieves relevant passages and the reader then reads the retrieved passages to form an answer.
Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup
Contrastive learning has been applied successfully to learn vector representations of text.
MATCH: Metadata-Aware Text Classification in A Large Hierarchy
Multi-label text classification refers to the problem of assigning each given document its most relevant labels from the label set.
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining
The first token-level task, Corrective Language Modeling, is to detect and correct tokens replaced by the auxiliary model, in order to better capture token-level semantics.
Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks
Specifically, we jointly train two modules with different inductive biases -- a text analysis module for text understanding and a network learning module for class-discriminative, scalable network learning.
Toward Tweet Entity Linking with Heterogeneous Information Networks
The task of linking an entity mention in a tweet with its corresponding entity in a heterogeneous information network is of great importance, for the purpose of enriching heterogeneous information networks with the abundant and fresh knowledge embedded in tweets.
Who Should Go First? A Self-Supervised Concept Sorting Model for Improving Taxonomy Expansion
Taxonomies have been widely used in various machine learning and text mining systems to organize knowledge and facilitate downstream tasks.
The Future is not One-dimensional: Complex Event Schema Induction by Graph Modeling for Event Prediction
We introduce a new concept of Temporal Complex Event Schema: a graph-based schema representation that encompasses events, arguments, temporal connections and argument relations.
Document-Level Event Argument Extraction by Conditional Generation
On the task of argument extraction, we achieve an absolute gain of 7. 6% F1 and 5. 7% F1 over the next best model on the RAMS and WikiEvents datasets respectively.
Document-level Event Extraction
Event Argument Extraction
+2
Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation
In this paper, we present a systematic analysis that studies whether current seq2seq models, especially pre-trained language models, are good enough for preserving important input concepts and to what extent explicitly guiding generation with the concepts as lexical constraints is beneficial.
UCPhrase: Unsupervised Context-aware Quality Phrase Tagging
Training a conventional neural tagger based on silver labels usually faces the risk of overfitting phrase surface names.
Ranked #1 on
Phrase Tagging
on KPTimes
Training ELECTRA Augmented with Multi-word Selection
In this study, we present a new text encoder pre-training method that improves ELECTRA based on multi-task learning.
RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System
We present a new information extraction system that can automatically construct temporal event graphs from a collection of news documents from multiple sources, multiple languages (English and Spanish for our experiment), and multiple data modalities (speech, text, image and video).
Event Time Extraction and Propagation via Graph Attention Networks
Grounding events into a precise timeline is important for natural language understanding but has received limited attention in recent work.
TaxoClass: Hierarchical Multi-Label Text Classification Using Only Class Names
Hierarchical multi-label text classification (HMTC) aims to tag each document with a set of classes from a taxonomic class hierarchy.
Multi Label Text Classification
Multi-Label Text Classification
+3
Eider: Empowering Document-level Relation Extraction with Efficient Evidence Extraction and Inference-stage Fusion
Typical DocRE methods blindly take the full document as input, while a subset of the sentences in the document, noted as the evidence, are often sufficient for humans to predict the relation of an entity pair.
Ranked #5 on
Relation Extraction
on DocRED
Multi-head or Single-head? An Empirical Comparison for Transformer Training
Multi-head attention plays a crucial role in the recent success of Transformer models, which leads to consistent performance improvements over conventional attention in various applications.
Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data
In this work we present a method, Shift-Robust GNN (SR-GNN), designed to account for distributional differences between biased training data and the graph's true inference distribution.
Corpus-based Open-Domain Event Type Induction
As events of the same type could be expressed in multiple ways, we propose to represent each event type as a cluster of <predicate sense, object head> pairs.
Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training
We study the problem of training named entity recognition (NER) models using only distantly-labeled data, which can be automatically obtained by matching entity mentions in the raw text with entity types in a knowledge base.
Chemical-Reaction-Aware Molecule Representation Learning
Molecule representation learning (MRL) methods aim to embed molecules into a real vector space.
SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction
Consequently, it is more challenging to encode the key information sources--relevant contexts and entity types.
Ranked #1 on
Relation Extraction
on CDR
Entity Linking Meets Deep Learning: Techniques and Solutions
Entity linking (EL) is the process of linking entity mentions appearing in web text with their corresponding entities in a knowledge base.
UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning
Recent parameter-efficient language model tuning (PELT) methods manage to match the performance of fine-tuning with much fewer trainable parameters and perform especially well when training data is limited.
Fine-Grained Opinion Summarization with Minimal Supervision
Opinion summarization aims to profile a target by extracting opinions from multiple documents.
MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information
We study the problem of weakly supervised text classification, which aims to classify text documents into a set of pre-defined categories with category surface names only and without any annotated training document provided.
Out-of-Category Document Identification Using Target-Category Names as Weak Supervision
To address this challenge, we introduce a new task referred to as out-of-category detection, which aims to distinguish the documents according to their semantic relevance to the inlier (or target) categories by using the category names as weak supervision.
Universal Graph Convolutional Networks
So can we reasonably utilize these segmentation rules to design a universal propagation mechanism independent of the network structural assumption?
TaxoCom: Topic Taxonomy Completion with Hierarchical Discovery of Novel Topic Clusters
Topic taxonomies, which represent the latent topic (or category) structure of document collections, provide valuable knowledge of contents in many applications such as web search and information filtering.
Unsupervised Multi-Granularity Summarization
In this paper, we propose the first unsupervised multi-granularity summarization framework, GranuSum.
Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations
Interestingly, there have not been standard approaches to deploy PLMs for topic discovery as better alternatives to topic models.
Generating Training Data with Language Models: Towards Zero-Shot Language Understanding
Pretrained language models (PLMs) have demonstrated remarkable performance in various natural language processing tasks: Unidirectional PLMs (e. g., GPT) are well known for their superior text generation capabilities; bidirectional PLMs (e. g., BERT) have been the prominent choice for natural language understanding (NLU) tasks.
Ranked #5 on
Zero-Shot Text Classification
on AG News
TaxoEnrich: Self-Supervised Taxonomy Completion via Structure-Semantic Representations
Taxonomies are fundamental to many real-world applications in various domains, serving as structural representations of knowledge.
Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification
Large-scale multi-label text classification (LMTC) aims to associate a document with its relevant labels from a large candidate set.
P4E: Few-Shot Event Detection as Prompt-Guided Identification and Localization
Our framework decomposes event detection into an identification task and a localization task.
Shift-Robust Node Classification via Graph Adversarial Clustering
Then a shift-robust classifier is optimized on training graph and adversarial samples on target graph, which are generated by cluster GNN.
Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators
We present a new framework AMOS that pretrains text encoders with an Adversarial learning curriculum via a Mixture Of Signals from multiple auxiliary generators.
OA-Mine: Open-World Attribute Mining for E-Commerce Products with Weak Supervision
Most prior works on this matter mine new values for a set of known attributes but cannot handle new attributes that arose from constantly changing data.
Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds
Discovering latent topics from text corpora has been studied for decades.
CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision
Scientific extreme summarization (TLDR) aims to form ultra-short summaries of scientific papers.
Ranked #1 on
Extreme Summarization
on CiteSum
Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networks
In heterogeneous text-rich networks, this task is more challenging due to (1) presence or absence of text: Some nodes are associated with rich textual information, while others are not; (2) diversity of types: Nodes and edges of multiple types form a heterogeneous network structure.
All Birds with One Stone: Multi-task Text Classification for Efficient Inference with One Forward Pass
Multi-Task Learning (MTL) models have shown their robustness, effectiveness, and efficiency for transferring learned knowledge across tasks.
Schema-Guided Event Graph Completion
Existing link prediction or graph completion methods have difficulty dealing with event graphs because they are usually designed for a single large graph such as a social network or a knowledge graph, rather than multiple small dynamic event graphs.
Unsupervised Key Event Detection from Massive Text Corpora
Automated event detection from news corpora is a crucial task towards mining fast-evolving structured knowledge.
TeKo: Text-Rich Graph Neural Networks with External Knowledge
Graph Neural Networks (GNNs) have gained great popularity in tackling various analytical tasks on graph-structured data (i. e., networks).
Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation and Instance Generation
We study the problem of few-shot Fine-grained Entity Typing (FET), where only a few annotated entity mentions with contexts are given for each entity type.
MentorGNN: Deriving Curriculum for Pre-Training GNNs
To comprehend heterogeneous graph signals at different granularities, we propose a curriculum learning paradigm that automatically re-weighs graph signals in order to ensure a good generalization in the target domain.
TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter
Most existing PLMs are not tailored to the noisy user-generated text on social media, and the pre-training does not factor in the valuable social engagement logs available in a social network.
Few-shot Text Classification with Dual Contrastive Consistency
In this paper, we explore how to utilize pre-trained language model to perform few-shot text classification where only a few annotated examples are given for each class.
Towards a Unified Multi-Dimensional Evaluator for Text Generation
We re-frame NLG evaluation as a Boolean Question Answering (QA) task, and by guiding the model with different questions, we can use one evaluator to evaluate from multiple dimensions.
Topic Taxonomy Expansion via Hierarchy-Aware Topic Phrase Generation
Topic taxonomies display hierarchical topic structures of a text corpus and provide topical knowledge to enhance various NLP applications.
Large Language Models Can Self-Improve
We show that our approach improves the general reasoning ability of a 540B-parameter LLM (74. 4%->82. 1% on GSM8K, 78. 2%->83. 0% on DROP, 90. 0%->94. 4% on OpenBookQA, and 63. 4%->67. 9% on ANLI-A3) and achieves state-of-the-art-level performance, without any ground truth label.
Ranked #1 on
Question Answering
on DROP
PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion
For instance, we outperform the fully finetuning approaches on a KG completion benchmark by tuning only 1% of the parameters.
Ranked #5 on
Link Prediction
on UMLS
Open-Vocabulary Argument Role Prediction for Event Extraction
Specifically, we formulate the role prediction problem as an in-filling task and construct prompts for a pre-trained language model to generate candidate roles.
Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning
In this work, we study few-shot learning with PLMs from a different perspective: We first tune an autoregressive PLM on the few-shot samples and then use it as a generator to synthesize a large amount of novel training samples which augment the original training set.
Open Relation and Event Type Discovery with Type Abstraction
To tackle this problem, we introduce the idea of type abstraction, where the model is prompted to generalize and name the type.
Entity Set Co-Expansion in StackOverflow
Given a few seed entities of a certain type (e. g., Software or Programming Language), entity set expansion aims to discover an extensive set of entities that share the same type as the seeds.
Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts
Instead of mining coherent topics from a given text corpus in a completely unsupervised manner, seed-guided topic discovery methods leverage user-provided seed words to extract distinctive and coherent topics so that the mined topics can better cater to the user's interest.
Representation Deficiency in Masked Language Modeling
In this work, we offer a new perspective on the consequence of such a discrepancy: We demonstrate empirically and theoretically that MLM pretraining allocates some model dimensions exclusively for representing $\texttt{[MASK]}$ tokens, resulting in a representation deficiency for real tokens and limiting the pretrained model's expressiveness when it is adapted to downstream data without $\texttt{[MASK]}$ tokens.
Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memories
In this paper we improve the zero-shot generalization ability of language models via Mixture-Of-Memory Augmentation (MoMA), a mechanism that retrieves augmentation documents from multiple information corpora ("external memories"), with the option to "plug in" new memory at inference time.
The Effect of Metadata on Scientific Literature Tagging: A Cross-Field Cross-Model Study
Due to the exponential growth of scientific publications on the Web, there is a pressing need to tag each paper with fine-grained topics so that researchers can track their interested fields of study rather than drowning in the whole literature.
PDSum: Prototype-driven Continuous Summarization of Evolving Multi-document Sets Stream
Summarizing text-rich documents has been long studied in the literature, but most of the existing efforts have been made to summarize a static and predefined multi-document set.
Edgeformers: Graph-Empowered Transformers for Representation Learning on Textual-Edge Networks
Edges in many real-world social/information networks are associated with rich text information (e. g., user-user communications or user-product reviews).
GLEN: General-Purpose Event Detection for Thousands of Types
Finally, we perform error analysis and show that label noise is still the largest challenge for improving performance for this new dataset.
MEGClass: Extremely Weakly Supervised Text Classification via Mutually-Enhancing Text Granularities
By preserving the heterogeneity of potential classes, MEGClass can select the most informative class-indicative documents as iterative feedback to enhance the initial word-based class representations and ultimately fine-tune a pre-trained text classifier.
Unsupervised Story Discovery from Continuous News Streams via Scalable Thematic Embedding
Unsupervised discovery of stories with correlated news articles in real-time helps people digest massive news streams without expensive human annotations.
Patton: Language Model Pretraining on Text-Rich Networks
A real-world text corpus sometimes comprises not only text documents but also semantic links between them (e. g., academic papers in a bibliographic network are linked by citations and co-authorships).
OntoType: Ontology-Guided Zero-Shot Fine-Grained Entity Typing with Weak Supervision from Pre-Trained Language Models
In this study, we vision that an ontology provides a semantics-rich, hierarchical structure, which will help select the best results generated by multiple PLM models and head words.
PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training
Weakly-supervised text classification trains a classifier using the label name of each target class as the only supervision, which largely reduces human annotation efforts.
Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation
Instruction tuning has emerged to enhance the capabilities of large language models (LLMs) to comprehend instructions and generate appropriate responses.
Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Models
The mission of open knowledge graph (KG) completion is to draw new findings from known facts.
Explaining and Adapting Graph Conditional Shift
Graph Neural Networks (GNNs) have shown remarkable performance on graph-structured data.
Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond
Firstly, to offer systematic evaluations, we select fifteen typical logical reasoning datasets and organize them into deductive, inductive, abductive and mixed-form reasoning settings.
Weakly Supervised Multi-Label Classification of Full-Text Scientific Papers
Instead of relying on human-annotated training samples to build a classifier, weakly supervised scientific paper classification aims to classify papers only using category descriptions (e. g., category names, category-indicative keywords).
ReactIE: Enhancing Chemical Reaction Extraction with Weak Supervision
Structured chemical reaction information plays a vital role for chemists engaged in laboratory work and advanced endeavors such as computer-aided drug design.
Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification
Event schemas are a form of world knowledge about the typical progression of events.
Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
In this study, we introduce adaptive KV cache compression, a plug-and-play method that reduces the memory footprint of generative inference for Large Language Models (LLMs).
Learning Multiplex Embeddings on Text-rich Networks with One Text Encoder
Mainstream text representation learning methods use pretrained language models (PLMs) to generate one embedding for each text unit, expecting that all types of relations between texts can be captured by these single-view embeddings.
Language Models As Semantic Indexers
Semantic identifier (ID) is an important concept in information retrieval that aims to preserve the semantics of objects such as documents and items inside their IDs.
Ontology Enrichment for Effective Fine-grained Entity Typing
In this study, we propose OnEFET, where we (1) enrich each node in the ontology structure with two types of extra information: instance information for training sample augmentation and topic information to relate types to contexts, and (2) develop a coarse-to-fine typing algorithm that exploits the enriched information by training an entailment model with contrasting topics and instance-based augmented training samples.
Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
In this paper, we seek to empirically investigate knowledge transfer from larger to smaller models through a parametric perspective.
The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions
Recent progress in Large Language Models (LLMs) has produced models that exhibit remarkable performance across a variety of NLP tasks.
"Why Should I Review This Paper?" Unifying Semantic, Topic, and Citation Factors for Paper-Reviewer Matching
As many academic conferences are overwhelmed by a rapidly increasing number of paper submissions, automatically finding appropriate reviewers for each submission becomes a more urgent need than ever.
