Search Results for author:
Found 57 papers, 42 papers with code
Do Pre-trained Models Benefit Knowledge Graph Completion? A Reliable Evaluation and a Reasonable Approach
In recent years, pre-trained language models (PLMs) have been shown to capture factual knowledge from massive texts, which encourages the proposal of PLM-based knowledge graph completion (KGC) models.
CodRED: A Cross-Document Relation Extraction Dataset for Acquiring Knowledge in the Wild
Existing relation extraction (RE) methods typically focus on extracting relational facts between entity pairs within single sentences or documents.
CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation
Existing reference-free metrics have obvious limitations for evaluating controlled text generation models.
A Roadmap for Big Model
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
ELLE: Efficient Lifelong Pre-training for Emerging Data
We experiment ELLE with streaming data from 5 domains on BERT and GPT.
A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models
Pre-trained language models (PLMs) cannot well recall rich factual knowledge of entities exhibited in large-scale corpora, especially those rare entities.
Model Uncertainty-Aware Knowledge Amalgamation for Pre-Trained Language Models
As many fine-tuned pre-trained language models~(PLMs) with promising performance are generously released, investigating better ways to reuse these models is vital as it can greatly reduce the retraining computational cost and the potential environmental side-effects.
On Transferability of Prompt Tuning for Natural Language Processing
To explore whether we can improve PT via prompt transfer, we empirically investigate the transferability of soft prompts across different downstream tasks and PLMs in this work.
Natural Language Processing
Natural Language Understanding
+2
Exploring Universal Intrinsic Task Subspace via Prompt Tuning
In the experiments, we study diverse few-shot NLP tasks and surprisingly find that in a 250-dimensional subspace found with 100 tasks, by only tuning 250 free parameters, we can recover 97% and 83% of the full prompt tuning performance for 100 seen tasks (using different training data) and 20 unseen tasks, respectively, showing great generalization ability of the found intrinsic task subspace.
RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models
Motivated by this observation, we construct a word-based robustness-aware perturbation to distinguish poisoned samples from clean samples to defend against the backdoor attacks on natural language processing (NLP) models.
Topology-Imbalance Learning for Semi-Supervised Node Classification
The class imbalance problem, as an important issue in learning node representations, has drawn increasing attention from the community.
MoEfication: Transformer Feed-forward Layers are Mixtures of Experts
In this work, we study the computational patterns of FFNs and observe that most inputs only activate a tiny ratio of neurons of FFNs.
Dynamic Knowledge Distillation for Pre-trained Language Models
Knowledge distillation~(KD) has been proved effective for compressing large-scale pre-trained language models.
Packed Levitated Marker for Entity and Relation Extraction
In particular, we propose a neighborhood-oriented packing strategy, which considers the neighbor spans integrally to better model the entity boundary information.
Ranked #1 on
Named Entity Recognition
on Few-NERD (SUP)
Joint Entity and Relation Extraction
Named Entity Recognition
On Length Divergence Bias in Textual Matching Models
To determine whether TM models have adopted such heuristic, we introduce an adversarial evaluation scheme which invalidates the heuristic.
Rethinking Stealthiness of Backdoor Attack against NLP Models
In this work, we point out a potential problem of current backdoor attacking research: its evaluation ignores the stealthiness of backdoor attacks, and most of existing backdoor attacking methods are not stealthy either to system deployers or to system users.
Fully Hyperbolic Neural Networks
Hyperbolic neural networks have shown great potential for modeling complex data.
CLEVE: Contrastive Pre-training for Event Extraction
Event extraction (EE) has considerably benefited from pre-trained language models (PLMs) by fine-tuning.
Knowledge Inheritance for Pre-trained Language Models
Specifically, we introduce a pre-training framework named "knowledge inheritance" (KI) and explore how could knowledge distillation serve as auxiliary supervision during pre-training to efficiently learn larger PLMs.
TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference
To address this issue, we propose a dynamic token reduction approach to accelerate PLMs' inference, named TR-BERT, which could flexibly adapt the layer number of each token in inference to avoid redundant calculation.
Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction
Distantly supervised (DS) relation extraction (RE) has attracted much attention in the past few years as it can utilize large-scale auto-labeled data.
CSS-LM: A Contrastive Framework for Semi-supervised Fine-tuning of Pre-trained Language Models
We then perform contrastive semi-supervised learning on both the retrieved unlabeled and original labeled instances to help PLMs capture crucial task-related semantic features.
Representation Learning for Natural Language Processing
This book aims to review and present the recent advances of distributed representation learning for NLP, including why representation learning can improve NLP, how representation learning takes part in various important topics of NLP, and what challenges are still not well addressed by distributed representation.
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning
Pre-trained Language Models (PLMs) have shown superior performance on various downstream Natural Language Processing (NLP) tasks.
CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade
On the other hand, the exiting decisions made by internal classifiers are unreliable, leading to wrongly emitted early predictions.
Rethinking the Promotion Brought by Contrastive Learning to Semi-Supervised Node Classification
Graph Contrastive Learning (GCL) has proven highly effective in promoting the performance of Semi-Supervised Node Classification (SSNC).
DisenE: Disentangling Knowledge Graph Embeddings
Knowledge graph embedding (KGE), aiming to embed entities and relations into low-dimensional vectors, has attracted wide attention recently.
Disentangle-based Continual Graph Representation Learning
Graph embedding (GE) methods embed nodes (and/or edges) in graph into a low-dimensional semantic space, and have shown its effectiveness in modeling multi-relational data.
Learning from Context or Names? An Empirical Study on Neural Relation Extraction
We find that (i) while context is the main source to support the predictions, RE models also heavily rely on the information from entity mentions, most of which is type information, and (ii) existing datasets may leak shallow heuristics via entity mentions and thus contribute to the high performance on RE benchmarks.
Ranked #17 on
Relation Extraction
on TACRED
CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models
In this paper, we propose a novel framework named Coke to dynamically select contextual knowledge and embed knowledge context according to textual context for PLMs, which can avoid the effect of redundant and ambiguous knowledge in KGs that cannot match the input text.
Continual Relation Learning via Episodic Memory Activation and Reconsolidation
Continual relation learning aims to continually train a model on new data to learn incessantly emerging novel relations while avoiding catastrophically forgetting old relations.
Learning to Recover from Multi-Modality Errors for Non-Autoregressive Neural Machine Translation
By dynamically determining segment length and deleting repetitive segments, RecoverSAT is capable of recovering from repetitive and missing token errors.
MAVEN: A Massive General Domain Event Detection Dataset
Most existing datasets exhibit the following issues that limit further development of ED: (1) Data scarcity.
Coreferential Reasoning Learning for Language Representation
Language representation models such as BERT could effectively capture contextual semantic information from plain text, and have been proved to achieve promising results in lots of downstream NLP tasks with appropriate fine-tuning.
Ranked #26 on
Relation Extraction
on DocRED
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction
Relational facts are an important component of human knowledge, which are hidden in vast amounts of text.
HighwayGraph: Modelling Long-distance Node Relations for Improving General Graph Neural Network
To address this issue, we propose to model long-distance node relations by simply relying on shallow GNN architectures with two solutions: (1) Implicitly modelling by learning to predict node pair relations (2) Explicitly modelling by adding edges between nodes that potentially have the same label.
Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information
Non-autoregressive neural machine translation (NAT) generates each target word in parallel and has achieved promising inference acceleration.
Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network
Multi-paragraph reasoning is indispensable for open-domain question answering (OpenQA), which receives less attention in the current OpenQA systems.
Ranked #60 on
Question Answering
on HotpotQA
NumNet: Machine Reading Comprehension with Numerical Reasoning
Numerical reasoning, such as addition, subtraction, sorting and counting is a critical skill in human's reading comprehension, which has not been well considered in existing machine reading comprehension (MRC) systems.
Ranked #7 on
Question Answering
on DROP Test
Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View
Graph Neural Networks (GNNs) have achieved promising performance on a wide range of graph-based tasks.
Ranked #48 on
Node Classification
on Cora
XQA: A Cross-lingual Open-domain Question Answering Dataset
Experimental results show that the multilingual BERT model achieves the best results in almost all target languages, while the performance of cross-lingual OpenQA is still much lower than that of English.
DocRED: A Large-Scale Document-Level Relation Extraction Dataset
Multiple entities in a document generally exhibit complex inter-sentence relations, and cannot be well handled by existing relation extraction (RE) methods that typically focus on extracting intra-sentence relations for single entity pairs.
Ranked #51 on
Relation Extraction
on DocRED
Fact Discovery from Knowledge Base via Facet Decomposition
We also propose a novel auto-encoder based facet component to estimate some facets of the fact.
Graph Neural Networks with Generated Parameters for Relation Extraction
Recently, progress has been made towards improving relational reasoning in machine learning field.
Knowledge Representation Learning: A Quantitative Review
Knowledge representation learning (KRL) aims to represent entities and relations in knowledge graph in low-dimensional semantic space, which have been widely used in massive knowledge-driven tasks.
DIAG-NRE: A Neural Pattern Diagnosis Framework for Distantly Supervised Neural Relation Extraction
To demonstrate the effectiveness of DIAG-NRE, we apply it to two real-world datasets and present both significant and interpretable improvements over state-of-the-art methods.
OpenKE: An Open Toolkit for Knowledge Embedding
We release an open toolkit for knowledge embedding (OpenKE), which provides a unified framework and various fundamental models to embed knowledge graphs into a continuous low-dimensional space.
Cross-lingual Lexical Sememe Prediction
We propose a novel framework to model correlations between sememes and multi-lingual words in low-dimensional semantic space for sememe prediction.
Denoise while Aggregating: Collaborative Learning in Open-Domain Question Answering
The open-domain question answering (OpenQA) task aims to extract answers that match specific questions from a distantly supervised corpus.
Adversarial Multi-lingual Neural Relation Extraction
To address these issues, we propose an adversarial multi-lingual neural relation extraction (AMNRE) model, which builds both consistent and individual representations for each sentence to consider the consistency and diversity among languages.
Denoising Distantly Supervised Open-Domain Question Answering
Distantly supervised open-domain question answering (DS-QA) aims to find answers in collections of unlabeled text.
Ranked #2 on
Open-Domain Question Answering
on Quasar
Neural Relation Extraction with Multi-lingual Attention
Relation extraction has been widely used for finding unknown relational facts from plain text.
Incorporating Relation Paths in Neural Relation Extraction
Distantly supervised relation extraction has been widely used to find novel relational facts from plain text.
Modeling Relation Paths for Representation Learning of Knowledge Bases
Representation learning of knowledge bases (KBs) aims to embed both entities and relations into a low-dimensional space.
Learning Entity and Relation Embeddings for Knowledge Graph Completion
Knowledge graph completion aims to perform link prediction between entities.
