Search Results for author:
Found 306 papers, 198 papers with code
Monte Carlo Methods for Maximum Margin Supervised Topic Models
An effective strategy to exploit the supervising side information for discovering predictive topic representations is to impose discriminative constraints induced by such information on the posterior distributions under a topic model.
Binary Tree based Chinese Word Segmentation
A binary tree based framework is also designed to overcome the granularity mismatch problem.
Reduce Meaningless Words for Joint Chinese Word Segmentation and Part-of-speech Tagging
Conventional statistics-based methods for joint Chinese word segmentation and part-of-speech tagging (S&T) have generalization ability to recognize new words that do not appear in the training data.
Contrastive Unsupervised Word Alignment with Non-Local Features
However, a major challenge still remains: it is intractable to calculate the expectations of non-local features that are critical for capturing the divergence between natural languages.
Learning Entity and Relation Embeddings for Knowledge Graph Completion
Knowledge graph completion aims to perform link prediction between entities.
Modeling Relation Paths for Representation Learning of Knowledge Bases
Representation learning of knowledge bases (KBs) aims to embed both entities and relations into a low-dimensional space.
Network Representation Learning with Rich Text Information
Representation learning has shown its effectiveness in many tasks such as image classification and text mining.
Minimum Risk Training for Neural Machine Translation
We propose minimum risk training for end-to-end neural machine translation.
Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation
The attentional mechanism has proven to be effective in improving end-to-end neural machine translation.
Generating Chinese Classical Poems with RNN Encoder-Decoder
We take the generation of Chinese classical poem lines as a sequence-to-sequence learning problem, and build a novel system based on the RNN Encoder-Decoder structure to generate quatrains (Jueju in Chinese), with a topic word as input.
Neural Headline Generation with Sentence-wise Optimization
Recently, neural models have been proposed for headline generation by learning to map documents to headlines with recurrent neural networks.
Agreement-based Learning of Parallel Lexicons and Phrases from Non-Parallel Corpora
We introduce an agreement-based approach to learning parallel lexicons and phrases from non-parallel corpora.
Semi-Supervised Learning for Neural Machine Translation
While end-to-end neural machine translation (NMT) has made remarkable progress recently, NMT systems only rely on parallel corpora for parameter estimation.
Topic Sensitive Neural Headline Generation
Neural models have recently been used in text summarization including headline generation.
Knowledge Representation via Joint Learning of Sequential Text and Knowledge Graphs
There are two main challenges for constructing knowledge representations from plain texts: (1) How to take full advantages of sequential contexts of entities in plain texts for KRL.
Image-embodied Knowledge Representation Learning
More specifically, we first construct representations for all images of an entity with a neural image encoder.
Incorporating Relation Paths in Neural Relation Extraction
Distantly supervised relation extraction has been widely used to find novel relational facts from plain text.
Joint Representation Learning of Text and Knowledge for Knowledge Graph Completion
Joint representation learning of text and knowledge within a unified semantic space enables us to perform knowledge graph completion more accurately.
Neural Machine Translation with Pivot Languages
While recent neural machine translation approaches have delivered state-of-the-art performance for resource-rich language pairs, they suffer from the data scarcity problem for resource-scarce language pairs.
A Unified Framework for Community Detection and Network Representation Learning
Network representation learning (NRL) aims to learn low-dimensional vectors for vertices in a network.
Social and Information Networks Physics and Society
Inducing Bilingual Lexica From Non-Parallel Data With Earth Mover's Distance Regularization
Being able to induce word translations from non-parallel data is often a prerequisite for cross-lingual processing in resource-scarce languages and domains.
Neural Emoji Recommendation in Dialogue Systems
It indicates that our method could well capture the contextual information and emotion flow in dialogues, which is significant for emoji recommendation.
Joint POS Tagging and Dependency Parsing with Transition-based Neural Networks
While part-of-speech (POS) tagging and dependency parsing are observed to be closely related, existing work on joint modeling with manually crafted feature templates suffers from the feature sparsity and incompleteness problems.
THUMT: An Open Source Toolkit for Neural Machine Translation
This paper introduces THUMT, an open-source toolkit for neural machine translation (NMT) developed by the Natural Language Processing Group at Tsinghua University.
Iterative Entity Alignment via Joint Knowledge Embeddings
During this process, we can align entities according to their semantic distance in this joint semantic space.
CANE: Context-Aware Network Embedding for Relation Modeling
Network embedding (NE) is playing a critical role in network analysis, due to its ability to represent vertices with efficient low-dimensional embedding vectors.
Visualizing and Understanding Neural Machine Translation
While neural machine translation (NMT) has made remarkable progress in recent years, it is hard to interpret its internal workings due to the continuous representations and non-linearity of neural networks.
Improved Word Representation Learning with Sememes
The key idea is to utilize word sememes to capture exact meanings of a word within specific contexts accurately.
Neural Relation Extraction with Multi-lingual Attention
Relation extraction has been widely used for finding unknown relational facts from plain text.
Adversarial Training for Unsupervised Bilingual Lexicon Induction
In this work, we show that such cross-lingual connection can actually be established without any form of supervision.
Domain-Specific New Words Detection in Chinese
With the explosive growth of Internet, more and more domain-specific environments appear, such as forums, blogs, MOOCs and etc.
Earth Mover's Distance Minimization for Unsupervised Bilingual Lexicon Induction
By viewing word embedding spaces as distributions, we propose to minimize their earth mover{'}s distance, a measure of divergence between distributions.
Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval
This paper presents the Entity-Duet Neural Ranking Model (EDRM), which introduces knowledge graphs to neural search systems.
Denoising Distant Supervision for Relation Extraction via Instance-Level Adversarial Training
As shown in the experiments on a large-scale benchmark dataset in relation extraction, our denoising method can effectively filter out noisy instances and achieve significant improvements as compared with the state-of-the-art models.
Incorporating Chinese Characters of Words for Lexical Sememe Prediction
However, existing methods of lexical sememe prediction typically rely on the external context of words to represent the meaning, which usually fails to deal with low-frequency and out-of-vocabulary words.
Denoising Distantly Supervised Open-Domain Question Answering
Distantly supervised open-domain question answering (DS-QA) aims to find answers in collections of unlabeled text.
Ranked #2 on
Open-Domain Question Answering
on Quasar
CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction
In this paper, we introduce the \textbf{C}hinese \textbf{AI} and \textbf{L}aw challenge dataset (CAIL2018), the first large-scale Chinese legal dataset for judgment prediction.
Adversarial Multi-lingual Neural Relation Extraction
To address these issues, we propose an adversarial multi-lingual neural relation extraction (AMNRE) model, which builds both consistent and individual representations for each sentence to consider the consistency and diversity among languages.
Few-Shot Charge Prediction with Discriminative Legal Attributes
Specifically, our model outperforms other baselines by more than 50{\%} in the few-shot scenario.
Chinese Poetry Generation with a Salient-Clue Mechanism
As a precious part of the human cultural heritage, Chinese poetry has influenced people for generations.
Chinese Poetry Generation with a Working Memory Model
Different from previous methods, our model explicitly maintains topics and informative limited history in a neural memory.
Automatic Judgment Prediction via Legal Reading Comprehension
It has been studied for several decades mainly by lawyers and judges, considered as a novel and prospective application of artificial intelligence techniques in the legal field.
Denoise while Aggregating: Collaborative Learning in Open-Domain Question Answering
The open-domain question answering (OpenQA) task aims to extract answers that match specific questions from a distantly supervised corpus.
Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention
In this paper, we aim to incorporate the hierarchical information of relations for distantly supervised relation extraction and propose a novel hierarchical attention scheme.
Automatic Poetry Generation with Mutual Reinforcement Learning
Human experts evaluate poetry in terms of some specific criteria, instead of word-level likelihood.
Stylistic Chinese Poetry Generation via Unsupervised Style Disentanglement
The ability to write diverse poems in different styles under the same poetic imagery is an important characteristic of human poetry writing.
Legal Judgment Prediction via Topological Learning
Legal Judgment Prediction (LJP) aims to predict the judgment result based on the facts of a case and becomes a promising application of artificial intelligence techniques in the legal field.
Cross-lingual Lexical Sememe Prediction
We propose a novel framework to model correlations between sememes and multi-lingual words in low-dimensional semantic space for sememe prediction.
Put It Back: Entity Typing with Language Model Enhancement
Entity typing aims to classify semantic types of an entity mention in a specific context.
A Multi-answer Multi-task Framework for Real-world Machine Reading Comprehension
The task of machine reading comprehension (MRC) has evolved from answering simple questions from well-edited text to answering real questions from users out of web data.
Improving the Transformer Translation Model with Document-Level Context
Although the Transformer translation model (Vaswani et al., 2017) has achieved state-of-the-art performance in a variety of translation tasks, how to use document-level context to deal with discourse phenomena problematic for Transformer still remains a challenge.
Overview of CAIL2018: Legal Judgment Prediction Competition
In this paper, we give an overview of the Legal Judgment Prediction (LJP) competition at Chinese AI and Law challenge (CAIL2018).
Enhancing Stock Movement Prediction with Adversarial Training
The key novelty is that we propose to employ adversarial training to improve the generalization of a neural network prediction model.
FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation
The relation of each sentence is first recognized by distant supervision methods, and then filtered by crowdworkers.
Language Modeling with Sparse Product of Sememe Experts
Most language modeling methods rely on large-scale data to statistically learn the sequential patterns of words.
OpenKE: An Open Toolkit for Knowledge Embedding
We release an open toolkit for knowledge embedding (OpenKE), which provides a unified framework and various fundamental models to embed knowledge graphs into a continuous low-dimensional space.
Prior Knowledge Integration for Neural Machine Translation using Posterior Regularization
Although neural machine translation has made significant progress recently, how to integrate multiple overlapping, arbitrary prior knowledge sources remains a challenge.
CED: Credible Early Detection of Social Media Rumors
By regarding all reposts to a rumor candidate as a sequence, the proposed model will seek an early point-in-time for making a credible prediction.
Social and Information Networks
Bandit Learning with Implicit Feedback
Implicit feedback, such as user clicks, although abundant in online information service systems, does not provide substantial evidence on users' evaluation of system's output.
Graph Neural Networks: A Review of Methods and Applications
Lots of learning tasks require dealing with graph data which contains rich relation information among elements.
COSINE: Compressive Network Embedding on Large-scale Information Networks
There is recently a surge in approaches that learn low-dimensional embeddings of nodes in networks.
Neural Diffusion Model for Microscopic Cascade Prediction
The strong assumptions oversimplify the complex diffusion mechanism and prevent these models from better fitting real-world cascade data.
Social and Information Networks Physics and Society
Knowledge Representation Learning: A Quantitative Review
Knowledge representation learning (KRL) aims to represent entities and relations in knowledge graph in low-dimensional semantic space, which have been widely used in massive knowledge-driven tasks.
OpenHowNet: An Open Sememe-based Lexical Knowledge Base
In this paper, we present an open sememe-based lexical knowledge base OpenHowNet.
Graph Neural Networks with Generated Parameters for Relation Extraction
Recently, progress has been made towards improving relational reasoning in machine learning field.
ERNIE: Enhanced Language Representation with Informative Entities
Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP tasks.
Ranked #1 on
Entity Linking
on FIGER
COS960: A Chinese Word Similarity Dataset of 960 Word Pairs
Word similarity computation is a widely recognized task in the field of lexical semantics.
Adversarial Training for Weakly Supervised Event Detection
Modern weakly supervised methods for event detection (ED) avoid time-consuming human annotation and achieve promising results by learning from auto-labeled data.
DocRED: A Large-Scale Document-Level Relation Extraction Dataset
Multiple entities in a document generally exhibit complex inter-sentence relations, and cannot be well handled by existing relation extraction (RE) methods that typically focus on extracting intra-sentence relations for single entity pairs.
Ranked #59 on
Relation Extraction
on DocRED
XQA: A Cross-lingual Open-domain Question Answering Dataset
Experimental results show that the multilingual BERT model achieves the best results in almost all target languages, while the performance of cross-lingual OpenQA is still much lower than that of English.
Reducing Word Omission Errors in Neural Machine Translation: A Contrastive Learning Approach
While neural machine translation (NMT) has achieved remarkable success, NMT systems are prone to make word omission errors.
Jiuge: A Human-Machine Collaborative Chinese Classical Poetry Generation System
By exposing the options of poetry genres, styles and revision modes, Jiuge, acting as a professional assistant, allows constant and active participation of users in poetic creation.
Modeling Semantic Compositionality with Sememe Knowledge
In this paper, we verify the effectiveness of sememes, the minimum semantic units of human languages, in modeling SC by a confirmatory experiment.
multi-word expression embedding
multi-word expression sememe prediction
Quantifying Similarity between Relations with Fact Distribution
We introduce a conceptually simple and effective method to quantify the similarity between relations in knowledge bases.
GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification
Fact verification (FV) is a challenging task which requires to retrieve relevant evidence from plain text and use the evidence to verify given claims.
Ranked #7 on
Fact Verification
on FEVER
Explore Entity Embedding Effectiveness in Entity Retrieval
Entity embedding learns lots of semantic information from the knowledge graph and represents entities with a low-dimensional representation, which provides an opportunity to establish interactions between query related entities and candidate entities for entity retrieval.
Neural Snowball for Few-Shot Relation Learning
To address new relations with few-shot instances, we propose a novel bootstrapping approach, Neural Snowball, to learn new relations by transferring semantic knowledge about existing relations.
Improving Back-Translation with Uncertainty-based Confidence Estimation
While back-translation is simple and effective in exploiting abundant monolingual corpora to improve low-resource neural machine translation (NMT), the synthetic bilingual corpora generated by NMT models trained on limited authentic bilingual data are inevitably noisy.
Using BERT for Word Sense Disambiguation
Word Sense Disambiguation (WSD), which aims to identify the correct sense of a given polyseme, is a long-standing problem in NLP.
OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction
OpenNRE is an open-source and extensible toolkit that provides a unified framework to implement neural models for relation extraction (RE).
FewRel 2.0: Towards More Challenging Few-Shot Relation Classification
We present FewRel 2. 0, a more challenging task to investigate two aspects of few-shot relation classification models: (1) Can they adapt to a new domain with only a handful of instances?
Improving Sequence Modeling Ability of Recurrent Neural Networks via Sememes
Sememes, the minimum semantic units of human languages, have been successfully utilized in various natural language processing applications.
Fine-grained Fact Verification with Kernel Graph Attention Network
Fact Verification requires fine-grained natural language inference capability that finds subtle clues to identify the syntactical and semantically correct but not well-supported claims.
Ranked #5 on
Fact Verification
on FEVER
Word-level Textual Adversarial Attacking as Combinatorial Optimization
Also, further experiments show our model has higher transferability and can bring more robustness enhancement to victim models by adversarial training.
Open Relation Extraction: Relational Knowledge Transfer from Supervised Data to Unsupervised Data
Open relation extraction (OpenRE) aims to extract relational facts from the open-domain corpus.
HMEAE: Hierarchical Modular Event Argument Extraction
Existing event extraction methods classify each argument role independently, ignoring the conceptual correlations between different argument roles.
Adversarial Language Games for Advanced Natural Language Intelligence
In this work, we propose a challenging adversarial language game called Adversarial Taboo as an example, in which an attacker and a defender compete around a target word.
Multi-Paragraph Reasoning with Knowledge-enhanced Graph Neural Network
Multi-paragraph reasoning is indispensable for open-domain question answering (OpenQA), which receives less attention in the current OpenQA systems.
Ranked #58 on
Question Answering
on HotpotQA
Learning to Copy for Automatic Post-Editing
To better identify translation errors, our method learns the representations of source sentences and system outputs in an interactive way.
CAIL2019-SCM: A Dataset of Similar Case Matching in Legal Domain
In this paper, we introduce CAIL2019-SCM, Chinese AI and Law 2019 Similar Case Matching dataset.
Neural Machine Translation with Explicit Phrase Alignment
The lack of alignment in NMT models leads to three problems: it is hard to (1) interpret the translation process, (2) impose lexical constraints, and (3) impose structural constraints.
JEC-QA: A Legal-Domain Question Answering Dataset
We present JEC-QA, the largest question answering dataset in the legal domain, collected from the National Judicial Examination of China.
Towards Building a Multilingual Sememe Knowledge Base: Predicting Sememes for BabelNet Synsets
We first build a dataset serving as the seed of the multilingual sememe KB.
Learning to Predict Explainable Plots for Neural Story Generation
While the use of neural networks has proven effective in improving story generation, how to learn to generate an explainable high-level plot still remains a major challenge.
Multi-channel Reverse Dictionary Model
A reverse dictionary takes the description of a target word as input and outputs the target word together with other words that match the description.
Lexical Sememe Prediction using Dictionary Definitions by Capturing Local Semantic Correspondence
We find that sememes of each word are usually semantically matched to different words in its dictionary definition, and we name this matching relationship local semantic correspondence.
Generating Major Types of Chinese Classical Poetry in a Uniformed Framework
In this paper, we propose a GPT-2 based uniformed framework for generating major types of Chinese classical poems.
MixPoet: Diverse Poetry Generation via Learning Controllable Mixed Latent Space
Though recent neural models make prominent progress in some criteria of poetry quality, generated poems still suffer from the problem of poor diversity.
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction
Relational facts are an important component of human knowledge, which are hidden in vast amounts of text.
Coreferential Reasoning Learning for Language Representation
Language representation models such as BERT could effectively capture contextual semantic information from plain text, and have been proved to achieve promising results in lots of downstream NLP tasks with appropriate fine-tuning.
Ranked #31 on
Relation Extraction
on DocRED
Train No Evil: Selective Masking for Task-Guided Pre-Training
In this stage, the model is trained by masked language modeling on in-domain unsupervised data to learn domain-specific patterns and we propose a novel selective masking strategy to learn task-specific patterns.
How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence
Legal Artificial Intelligence (LegalAI) focuses on applying the technology of artificial intelligence, especially natural language processing, to benefit tasks in the legal domain.
KACC: A Multi-task Benchmark for Knowledge Abstraction, Concretization and Completion
Based on the datasets, we propose novel tasks such as multi-hop knowledge abstraction (MKA), multi-hop knowledge concretization (MKC) and then design a comprehensive benchmark.
Fast Network Embedding Enhancement via High Order Proximity Approximation
Many Network Representation Learning (NRL) methods have been proposed to learn vector representations for vertices in a network recently.
Continual Relation Learning via Episodic Memory Activation and Reconsolidation
Continual relation learning aims to continually train a model on new data to learn incessantly emerging novel relations while avoiding catastrophically forgetting old relations.
Modeling Voting for System Combination in Machine Translation
System combination is an important technique for combining the hypotheses of different machine translation systems to improve translation performance.
Country Image in COVID-19 Pandemic: A Case Study of China
Therefore, in this study, we take China as a specific and typical case and investigate its image with aspect-based sentiment analysis on a large-scale Twitter dataset.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
Learning to Attack: Towards Textual Adversarial Attacking in Real-world Situations
Adversarial attacking aims to fool deep neural networks with adversarial examples.
OpenAttack: An Open-source Textual Adversarial Attack Toolkit
Textual adversarial attacking has received wide and increasing attention in recent years.
Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect
Recommender systems aim to provide item recommendations for users, and are usually faced with data sparsity problem (e. g., cold start) in real-world scenarios.
CokeBERT: Contextual Knowledge Selection and Embedding towards Enhanced Pre-Trained Language Models
In this paper, we propose a novel framework named Coke to dynamically select contextual knowledge and embed knowledge context according to textual context for PLMs, which can avoid the effect of redundant and ambiguous knowledge in KGs that cannot match the input text.
IsOBS: An Information System for Oracle Bone Script
Oracle bone script (OBS) is the earliest known ancient Chinese writing system and the ancestor of modern Chinese.
WantWords: An Open-source Online Reverse Dictionary System
A reverse dictionary takes descriptions of words as input and outputs words semantically matching the input descriptions.
Learning from Context or Names? An Empirical Study on Neural Relation Extraction
We find that (i) while context is the main source to support the predictions, RE models also heavily rely on the information from entity mentions, most of which is type information, and (ii) existing datasets may leak shallow heuristics via entity mentions and thus contribute to the high performance on RE benchmarks.
Ranked #23 on
Relation Extraction
on TACRED
Towards Interpretable Natural Language Understanding with Explanations as Latent Variables
In this paper, we develop a general framework for interpretable natural language understanding that requires only a small set of human annotated explanations for training.
Adapting Open Domain Fact Extraction and Verification to COVID-FACT through In-Domain Language Modeling
With the epidemic of COVID-19, verifying the scientifically false online information, such as fake news and maliciously fabricated statements, has become crucial.
Know What You Don't Need: Single-Shot Meta-Pruning for Attention Heads
To measure the informativeness of attention heads, we train our Single-Shot Meta-Pruner (SMP) with a meta-learning paradigm aiming to maintain the distribution of text representations after pruning.
Denoising Relation Extraction from Document-level Distant Supervision
Distant supervision (DS) has been widely used to generate auto-labeled data for sentence-level relation extraction (RE), which improves RE performance.
ONION: A Simple and Effective Defense Against Textual Backdoor Attacks
Nevertheless, there are few studies on defending against textual backdoor attacks.
CPM: A Large-scale Generative Chinese Pre-trained Language Model
However, applying GPT-3 to address Chinese NLP tasks is still challenging, as the training corpus of GPT-3 is primarily English, and the parameters are not publicly available.
Meta-Information Guided Meta-Learning for Few-Shot Relation Classification
Few-shot classification requires classifiers to adapt to new classes with only a few training instances.
Try to Substitute: An Unsupervised Chinese Word Sense Disambiguation Method Based on HowNet
In this paper, we propose a new unsupervised method for HowNet-based Chinese WSD, which exploits the masked language model task of pre-trained language models.
Mask-Align: Self-Supervised Neural Word Alignment
Word alignment, which aims to align translationally equivalent words between source and target sentences, plays an important role in many natural language processing tasks.
Towards a Universal Continuous Knowledge Base
In this work, we propose a method for building a continuous knowledge base (CKB) that can store knowledge imported from multiple, diverse neural networks.
ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning
Pre-trained Language Models (PLMs) have shown superior performance on various downstream Natural Language Processing (NLP) tasks.
Neural Machine Translation: A Review of Methods, Resources, and Tools
Machine translation (MT) is an important sub-field of natural language processing that aims to translate natural languages using computers.
Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
In this work, we propose a simple and effective method to cover a much larger proportion of the attack search space, called Adversarial and Mixup Data Augmentation (AMDA).
Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
In this work, we demonstrate the universal vulnerability of PTMs, where fine-tuned PTMs can be easily controlled by backdoor attacks in arbitrary downstream tasks.
OpenMatch: An Open Source Library for Neu-IR Research
OpenMatch is a Python-based library that serves for Neural Information Retrieval (Neu-IR) research.
CSS-LM: A Contrastive Framework for Semi-supervised Fine-tuning of Pre-trained Language Models
We then perform contrastive semi-supervised learning on both the retrieved unlabeled and original labeled instances to help PLMs capture crucial task-related semantic features.
Representation Learning for Natural Language Processing
This book aims to review and present the recent advances of distributed representation learning for NLP, including why representation learning can improve NLP, how representation learning takes part in various important topics of NLP, and what challenges are still not well addressed by distributed representation.
UPRec: User-Aware Pre-training for Recommender Systems
Existing sequential recommendation methods rely on large amounts of training data and usually suffer from the data sparsity problem.
Improving Diversity of Neural Text Generation via Inverse Probability Weighting
Traditional stochastic sampling methods only focus on truncating the unreliable "tail" of the distribution, and do not address the "head" part, which we show might contain tedious or even repetitive candidates with high probability that lead to repetition loops.
Optimal Embedding Calibration for Symbolic Music Similarity
In natural language processing (NLP), the semantic similarity task requires large-scale, high-quality human-annotated labels for fine-tuning or evaluation.
Equality before the Law: Legal Judgment Consistency Analysis for Fairness
We propose to simulate judges from different groups with legal judgment prediction (LJP) models and measure the judicial inconsistency with the disagreement of the judgment results given by LJP models trained on different groups.
Visual Distant Supervision for Scene Graph Generation
In this work, we propose visual distant supervision, a novel paradigm of visual relation learning, which can train scene graph models without any human-labeled data.
Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents
Legal artificial intelligence (LegalAI) aims to benefit legal systems with the technology of artificial intelligence, especially natural language processing (NLP).
Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction
Grammatical Error Correction (GEC) aims to correct writing errors and help language learners improve their writing skills.
Ranked #1 on
Grammatical Error Detection
on FCE
Dynamic Multi-Branch Layers for On-Device Neural Machine Translation
With the rapid development of artificial intelligence (AI), there is a trend in moving AI applications, such as neural machine translation (NMT), from cloud to mobile devices.
Manual Evaluation Matters: Reviewing Test Protocols of Distantly Supervised Relation Extraction
Distantly supervised (DS) relation extraction (RE) has attracted much attention in the past few years as it can utilize large-scale auto-labeled data.
PTR: Prompt Tuning with Rules for Text Classification
This indicates that PTR is a promising approach to take advantage of both human prior knowledge and PLMs for those complicated classification tasks.
TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference
To address this issue, we propose a dynamic token reduction approach to accelerate PLMs' inference, named TR-BERT, which could flexibly adapt the layer number of each token in inference to avoid redundant calculation.
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
As far as we know, almost all existing textual backdoor attack methods insert additional contents into normal samples as triggers, which causes the trigger-embedded samples to be detected and the backdoor attacks to be blocked without much effort.
Automatic Construction of Sememe Knowledge Bases via Dictionaries
We use this method to build an English SKB and a French SKB, and conduct comprehensive evaluations from both intrinsic and extrinsic perspectives.
Knowledge Inheritance for Pre-trained Language Models
Specifically, we introduce a pre-training framework named "knowledge inheritance" (KI) and explore how could knowledge distillation serve as auxiliary supervision during pre-training to efficiently learn larger PLMs.
Fully Hyperbolic Neural Networks
Hyperbolic neural networks have shown great potential for modeling complex data.
Transfer Learning for Sequence Generation: from Single-source to Multi-source
Although directly finetuning pretrained models on MSG tasks and concatenating multiple sources into a single long sequence is regarded as a simple method to transfer pretrained models to MSG tasks, we conjecture that the direct finetuning method leads to catastrophic forgetting and solely relying on pretrained self-attention layers to capture cross-source information is not sufficient.
Open Hierarchical Relation Extraction
To establish the bidirectional connections between OpenRE and relation hierarchy, we propose the task of open hierarchical relation extraction and present a novel OHRE framework for the task.
Sub-Character Tokenization for Chinese Pretrained Language Models
2) Pronunciation-based SubChar tokenizers can encode Chinese homophones into the same transliteration sequences and produce the same tokenization output, hence being robust to homophone typos.
CCPM: A Chinese Classical Poetry Matching Dataset
We hope this dataset can further enhance the study on incorporating deep semantics into the understanding and generation system of Chinese classical poetry.
On the Language Coverage Bias for Neural Machine Translation
Language coverage bias, which indicates the content-dependent differences between sentence pairs originating from the source and target languages, is important for neural machine translation (NMT) because the target-original training data is not well exploited in current practice.
Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution
Recent studies show that neural natural language processing (NLP) models are vulnerable to backdoor attacks.
Alternated Training with Synthetic and Authentic Data for Neural Machine Translation
In this work, we propose alternated training with synthetic and authentic data for NMT.
CPM-2: Large-scale Cost-effective Pre-trained Language Models
We present a suite of cost-effective techniques for the use of PLMs to deal with the efficiency issues of pre-training, fine-tuning, and inference.
Language Models are Good Translators
Inspired by the recent progress of large-scale pre-trained language models on machine translation in a limited scenario, we firstly demonstrate that a single language model (LM4MT) can achieve comparable performance with strong encoder-decoder NMT models on standard machine translation benchmarks, using the same training data and similar amount of model parameters.
Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification
Tuning pre-trained language models (PLMs) with task-specific prompts has been a promising approach for text classification.
Lingxi: A Diversity-aware Chinese Modern Poetry Generation System
We propose nucleus sampling with randomized head (NS-RH) algorithm, which randomizes the high frequency part ("head") of the predicted distribution, in order to emphasize on the "comparatively low frequency" words.
Packed Levitated Marker for Entity and Relation Extraction
In particular, we propose a neighborhood-oriented packing strategy, which considers the neighbor spans integrally to better model the entity boundary information.
Ranked #1 on
Named Entity Recognition (NER)
on Few-NERD (SUP)
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models
Pre-Trained Vision-Language Models (VL-PTMs) have shown promising capabilities in grounding natural language in image data, facilitating a broad variety of cross-modal tasks.
MoEfication: Transformer Feed-forward Layers are Mixtures of Experts
In this work, we study the computational patterns of FFNs and observe that most inputs only activate a tiny ratio of neurons of FFNs.
Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer
In this paper, we make the first attempt to conduct adversarial and backdoor attacks based on text style transfer, which is aimed at altering the style of a sentence while preserving its meaning.
Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks
In this paper, we find two simple tricks that can make existing textual backdoor attacks much more harmful.
Exploring Universal Intrinsic Task Subspace via Prompt Tuning
In the experiments, we study diverse few-shot NLP tasks and surprisingly find that in a 250-dimensional subspace found with 100 tasks, by only tuning 250 free parameters, we can recover 97% and 83% of the full prompt tuning performance for 100 seen tasks (using different training data) and 20 unseen tasks, respectively, showing great generalization ability of the found intrinsic task subspace.
OpenPrompt: An Open-source Framework for Prompt-learning
Prompt-learning has become a new paradigm in modern natural language processing, which directly adapts pre-trained language models (PLMs) to $cloze$-style prediction, autoregressive modeling, or sequence to sequence generation, resulting in promising performances on various tasks.
On Transferability of Prompt Tuning for Natural Language Processing
To explore whether we can improve PT via prompt transfer, we empirically investigate the transferability of soft prompts across different downstream tasks and PLMs in this work.
CUGE: A Chinese Language Understanding and Generation Evaluation Benchmark
We argue that for general-purpose language intelligence evaluation, the benchmark itself needs to be comprehensive and systematic.
YACLC: A Chinese Learner Corpus with Multidimensional Annotation
This resource is of great relevance for second language acquisition research, foreign-language teaching, and automatic grammatical error correction.
Chord-Conditioned Melody Harmonization with Controllable Harmonicity
Melody harmonization has long been closely associated with chorales composed by Johann Sebastian Bach.
QuoteR: A Benchmark of Quote Recommendation for Writing
To facilitate the research on this task, we build a large and fully open quote recommendation dataset called QuoteR, which comprises three parts including English, standard Chinese and classical Chinese.
A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models
Pre-trained language models (PLMs) cannot well recall rich factual knowledge of entities exhibited in large-scale corpora, especially those rare entities.
ELLE: Efficient Lifelong Pre-training for Emerging Data
We experiment ELLE with streaming data from 5 domains on BERT and GPT.
Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models
This necessitates a new branch of research focusing on the parameter-efficient adaptation of PLMs, dubbed as delta tuning in this paper.
Sememe Prediction for BabelNet Synsets using Multilingual and Multimodal Information
In this paper, we utilize the multilingual synonyms, multilingual glosses and images in BabelNet for SPBS.
LEVEN: A Large-Scale Chinese Legal Event Detection Dataset
However, existing Legal Event Detection (LED) datasets only concern incomprehensive event types and have limited annotated data, which restricts the development of LED methods and their downstream applications.
Fine-Grained Scene Graph Generation with Data Transfer
Scene graph generation (SGG) is designed to extract (subject, predicate, object) triplets in images.
Ranked #1 on
Predicate Classification
on Visual Genome
A Roadmap for Big Model
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
Symphony Generation with Permutation Invariant Language Model
In this work, we propose a permutation invariant language model, SymphonyNet, as a solution for symbolic symphony music generation.
Ranked #1 on
Audio Generation
on Symphony music
Efficient and Training-Free Control of Language Generation
In recent years, there has been a growing interest in the development of language models capable of generating text with controllable attributes.
A Template-based Method for Constrained Neural Machine Translation
In this work, we propose a template-based method that can yield results with high translation quality and match accuracy and the inference speed of our method is comparable with unconstrained NMT models.
PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models
We show that PEVL enables state-of-the-art performance of detector-free VLP models on position-sensitive tasks such as referring expression comprehension and phrase grounding, and also improves the performance on position-insensitive tasks with grounded inputs.
Ranked #1 on
Visual Commonsense Reasoning
on VCR (Q-AR) test
Prompt Tuning for Discriminative Pre-trained Language Models
Recent works have shown promising results of prompt tuning in stimulating pre-trained language models (PLMs) for natural language processing (NLP) tasks.
Sparse Structure Search for Parameter-Efficient Tuning
The searched structures preserve more than 99\% fine-tuning performance with 0. 01\% trainable parameters.
A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks
However, we highlight two issues in previous backdoor learning evaluations: (1) The differences between real-world scenarios (e. g. releasing poisoned datasets or models) are neglected, and we argue that each scenario has its own constraints and concerns, thus requires specific evaluation protocols; (2) The evaluation metrics only consider whether the attacks could flip the models' predictions on poisoned samples and retain performances on benign samples, but ignore that poisoned samples should also be stealthy and semantic-preserving.
