Search Results for author:
Found 223 papers, 125 papers with code
BP-Transformer: Modelling Long-Range Context via Binary Partitioning
The Transformer model is widely successful on many natural language processing tasks.
Ranked #1 on
Machine Translation
on IWSLT2015 Chinese-English
Star-Transformer
Although Transformer has achieved great successes on many NLP tasks, its heavy structure with fully-connected attention connections leads to dependencies on large training data.
Ranked #13 on
Sentiment Analysis
on SST-5 Fine-grained classification
Named Entity Recognition (NER)
Natural Language Inference
+2
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
Recently, there has been growing interest in extending the context length of large language models (LLMs), aiming to effectively process long inputs of one turn or conversations with more extensive histories.
The Rise and Potential of Large Language Model Based Agents: A Survey
Many efforts have been made to develop intelligent agents, but they mainly focus on advancement in algorithms or training strategies to enhance specific capabilities or performance on particular tasks.
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
TENER: Adapting Transformer Encoder for Named Entity Recognition
The Bidirectional long short-term memory networks (BiLSTM) have been widely used as an encoder in models solving the named entity recognition (NER) task.
Ranked #11 on
Chinese Named Entity Recognition
on Resume NER
BERT-ATTACK: Adversarial Attack Against BERT Using BERT
Adversarial attacks for discrete data (such as texts) have been proved significantly more challenging than continuous data (such as images) since it is difficult to generate adversarial samples with gradient-based methods.
Pre-trained Models for Natural Language Processing: A Survey
Recently, the emergence of pre-trained models (PTMs) has brought natural language processing (NLP) to a new era.
Secrets of RLHF in Large Language Models Part I: PPO
Therefore, we explore the PPO-max, an advanced version of PPO algorithm, to efficiently improve the training stability of the policy model.
Secrets of RLHF in Large Language Models Part II: Reward Modeling
We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset and fully leverage high-quality preference data.
FLAT: Chinese NER Using Flat-Lattice Transformer
Recently, the character-word lattice structure has been proved to be effective for Chinese named entity recognition (NER) by incorporating the word information.
Ranked #5 on
Chinese Named Entity Recognition
on MSRA
Chinese Named Entity Recognition
named-entity-recognition
+3
Full Parameter Fine-tuning for Large Language Models with Limited Resources
Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) but demand massive GPU resources for training.
AdaLomo: Low-memory Optimization with Adaptive Learning Rate
Building on this insight, we introduce the low-memory optimization with adaptive learning rate (AdaLomo), which offers an adaptive learning rate for each parameter.
CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems
In this paper, we perform an in-depth analysis of characteristics of different datasets and investigate the performance of different summarization models under a cross-dataset setting, in which a summarizer trained on one corpus will be evaluated on a range of out-of-domain corpora.
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities
Multi-modal large language models are regarded as a crucial step towards Artificial General Intelligence (AGI) and have garnered significant interest with the emergence of ChatGPT.
SpeechGPT-Gen: Scaling Chain-of-Information Speech Generation
It comprises an autoregressive model based on LLM for semantic information modeling and a non-autoregressive model employing flow matching for perceptual information modeling.
SpeechAlign: Aligning Speech Generation to Human Preferences
However, the integration of human feedback to align speech outputs to human preferences is often neglected.
fastHan: A BERT-based Multi-Task Toolkit for Chinese NLP
The joint-model is trained and evaluated on 13 corpora of four tasks, yielding near state-of-the-art (SOTA) performance in dependency parsing and NER, achieving SOTA performance in CWS and POS.
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
To guarantee user acceptability, all the text transformations are linguistically based, and we provide a human evaluation for each one.
How to Fine-Tune BERT for Text Classification?
Language model pre-training has proven to be useful in learning universal language representations.
Ranked #1 on
Text Classification
on Yahoo! Answers
EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education
Currently, EduChat is available online as an open-source project, with its code, data, and model parameters available on platforms (e. g., GitHub https://github. com/icalk-nlp/EduChat, Hugging Face https://huggingface. co/ecnu-icalk ).
Extractive Summarization as Text Matching
This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems.
Ranked #1 on
Text Summarization
on BBC XSum
Evaluating Hallucinations in Chinese Large Language Models
We analyze the primary types of hallucinations in different types of models and their causes.
Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence
Aspect-based sentiment analysis (ABSA), which aims to identify fine-grained opinion polarity towards a specific aspect, is a challenging subtask of sentiment analysis (SA).
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+5
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation
In this paper, we take the advantage of previous pre-trained models (PTMs) and propose a novel Chinese Pre-trained Unbalanced Transformer (CPT).
Evaluating the Performance of Large Language Models on GAOKAO Benchmark
Large Language Models(LLMs) have demonstrated remarkable performance across various natural language processing tasks; however, how to comprehensively and accurately assess their performance becomes an urgent issue to be addressed.
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling
We introduce AnyGPT, an any-to-any multimodal language model that utilizes discrete representations for the unified processing of various modalities, including speech, text, images, and music.
CoNT: Contrastive Neural Text Generation
We validate CoNT on five generation tasks with ten benchmarks, including machine translation, summarization, code comment generation, data-to-text generation and commonsense generation.
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way
This paper introduces CoLLiE, an efficient library that facilitates collaborative training of large language models using 3D parallelism, parameter-efficient fine-tuning (PEFT) methods, and optimizers such as Lion, Adan, Sophia, LOMO and AdaLomo.
A Survey of Reasoning with Foundation Models
Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation.
Character-LLM: A Trainable Agent for Role-Playing
Large language models (LLMs) can be used to serve as agents to simulate human behaviors, given the powerful ability to understand human instructions and provide high-quality generated texts.
Finding Sparse Structures for Domain Specific Neural Machine Translation
Neural machine translation often adopts the fine-tuning approach to adapt to specific domains.
SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
Therefore, we propose SpeechTokenizer, a unified speech tokenizer for speech large language models.
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models
We present DiffusionBERT, a new generative masked language model based on discrete diffusion models.
Black-Box Tuning for Language-Model-as-a-Service
In such a scenario, which we call Language-Model-as-a-Service (LMaaS), the gradients of PTMs are usually unavailable.
BBTv2: Towards a Gradient-Free Future with Large Language Models
By contrast, gradient-free methods only require the forward computation of the PTM to tune the prompt, retaining the benefits of efficient tuning and deployment.
Heterogeneous Graph Neural Networks for Extractive Document Summarization
An intuitive way is to put them in the graph-based neural network, which has a more complex structure for capturing inter-sentence relationships.
A Unified Generative Framework for Various NER Subtasks
To that end, we propose to formulate the NER subtasks as an entity span sequence generation task, which can be solved by a unified sequence-to-sequence (Seq2Seq) framework.
Ranked #10 on
Nested Named Entity Recognition
on GENIA
InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning
We further explore how to use LEAN to solve math problems and study its performance under the setting of multi-task learning which shows the possibility of using LEAN as a unified platform for solving and proving in math.
Training-Free Long-Context Scaling of Large Language Models
The ability of Large Language Models (LLMs) to process and generate coherent text is markedly weakened when the number of input tokens exceeds their pretraining length.
Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation
Disentangling the content and style in the latent space is prevalent in unpaired text style transfer.
Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information
We investigate the following question for machine translation (MT): can we develop a single universal MT model to serve as the common seed and obtain derivative and improved models on arbitrary language pairs?
Ranked #3 on
Machine Translation
on WMT2014 English-French
(using extra training data)
A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond
Building on our examination of the developmental trajectories, we further investigate the emerging synergies between code intelligence and broader machine intelligence, uncovering new cross-domain opportunities and illustrating the substantial influence of code intelligence across various domains.
A Unified Generative Framework for Aspect-Based Sentiment Analysis
Aspect-based Sentiment Analysis (ABSA) aims to identify the aspect terms, their corresponding sentiment polarities, and the opinion terms.
Ranked #1 on
Aspect Sentiment Triplet Extraction
on SemEval
Aspect-Based Sentiment Analysis
Aspect-oriented Opinion Extraction
+2
Reinforced Mnemonic Reader for Machine Reading Comprehension
In this paper, we introduce the Reinforced Mnemonic Reader for machine reading comprehension tasks, which enhances previous attentive readers in two aspects.
Ranked #17 on
Question Answering
on SQuAD1.1 dev
Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa
In this paper, we firstly compare the induced trees from PTMs and the dependency parsing trees on several popular models for the ABSA task, showing that the induced tree from fine-tuned RoBERTa (FT-RoBERTa) outperforms the parser-provided tree.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+1
Overview of the NLPCC 2017 Shared Task: Chinese News Headline Categorization
In this paper, we give an overview for the shared task at the CCF Conference on Natural Language Processing \& Chinese Computing (NLPCC 2017): Chinese News Headline Categorization.
CoLAKE: Contextualized Language and Knowledge Embedding
With the emerging branch of incorporating factual knowledge into pre-trained language models such as BERT, most existing models consider shallow, static, and separately pre-trained entity embeddings, which limits the performance gains of these models.
QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
As increasing numbers of meetings are recorded and transcribed, meeting summaries have become essential to remind those who may or may not have attended the meetings about the key decisions made and the tasks to be completed.
An Embarrassingly Easy but Strong Baseline for Nested Named Entity Recognition
In this paper, we propose using Convolutional Neural Network (CNN) to model these spatial relations in the score matrix.
Ranked #3 on
Nested Named Entity Recognition
on ACE 2005
Searching for Effective Neural Extractive Summarization: What Works and What's Next
The recent years have seen remarkable success in the use of deep neural networks on text summarization.
Ranked #6 on
Extractive Text Summarization
on CNN / Daily Mail
GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge
Word Sense Disambiguation (WSD) aims to find the exact sense of an ambiguous word in a particular context.
Ranked #3 on
Word Sense Disambiguation
on WiC-TSV
Learning Sparse Sharing Architectures for Multiple Tasks
Most existing deep multi-task learning models are based on parameter sharing, such as hard sharing, hierarchical sharing, and soft sharing.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Due to the difficulty and high cost of data collection, the supervised data available in the two fields are usually on the magnitude of tens of thousands, for example, 18K in the WebNLG~2017 dataset after preprocessing, which is far fewer than the millions of data for other tasks such as machine translation.
Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings
Cycle-consistent training is widely used for jointly learning a forward and inverse mapping between two domains of interest without the cumbersome requirement of collecting matched pairs within each domain.
Toward Diverse Text Generation with Inverse Reinforcement Learning
Similar to the adversarial models, the reward and policy function in IRL are optimized alternately.
Do Large Language Models Know What They Don't Know?
Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks.
SpeechAgents: Human-Communication Simulation with Multi-Modal Multi-Agent Systems
In this paper, we propose SpeechAgents, a multi-modal LLM based multi-agent system designed for simulating human communication.
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
Pretraining data of large language models composes multiple domains (e. g., web texts, academic papers, codes), whose mixture proportions crucially impact the competence of outcome models.
CodeRetriever: Unimodal and Bimodal Contrastive Learning for Code Search
For bimodal contrastive learning, we leverage the documentation and in-line comments of code to build code-text pairs.
Soft-Labeled Contrastive Pre-training for Function-level Code Representation
In this paper, we present \textbf{SCodeR}, a \textbf{S}oft-labeled contrastive pre-training framework with two positive sample construction methods to learn functional-level \textbf{Code} \textbf{R}epresentation.
Scaling Laws of RoPE-based Extrapolation
The extrapolation capability of Large Language Models (LLMs) based on Rotary Position Embedding is currently a topic of considerable interest.
MouSi: Poly-Visual-Expert Vision-Language Models
This technique introduces a fusion network to unify the processing of outputs from different visual experts, while bridging the gap between image encoders and pre-trained LLMs.
Ranked #39 on
Visual Question Answering
on MM-Vet
U-Net: Machine Reading Comprehension with Unanswerable Questions
A key subtask is to reliably predict whether the question is unanswerable.
Ranked #12 on
Question Answering
on SQuAD2.0 dev
Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation
Fine-tuning pre-trained language models like BERT has become an effective way in NLP and yields state-of-the-art results on many downstream tasks.
Towards Efficient NLP: A Standard Evaluation and A Strong Baseline
ELUE is dedicated to depict the Pareto Frontier for various language understanding tasks, such that it can tell whether and how much a method achieves Pareto improvement.
SeqXGPT: Sentence-Level AI-Generated Text Detection
Therefore, it is important to build strong AI-generated text (AIGT) detectors.
Alignment for Honesty
Recent research has made significant strides in applying alignment techniques to enhance the helpfulness and harmlessness of large language models (LLMs) in accordance with human intentions.
Contrast and Generation Make BART a Good Dialogue Emotion Recognizer
Meanwhile, we utilize an auxiliary response generation task to enhance the model's ability of handling context information, thereby forcing the model to recognize emotions with similar semantics in diverse contexts.
Ranked #11 on
Emotion Recognition in Conversation
on EmoryNLP
Information Aggregation via Dynamic Routing for Sequence Encoding
The dynamic routing policy is dynamically deciding that what and how much information need be transferred from each word to the final encoding of the text sequence.
Ranked #43 on
Sentiment Analysis
on IMDb
Can AI Assistants Know What They Don't Know?
To answer this question, we construct a model-specific "I don't know" (Idk) dataset for an assistant, which contains its known and unknown questions, based on existing open-domain question answering datasets.
Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning
MP2 is a set of combinable prompts pre-trained on 38 Chinese tasks.
A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing
Our graph-based joint model achieves better performance than previous joint models and state-of-the-art results in both Chinese word segmentation and dependency parsing.
BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation
Automatic evaluation metrics are crucial to the development of generative systems.
Improving Contrastive Learning of Sentence Embeddings from AI Feedback
Our method utilizes AI feedback from large pre-trained language models (LLMs) to construct sample pairs with fine-grained sample similarity scores to improve contrastive learning.
Improving Image Captioning with Better Use of Captions
Image captioning is a multimodal problem that has drawn extensive attention in both the natural language processing and computer vision community.
Accelerating BERT Inference for Sequence Labeling via Early-Exit
To alleviate this problem, we extend the recent successful early-exit mechanism to accelerate the inference of PTMs for sequence labeling tasks.
Relation of the Relations: A New Paradigm of the Relation Extraction Problem
In natural language, often multiple entities appear in the same text.
Unified Demonstration Retriever for In-Context Learning
To train UDR, we cast various tasks' training signals into a unified list-wise ranking formulation by language model's feedback.
TAVAT: Token-Aware Virtual Adversarial Training for Language Understanding
Gradient-based adversarial training is widely used in improving the robustness of neural networks, while it cannot be easily adapted to natural language processing tasks since the embedding space is discrete.
CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors
A common practice is to recast the task into a text-to-text format such that generative LLMs of natural language (NL-LLMs) like GPT-3 can be prompted to solve it.
Enhancing Scientific Papers Summarization with Citation Graph
Previous work for text summarization in scientific domain mainly focused on the content of the input document, but seldom considering its citation network.
COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization
Traditional training paradigms for extractive and abstractive summarization systems always only use token-level or sentence-level training objectives.
LLatrieval: LLM-Verified Retrieval for Verifiable Generation
Thus, the LLM can iteratively provide feedback to retrieval and facilitate the retrieval result to fully support verifiable generation.
LongWanjuan: Towards Systematic Measurement for Long Text Quality
The quality of training data are crucial for enhancing the long-text capabilities of foundation models.
Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts
As large language models (LLMs) have shown effectiveness with different prompting methods, such as Chain of Thought, Program of Thought, we find that these methods have formed a great complementarity to each other on math reasoning tasks.
GenWiki: A Dataset of 1.3 Million Content-Sharing Text and Graphs for Unsupervised Graph-to-Text Generation
With a human-annotated test set, we provide this new benchmark dataset for future research on unsupervised text generation from knowledge graphs.
Ranked #1 on
Unsupervised KG-to-Text Generation
on GenWiki (Fine)
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier
Pre-trained models are widely used in fine-tuning downstream tasks with linear classifiers optimized by the cross-entropy loss, which might face robustness and stability problems.
BERT for Monolingual and Cross-Lingual Reverse Dictionary
Reverse dictionary is the task to find the proper target word given the word description.
An AMR-based Link Prediction Approach for Document-level Event Argument Extraction
Motivated by the fact that all event structures can be inferred from AMR, this work reformulates EAE as a link prediction problem on AMR graphs.
From Hypergraph Energy Functions to Hypergraph Neural Networks
Hypergraphs are a powerful abstraction for representing higher-order interactions between entities of interest.
GAOKAO-MM: A Chinese Human-Level Benchmark for Multimodal Models Evaluation
The Large Vision-Language Models (LVLMs) have demonstrated great abilities in image perception and language understanding.
A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder
Multi-criteria Chinese word segmentation (MCCWS) aims to exploit the relations among the multiple heterogeneous segmentation criteria and further improve the performance of each single criterion.
AutoTrans: Automating Transformer Design via Reinforced Architecture Search
Though the transformer architectures have shown dominance in many natural language understanding tasks, there are still unsolved issues for the training of transformer models, especially the need for a principled way of warm-up which has shown importance for stable training of a transformer, as well as whether the task at hand prefer to scale the attention product or not.
Watermarking LLMs with Weight Quantization
Abuse of large language models reveals high risks as large language models are being deployed at an astonishing speed.
VCWE: Visual Character-Enhanced Word Embeddings
Chinese is a logographic writing system, and the shape of Chinese characters contain rich syntactic and semantic information.
Joint Parsing and Generation for Abstractive Summarization
If generating a word can introduce an erroneous relation to the summary, the behavior must be discouraged.
Ranked #27 on
Text Summarization
on GigaWord
MarkBERT: Marking Word Boundaries Improves Chinese BERT
We present a Chinese BERT model dubbed MarkBERT that uses word information in this work.
Chinese Named Entity Recognition
named-entity-recognition
+7
Flames: Benchmarking Value Alignment of LLMs in Chinese
The widespread adoption of large language models (LLMs) across various regions underscores the urgent need to evaluate their alignment with human values.
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance
With the rapid development of large language models (LLMs), they are not only used as general-purpose AI assistants but are also customized through further fine-tuning to meet the requirements of different applications.
SpellBERT: A Lightweight Pretrained Model for Chinese Spelling Check
Chinese Spelling Check (CSC) is to detect and correct Chinese spelling errors.
Keyphrase Generation with Fine-Grained Evaluation-Guided Reinforcement Learning
In response to this problem, we propose a new fine-grained evaluation metric to improve the RL framework, which considers different granularities: token-level $F_1$ score, edit distance, duplication, and prediction quantities.
MoT: Memory-of-Thought Enables ChatGPT to Self-Improve
Specifically, MoT is divided into two stages: 1. before the test stage, the LLM pre-thinks on the unlabeled dataset and saves the high-confidence thoughts as external memory; 2.
Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration
Large Language Models (LLMs) are evolving at an unprecedented pace and have exhibited considerable capability in the realm of natural language processing (NLP) with world knowledge.
DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learning
These methods regularize the representation space by pulling similar sentence representations closer and pushing away the dissimilar ones and have been proven effective in various NLP tasks, e. g., semantic textual similarity (STS) tasks.
DORE: Document Ordered Relation Extraction based on Generative Framework
In this paper, we investigate the root cause of the underwhelming performance of the existing generative DocRE models and discover that the culprit is the inadequacy of the training paradigm, instead of the capacities of the models.
RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Trees
RLET iteratively performs single step reasoning with sentence selection and deduction generation modules, from which the training signal is accumulated across the tree with elaborately designed aligned reward function that is consistent with the evaluation.
Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication
Large Language Models (LLMs) have recently made significant strides in complex reasoning tasks through the Chain-of-Thought technique.
PromptNER: A Prompting Method for Few-shot Named Entity Recognition via k Nearest Neighbor Search
We use prompts that contains entity category information to construct label prototypes, which enables our model to fine-tune with only the support set.
Are Factuality Checkers Reliable? Adversarial Meta-evaluation of Factuality in Summarization
In this paper, we present an adversarial meta-evaluation methodology that allows us to (i) diagnose the fine-grained strengths and weaknesses of 6 existing top-performing metrics over 24 diagnostic test datasets, (ii) search for directions for further improvement by data augmentation.
What Dense Graph Do You Need for Self-Attention?
Transformers have made progress in miscellaneous tasks, but suffer from quadratic computational and memory complexities.
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts
Through extensive experimental results across various tasks and PTMs, we show that LPT can achieve competitive performance to full model tuning and other PETuning methods under both full-data and few-shot scenarios while possessing faster training speed and lower memory cost.
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation
Previous works usually adopt heuristic metrics such as the entropy of internal outputs to measure instance difficulty, which suffers from generalization and threshold-tuning.
Dialogue Meaning Representation for Task-Oriented Dialogue Systems
We propose Dialogue Meaning Representation (DMR), a pliable and easily extendable representation for task-oriented dialogue.
Is MultiWOZ a Solved Task? An Interactive TOD Evaluation Framework with User Simulator
Besides, we introduce a sentence-level and a session-level score to measure the sentence fluency and session coherence in the interactive evaluation.
PerturbScore: Connecting Discrete and Continuous Perturbations in NLP
Through experimental results, we find that we can build a connection between discrete and continuous perturbations and use the proposed PerturbScore to learn such correlation, surpassing previous methods used in discrete perturbation measuring.
Rethinking Label Smoothing on Multi-hop Question Answering
Multi-Hop Question Answering (MHQA) is a significant area in question answering, requiring multiple reasoning components, including document retrieval, supporting sentence prediction, and answer span extraction.
F-Eval: Asssessing Fundamental Abilities with Refined Evaluation Methods
However, these evaluation benchmarks are limited to assessing the instruction-following capabilities, overlooking the fundamental abilities that emerge during the pre-training stage.
Hint-before-Solving Prompting: Guiding LLMs to Effectively Utilize Encoded Knowledge
Large Language Models (LLMs) have recently showcased remarkable generalizability in various domains.
The Open-World Lottery Ticket Hypothesis for OOD Intent Classification
Most existing methods of Out-of-Domain (OOD) intent classification rely on extensive auxiliary OOD corpora or specific training paradigms.
Mitigating Negative Style Transfer in Hybrid Dialogue System
As the functionality of dialogue systems evolves, hybrid dialogue systems that accomplish user-specific goals and participate in open-topic chitchat with users are attracting growing attention.
Same Representation, Different Attentions: Shareable Sentence Representation Learning from Multiple Tasks
More specifically, all tasks share the same sentence representation and each task can select the task-specific information from the shared sentence representation with attention mechanism.
Incorporating Discriminator in Sentence Generation: a Gibbs Sampling Method
Generating plausible and fluent sentence with desired properties has long been a challenge.
Meta Multi-Task Learning for Sequence Modeling
Specifically, we use a shared meta-network to capture the meta-knowledge of semantic composition and generate the parameters of the task-specific semantic composition models.
A Feature-Enriched Neural Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging
Recently, neural network models for natural language processing tasks have been increasingly focused on for their ability of alleviating the burden of manual feature engineering.
DAG-based Long Short-Term Memory for Neural Word Segmentation
In this paper, we propose a new neural model to incorporate the word-level information for Chinese word segmentation.
Dynamic Compositional Neural Networks over Tree Structure
Tree-structured neural networks have proven to be effective in learning semantic representations by exploiting syntactic information.
Adversarial Multi-Criteria Learning for Chinese Word Segmentation
Different linguistic perspectives causes many diverse segmentation criteria for Chinese word segmentation (CWS).
Adversarial Multi-task Learning for Text Classification
Neural network models have shown their promising opportunities for multi-task learning, which focus on learning the shared layers to extract the common and task-invariant features.
Knowledge Graph Representation with Jointly Structural and Textual Encoding
In this paper, we propose a novel deep architecture to utilize both structural and textual information of entities.
End-to-End Neural Sentence Ordering Using Pointer Network
However, it is nontrivial for pair-wise models to incorporate the contextual sentence information.
Cached Long Short-Term Memory Neural Networks for Document-Level Sentiment Classification
Recently, neural networks have achieved great success on sentiment classification due to their ability to alleviate feature engineering.
Deep Multi-Task Learning with Shared Memory
Neural network based models have achieved impressive results on various specific tasks.
Learning Word Embeddings from Intrinsic and Extrinsic Views
To prove the effectiveness of our model, we evaluate it on four tasks, including word similarity, reverse dictionaries, Wiki link prediction, and document classification.
Neural Sentence Ordering
Sentence ordering is a general and critical task for natural language generation applications.
Syntax-based Attention Model for Natural Language Inference
Introducing attentional mechanism in neural network is a powerful concept, and has achieved impressive results in many natural language processing tasks.
Modelling Interaction of Sentence Pair with coupled-LSTMs
Recently, there is rising interest in modelling the interactions of two sentences with deep neural networks.
Ranked #73 on
Natural Language Inference
on SNLI
Recurrent Neural Network for Text Classification with Multi-Task Learning
Neural network based methods have obtained great progress on a variety of natural language processing tasks.
Ranked #10 on
Emotion Recognition in Conversation
on CPED
Emotion Recognition in Conversation
General Classification
+3
Bridging LSTM Architecture and the Neural Dynamics during Reading
Recently, the long short-term memory neural network (LSTM) has attracted wide interest due to its success in many tasks.
Gaussian Mixture Embeddings for Multiple Word Prototypes
In this paper, we propose the Gaussian mixture skip-gram (GMSG) model to learn the Gaussian mixture embeddings for words based on skip-gram framework.
Overview of the NLPCC 2015 Shared Task: Chinese Word Segmentation and POS Tagging for Micro-blog Texts
In this paper, we give an overview for the shared task at the 4th CCF Conference on Natural Language Processing \& Chinese Computing (NLPCC 2015): Chinese word segmentation and part-of-speech (POS) tagging for micro-blog texts.
A Re-ranking Model for Dependency Parser with Recursive Convolutional Neural Network
In this work, we address the problem to model all the nodes (words or phrases) in a dependency tree with the dense representations.
Top-Down Tree Structured Text Generation
Text generation is a fundamental building block in natural language processing tasks.
Gaussian Word Embedding with a Wasserstein Distance Loss
Therefore, with the aim of representing words in a highly efficient way, we propose to operate a Gaussian word embedding model with a loss function based on the Wasserstein distance.
Exploring Shared Structures and Hierarchies for Multiple NLP Tasks
Designing shared neural architecture plays an important role in multi-task learning.
Deformable Stacked Structure for Named Entity Recognition
Neural architecture for named entity recognition has achieved great success in the field of natural language processing.
Neural Arithmetic Expression Calculator
With curriculum learning, our model can deal with a complex arithmetic expression calculation with the deep hierarchical structure of skill models.
Multi-task Learning over Graph Structures
We present two architectures for multi-task learning with neural sequence models.
Switch-LSTMs for Multi-Criteria Chinese Word Segmentation
With these auto-switched LSTMs, our model provides a more flexible solution for multi-criteria CWS, which is also easy to transfer the learned knowledge to new criteria.
Convolutional Interaction Network for Natural Language Inference
Attention-based neural models have achieved great success in natural language inference (NLI).
A Simple yet Effective Joint Training Method for Cross-Lingual Universal Dependency Parsing
This paper describes Fudan{'}s submission to CoNLL 2018{'}s shared task Universal Dependency Parsing.
Idiom-Aware Compositional Distributed Semantics
Idioms are peculiar linguistic constructions that impose great challenges for representing the semantics of language, especially in current prevailing end-to-end neural models, which assume that the semantics of a phrase or sentence can be literally composed from its constitutive words.
DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks
Variants dropout methods have been designed for the fully-connected layer, convolutional layer and recurrent layer in neural networks, and shown to be effective to avoid overfitting.
Exploring Domain Shift in Extractive Text Summarization
Although domain shift has been well explored in many NLP applications, it still has received little attention in the domain of extractive text summarization.
A Closer Look at Data Bias in Neural Extractive Summarization Models
In this paper, we take stock of the current state of summarization datasets and explore how different factors of datasets influence the generalization behaviour of neural extractive summarization models.
Multi-Scale Self-Attention for Text Classification
In this paper, we introduce the prior knowledge, multi-scale structure, into self-attention modules.
Unified Multi-Criteria Chinese Word Segmentation with BERT
Besides, the pre-trained BERT language model has been also introduced into the MCCWS task in a multi-task learning framework.
Improving Image Captioning with Better Use of Caption
Image captioning is a multimodal problem that has drawn extensive attention in both the natural language processing and computer vision community.
AutoRC: Improving BERT Based Relation Classification Models via Architecture Search
Ablation study demonstrates the necessity of our search space design and the effectiveness of our search method.
Pre-training with Meta Learning for Chinese Word Segmentation
Recent researches show that pre-trained models (PTMs) are beneficial to Chinese Word Segmentation (CWS).
Text Information Aggregation with Centrality Attention
A lot of natural language processing problems need to encode the text sequence as a fix-length vector, which usually involves aggregation process of combining the representations of all the words, such as pooling or self-attention.
Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces
The substitutions in the generated adversarial examples are not characters or words but \textit{'pieces'}, which are more natural to Chinese readers.
Early Exiting with Ensemble Internal Classifiers
In this paper, we show that a novel objective function for the training of the ensemble internal classifiers can be naturally induced from the perspective of ensemble learning and information theory.
A Survey of Transformers
X-formers) have been proposed, however, a systematic and comprehensive literature review on these Transformer variants is still missing.
Pre-Trained Models: Past, Present and Future
Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI).
Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning
\textbf{P}re-\textbf{T}rained \textbf{M}odel\textbf{s} have been widely applied and recently proved vulnerable under backdoor attacks: the released pre-trained weights can be maliciously poisoned with certain triggers.
Learning to Teach with Student Feedback
This one-way interaction leads to the teacher's inability to perceive the characteristics of the student and its training progress.
RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarization
Existing summarization systems mostly generate summaries purely relying on the content of the source document.
Towards More Effective and Economic Sparsely-Activated Model
To increase the number of activated experts without an increase in computational cost, we propose SAM (Switch and Mixture) routing, an efficient hierarchical routing mechanism that activates multiple experts in a same device (GPU).
{\mathcal{P}^2}: A Plan-and-Pretrain Approach for Knowledge Graph-to-Text Generation
Text verbalization of knowledge graphs is an important problem with wide application to natural language generation (NLG) systems.
Towards Collaborative Question Answering: A Preliminary Study
Knowledge and expertise in the real-world can be disjointedly owned.
TURNER: The Uncertainty-based Retrieval Framework for Chinese NER
Chinese NER is a difficult undertaking due to the ambiguity of Chinese characters and the absence of word boundaries.
$\mathcal{Y}$-Tuning: An Efficient Tuning Paradigm for Large-Scale Pre-Trained Models via Label Representation Learning
With the success of large-scale pre-trained models (PTMs), how efficiently adapting PTMs to downstream tasks has attracted tremendous attention, especially for PTMs with billions of parameters.
"Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction
The meaning of a word in Chinese is different in that a word is a compositional unit consisting of multiple characters.
Text Adversarial Purification as Defense against Adversarial Attacks
Adversarial purification is a successful defense mechanism against adversarial attacks without requiring knowledge of the form of the incoming attack.
“Is Whole Word Masking Always Better for Chinese BERT?”: Probing on Chinese Grammatical Error Correction
The meaning of a word in Chinese is different in that a word is a compositional unit consisting of multiple characters.
KNN-Contrastive Learning for Out-of-Domain Intent Classification
The Out-of-Domain (OOD) intent classification is a basic and challenging task for dialogue systems.
Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding
Our experiments on 13 benchmark datasets across five natural language understanding tasks demonstrate the superiority of our method.
A Unified Generative Framework based on Prompt Learning for Various Information Extraction Tasks
In this paper, we propose a novel composable prompt-based generative framework, which could be applied to a wide range of tasks in the field of Information Extraction.
Improving Abstractive Dialogue Summarization with Speaker-Aware Supervised Contrastive Learning
For dialogue summarization, the subdomain of text summarization, utterances are concatenated to flat text before being processed.
Discovering New Intents Using Latent Variables
In this paper, starting from the intuition that discovering intents could be beneficial to the identification of the known intents, we propose a probabilistic framework for discovering intents where intent assignments are treated as latent variables.
