Search Results for author:
Found 260 papers, 119 papers with code
A Knowledge-Aware Sequence-to-Tree Network for Math Word Problem Solving
With the advancements in natural language processing tasks, math word problem solving has received increasing attention.
A Structure-Aware Argument Encoder for Literature Discourse Analysis
Existing research for argument representation learning mainly treats tokens in the sentence equally and ignores the implied structure information of argumentative context.
Making Parameter-efficient Tuning More Efficient: A Unified Framework for Classification Tasks
Specifically, we re-formulate both token and sentence classification tasks into a unified language modeling task, and map label spaces of different tasks into the same vocabulary space.
CQG: A Simple and Effective Controlled Generation Framework for Multi-hop Question Generation
CQG employs a simple method to generate the multi-hop questions that contain key entities in multi-hop reasoning chains, which ensure the complexity and quality of the questions.
Flooding-X: Improving BERT’s Resistance to Adversarial Attacks via Loss-Restricted Fine-Tuning
Adversarial robustness has attracted much attention recently, and the mainstream solution is adversarial training.
PathQG: Neural Question Generation from Facts
Human evaluation also proves that our model is able to generate relevant and informative questions.
LFKQG: A Controlled Generation Framework with Local Fine-tuning for Question Generation over Knowledge Bases
Existing KBQG models still face two main challenges: (1) Most models often focus on the most relevant part of the answer entity, while neglecting the rest of the subgraph.
Multi-task Learning with Gradient Communication
In this paper, we describe a general framework to systematically analyze current neural models for multi-task learning, in which we find that existing models expect to disentangle features into different spaces while features learned in practice are still entangled in shared space, leaving potential hazards for other training or unseen tasks.
Towards Adversarially Robust Text Classifiers by Learning to Reweight Clean Examples
Most of the existing defense methods improve the adversarial robustness by making the models adapt to the training set augmented with some adversarial examples.
A Progressive Framework for Role-Aware Rumor Resolution
Existing works on rumor resolution have shown great potential in recognizing word appearance and user participation.
PlugAT: A Plug and Play Module to Defend against Textual Adversarial Attack
To reduce the potential side effects of using defense modules, we further propose a novel forgetting restricted adversarial training, which filters out bad adversarial examples that impair the performance of original ones.
Improving Abstractive Dialogue Summarization with Speaker-Aware Supervised Contrastive Learning
For dialogue summarization, the subdomain of text summarization, utterances are concatenated to flat text before being processed.
DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learning
For task completion, the agent needs to align and integrate various navigation modalities, including instruction, observation and navigation history.
Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models
The generated demos strategically interpolate between existing demos and the given query, transforming the query from OOD to ID.
Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals
We first empirically show that the features of either clean signals or adversarial perturbations are redundant and span in low-dimensional linear subspaces respectively with minimal overlap, and the classical low-dimensional subspace projection can suppress perturbation features out of the subspace of clean signals.
EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models
This paper introduces EasyJailbreak, a unified framework simplifying the construction and evaluation of jailbreak attacks against LLMs.
Decoding Continuous Character-based Language from Non-invasive Brain Recordings
Deciphering natural language from brain activity through non-invasive devices remains a formidable challenge.
Debatrix: Multi-dimensinal Debate Judge with Iterative Chronological Analysis Based on LLM
How can we construct an automated debate judge to evaluate an extensive, vibrant, multi-turn debate?
ALaRM: Align Language Models via Hierarchical Rewards Modeling
We introduce ALaRM, the first framework modeling hierarchical rewards in reinforcement learning from human feedback (RLHF), which is designed to enhance the alignment of large language models (LLMs) with human preferences.
CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models
Adversarial misuse, particularly through `jailbreaking' that circumvents a model's safety and ethical protocols, poses a significant challenge for Large Language Models (LLMs).
Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulation
Simulating the response of the public and forecasting the potential impact has become increasingly important.
RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions
In this paper, drawing inspiration from recent works that LLMs are sensitive to the design of the instructions, we utilize instructions in code style, which are more structural and less ambiguous, to replace typically natural language instructions.
Advancing Parameter Efficiency in Fine-tuning via Representation Editing
Parameter Efficient Fine-Tuning (PEFT) has gained significant attention for its ability to achieve competitive results while updating only a small subset of trainable parameters.
On the Tip of the Tongue: Analyzing Conceptual Representation in Large Language Models with Reverse-Dictionary Probe
Here we re-purpose the reverse dictionary task as a case study to probe LLMs' capacity for conceptual inference.
Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysis
Domain adaption has been widely adapted for cross-domain sentiment analysis to transfer knowledge from the source domain to the target domain.
LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition
To overcome the limitations of existing data augmentation methods that compromise semantic integrity and address the uncertainty inherent in LLM-generated text, we leverage the distinctive characteristics of the NER task by augmenting the original data at both the contextual and entity levels.
Unveiling Linguistic Regions in Large Language Models
Furthermore, this core region exhibits significant dimensional dependency, perturbations to even a single parameter on specific dimensions leading to a loss of linguistic competence.
SoMeLVLM: A Large Vision Language Model for Social Media Processing
The powerful Large Vision Language Models make it possible to handle a variety of tasks simultaneously, but even with carefully designed prompting methods, the general domain models often fall short in aligning with the unique speaking style and context of social media tasks.
Advancing Translation Preference Modeling with RLHF: A Step Towards Cost-Effective Solution
To address this issue, we propose a cost-effective preference learning strategy, optimizing reward models by distinguishing between human and machine translations.
LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration
Large language models (LLMs) have demonstrated impressive performance in understanding language and executing complex reasoning tasks.
Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation
Towards a more scalable, robust and fine-grained evaluation, we implement six reframing operations to construct evolving instances testing LLMs against diverse queries, data noise and probing their problem-solving sub-abilities.
LLM can Achieve Self-Regulation via Hyperparameter Aware Generation
HAG extends the current paradigm in the text generation process, highlighting the feasibility of endowing the LLMs with self-regulate decoding strategies.
LongHeads: Multi-Head Attention is Secretly a Long Context Processor
Instead of allowing each head to attend to the full sentence, which struggles with generalizing to longer sequences due to out-of-distribution (OOD) issues, we allow each head to process in-distribution length by selecting and attending to important context chunks.
ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages
Tool learning is widely acknowledged as a foundational approach or deploying large language models (LLMs) in real-world scenarios.
Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
In this paper, we propose R$^3$: Learning Reasoning through Reverse Curriculum Reinforcement Learning (RL), a novel method that employs only outcome supervision to achieve the benefits of process supervision for large language models.
Are Large Language Models Good Prompt Optimizers?
Despite the success, the underlying mechanism of this approach remains unexplored, and the true effectiveness of LLMs as Prompt Optimizers requires further validation.
StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
The advancement of large language models (LLMs) has significantly propelled the field of code generation.
Efficient and Effective Time-Series Forecasting with Spiking Neural Networks
Spiking neural networks (SNNs), inspired by the spiking behavior of biological neurons, provide a unique pathway for capturing the intricacies of temporal data.
Navigating the OverKill in Large Language Models
Large language models are meticulously aligned to be both helpful and harmless.
CauESC: A Causal Aware Model for Emotional Support Conversation
Emotional Support Conversation aims at reducing the seeker's emotional distress through supportive response.
MouSi: Poly-Visual-Expert Vision-Language Models
This technique introduces a fusion network to unify the processing of outputs from different visual experts, while bridging the gap between image encoders and pre-trained LLMs.
Ranked #39 on
Visual Question Answering
on MM-Vet
F-Eval: Asssessing Fundamental Abilities with Refined Evaluation Methods
However, these evaluation benchmarks are limited to assessing the instruction-following capabilities, overlooking the fundamental abilities that emerge during the pre-training stage.
RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning
To bridge this gap, we introduce RoTBench, a multi-level benchmark for evaluating the robustness of LLMs in tool learning.
Open the Pandora's Box of LLMs: Jailbreaking LLMs through Representation Engineering
To overcome these limitations, we propose a novel jailbreaking approach, named Jailbreaking LLMs through Representation Engineering (JRE).
Secrets of RLHF in Large Language Models Part II: Reward Modeling
We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset and fully leverage high-quality preference data.
LLaMA Beyond English: An Empirical Study on Language Capability Transfer
In recent times, substantial advancements have been witnessed in large language models (LLMs), exemplified by ChatGPT, showcasing remarkable proficiency across a range of complex tasks.
ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios
Furthermore, a sole emphasis on outcomes disregards the intricate capabilities essential for LLMs to effectively utilize tools.
Aligning Large Language Models with Human Preferences through Representation Engineering
Aligning large language models (LLMs) with human preferences is crucial for enhancing their utility in terms of helpfulness, truthfulness, safety, harmlessness, and interestingness.
Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation
The results show the competitiveness of our proposed framework and evaluator in counter-argument generation tasks.
K-ESConv: Knowledge Injection for Emotional Support Dialogue Systems via Prompt Learning
Automatic psychological counseling requires mass of professional knowledge that can be found in online counseling forums.
A Soft Contrastive Learning-based Prompt Model for Few-shot Sentiment Analysis
Few-shot text classification has attracted great interest in both academia and industry due to the lack of labeled data in many fields.
LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks.
LLMEval: A Preliminary Study on How to Evaluate Large Language Models
The three crucial questions for LLM evaluation are ``what, where, and how to evaluate''.
Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication
Large Language Models (LLMs) have recently made significant strides in complex reasoning tasks through the Chain-of-Thought technique.
Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs in Language Pretraining
In this paper, we propose the Hierarchical Argumentation Graph (Hi-ArG), a new structure to organize arguments.
Making Harmful Behaviors Unlearnable for Large Language Models
Specifically, we introduce ``security vectors'', a few new parameters that can be separated from the LLM, to ensure LLM's responses are consistent with the harmful behavior.
Tailoring Personality Traits in Large Language Models via Unsupervisedly-Built Personalized Lexicons
Users can utilize UBPL to adjust the probability vectors of predicted words in the decoding phase of LLMs, thus influencing the personality expression of LLMs.
DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuning
We propose Multiple Experts Fine-tuning Framework to build a financial large language model (LLM), DISC-FinLLM.
Unveiling A Core Linguistic Region in Large Language Models
We have discovered a core region in LLMs that corresponds to linguistic competence, accounting for approximately 1% of the total model parameters.
Orthogonal Subspace Learning for Language Model Continual Learning
In this paper, we propose orthogonal low-rank adaptation (O-LoRA), a simple and efficient approach for continual learning in language models, effectively mitigating catastrophic forgetting while learning new tasks.
Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization
We then propose a meta-learning-based method to learn to align conceptual spaces of different languages, which facilitates zero-shot and few-shot generalization in concept classification and also offers insights into the cross-lingual in-context learning phenomenon.
Improving Generalization of Alignment with Human Preferences through Group Invariant Learning
In this work, we propose a novel approach that can learn a consistent policy via RL across various data groups or domains.
RealBehavior: A Framework for Faithfully Characterizing Foundation Models' Human-like Behavior Mechanisms
Reports of human-like behaviors in foundation models are growing, with psychological theories providing enduring tools to investigate these behaviors.
Efficient Link Prediction via GNN Layers Induced by Negative Sampling
Graph neural networks (GNNs) for link prediction can loosely be divided into two broad categories.
RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification
Recently, Target-oriented Multimodal Sentiment Classification (TMSC) has gained significant attention among scholars.
SpikeCLIP: A Contrastive Language-Image Pretrained Spiking Neural Network
Spiking neural networks (SNNs) have demonstrated the capability to achieve comparable performance to deep neural networks (DNNs) in both visual and linguistic domains while offering the advantages of improved energy efficiency and adherence to biological plausibility.
TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
In this paper, we introduce TRACE, a novel benchmark designed to evaluate continual learning in LLMs.
Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback serves as a crucial bridge, aligning large language models with human and societal values.
ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarks
Recent years have witnessed remarkable progress in the development of large vision-language models (LVLMs).
DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services
We propose DISC-LawLLM, an intelligent legal system utilizing large language models (LLMs) to provide a wide range of legal services.
The Rise and Potential of Large Language Model Based Agents: A Survey
Many efforts have been made to develop intelligent agents, but they mainly focus on advancement in algorithms or training strategies to enhance specific capabilities or performance on particular tasks.
SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillation
Spiking neural networks (SNNs) offer a promising avenue to implement deep neural networks in a more energy-efficient way.
DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation
We propose DISC-MedLLM, a comprehensive solution that leverages Large Language Models (LLMs) to provide accurate and truthful medical response in end-to-end conversational healthcare services.
Secrets of RLHF in Large Language Models Part I: PPO
Therefore, we explore the PPO-max, an advanced version of PPO algorithm, to efficiently improve the training stability of the policy model.
On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection
Detecting adversarial samples that are carefully crafted to fool the model is a critical step to socially-secure applications.
From Hypergraph Energy Functions to Hypergraph Neural Networks
Hypergraphs are a powerful abstraction for representing higher-order interactions between entities of interest.
Open Set Relation Extraction via Unknown-Aware Training
Inspired by text adversarial attacks, we adaptively apply small but critical perturbations to original training instances and thus synthesizing negative instances that are more likely to be mistaken by the model as known relations.
Do Large Language Models Know What They Don't Know?
Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks.
KNSE: A Knowledge-aware Natural Language Inference Framework for Dialogue Symptom Status Recognition
Symptom diagnosis in medical conversations aims to correctly extract both symptom entities and their status from the doctor-patient dialogue.
Query Structure Modeling for Inductive Logical Reasoning Over Knowledge Graphs
The results demonstrate the effectiveness of our method on logical reasoning over KGs in both inductive and transductive settings.
Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement
For example, with Text-davinci-003, our method boosts the performance of standard few-shot prompting by $8. 0\%$ on GSM8K and $17. 8\%$ on MultiArith; it also improves the performance of CoT by $6. 0\%$ on GSM8K and $6. 0\%$ on MathQA, respectively.
A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition
Particularly, we propose a Confidence-based Partial Label Learning (CPLL) method to integrate the prior confidence (given by annotators) and posterior confidences (learned by models) for crowd-annotated NER.
Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization
Models trained with empirical risk minimization (ERM) are revealed to easily rely on spurious correlations, resulting in poor generalization.
Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild
The principle of continual relation extraction~(CRE) involves adapting to emerging novel relations while preserving od knowledge.
CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors
A common practice is to recast the task into a text-to-text format such that generative LLMs of natural language (NL-LLMs) like GPT-3 can be prompted to solve it.
CausalAPM: Generalizable Literal Disentanglement for NLU Debiasing
Dataset bias, i. e., the over-reliance on dataset-specific literal heuristics, is getting increasing attention for its detrimental effect on the generalization ability of NLU models.
A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models
GPT series models, such as GPT-3, CodeX, InstructGPT, ChatGPT, and so on, have gained considerable attention due to their exceptional natural language processing capabilities.
How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks
The GPT-3. 5 models have demonstrated impressive performance in various Natural Language Processing (NLP) tasks, showcasing their strong understanding and reasoning capabilities.
Natural Language Inference
Natural Language Understanding
+1
Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer?
We demonstrate that the distance between the distributions of different languages is highly consistent with the syntactic difference in terms of linguistic formalisms.
Rethinking Label Smoothing on Multi-hop Question Answering
Multi-Hop Question Answering (MHQA) is a significant area in question answering, requiring multiple reasoning components, including document retrieval, supporting sentence prediction, and answer span extraction.
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models
We present DiffusionBERT, a new generative masked language model based on discrete diffusion models.
Efficient Adversarial Training with Robust Early-Bird Tickets
Adversarial training is one of the most powerful methods to improve the robustness of pre-trained language models (PLMs).
Robust Lottery Tickets for Pre-trained Language Models
Recent works on Lottery Ticket Hypothesis have shown that pre-trained language models (PLMs) contain smaller matching subnetworks(winning tickets) which are capable of reaching accuracy comparable to the original models.
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts
Through extensive experimental results across various tasks and PTMs, we show that LPT can achieve competitive performance to full model tuning and other PETuning methods under both full-data and few-shot scenarios while possessing faster training speed and lower memory cost.
Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding
Dataset bias has attracted increasing attention recently for its detrimental effect on the generalization ability of fine-tuned models.
Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning
MP2 is a set of combinable prompts pre-trained on 38 Chinese tasks.
BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation
Automatic evaluation metrics are crucial to the development of generative systems.
COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization
Traditional training paradigms for extractive and abstractive summarization systems always only use token-level or sentence-level training objectives.
A Multi-Format Transfer Learning Model for Event Argument Extraction via Variational Information Bottleneck
Event argument extraction (EAE) aims to extract arguments with given roles from texts, which have been widely studied in natural language processing.
Locate Then Ask: Interpretable Stepwise Reasoning for Multi-hop Question Answering
In this paper, we propose an interpretable stepwise reasoning framework to incorporate both single-hop supporting sentence identification and single-hop question generation at each intermediate step, and utilize the inference of the current hop for the next until reasoning out the final result.
Causal Intervention Improves Implicit Sentiment Analysis
In this work, we propose a causal intervention model for Implicit Sentiment Analysis using Instrumental Variable (ISAIV).
A Unified Continuous Learning Framework for Multi-modal Knowledge Discovery and Pre-training
These two steps are iteratively performed in our framework for continuous learning.
CoNT: Contrastive Neural Text Generation
We validate CoNT on five generation tasks with ten benchmarks, including machine translation, summarization, code comment generation, data-to-text generation and commonsense generation.
What Dense Graph Do You Need for Self-Attention?
Transformers have made progress in miscellaneous tasks, but suffer from quadratic computational and memory complexities.
BBTv2: Towards a Gradient-Free Future with Large Language Models
By contrast, gradient-free methods only require the forward computation of the PTM to tune the prompt, retaining the benefits of efficient tuning and deployment.
A Benchmark for Automatic Medical Consultation System: Frameworks, Tasks and Datasets
In recent years, interest has arisen in using machine learning to improve the efficiency of automatic medical consultation and enhance patient experience.
MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective
NER model has achieved promising performance on standard NER benchmarks.
Ranked #8 on
Named Entity Recognition (NER)
on WNUT 2017
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation
Previous works usually adopt heuristic metrics such as the entropy of internal outputs to measure instance difficulty, which suffers from generalization and threshold-tuning.
Decorrelate Irrelevant, Purify Relevant: Overcome Textual Spurious Correlations from a Feature Perspective
Most of the existing debiasing methods often identify and weaken these samples with biased features (i. e., superficial surface features that cause such spurious correlations).
MVPTR: Multi-Level Semantic Alignment for Vision-Language Pre-Training via Multi-Stage Learning
In MVPTR, we follow the nested structure of both modalities to introduce concepts as high-level semantics.
Black-Box Tuning for Language-Model-as-a-Service
In such a scenario, which we call Language-Model-as-a-Service (LMaaS), the gradients of PTMs are usually unavailable.
Plug-Tagger: A Pluggable Sequence Labeling Framework Using Language Models
Specifically, for each task, a label word set is first constructed by selecting a high-frequency word for each class respectively, and then, task-specific vectors are inserted into the inputs and optimized to manipulate the model predictions towards the corresponding label words.
Towards Efficient NLP: A Standard Evaluation and A Strong Baseline
ELUE is dedicated to depict the Pareto Frontier for various language understanding tasks, such that it can tell whether and how much a method achieves Pareto improvement.
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier
Pre-trained models are widely used in fine-tuning downstream tasks with linear classifiers optimized by the cross-entropy loss, which might face robustness and stability problems.
Template-free Prompt Tuning for Few-shot NER
Prompt-based methods have been successfully applied in sentence-level few-shot learning tasks, mostly owing to the sophisticated design of templates and label words.
Constructing Phrase-level Semantic Labels to Form Multi-Grained Supervision for Image-Text Retrieval
Existing research for image text retrieval mainly relies on sentence-level supervision to distinguish matched and mismatched sentences for a query image.
Learning to Teach with Student Feedback
This one-way interaction leads to the teacher's inability to perceive the characteristics of the student and its training progress.
Fine-Grained Element Identification in Complaint Text of Internet Fraud
Existing system dealing with online complaint provides a final decision without explanations.
Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble
Although deep neural networks have achieved prominent performance on many NLP tasks, they are vulnerable to adversarial examples.
Math Word Problem Solving with Explicit Numerical Values
In recent years, math word problem solving has received considerable attention and achieved promising results, but previous methods rarely take numerical values into consideration.
Align Voting Behavior with Public Statements for Legislator Representation Learning
In addition, we propose a novel task, namely hashtag usage prediction to model the ideology of legislators on Twitter.
SENT: Sentence-level Distant Relation Extraction via Negative Training
In this work, we propose the use of negative training (NT), in which a model is trained using complementary labels regarding that ``the instance does not belong to these complementary labels".
TCIC: Theme Concepts Learning Cross Language and Vision for Image Captioning
Considering that theme concepts can be learned from both images and captions, we propose two settings for their representations learning based on TTN.
SpanNER: Named Entity Re-/Recognition as Span Prediction
Recent years have seen the paradigm shift of Named Entity Recognition (NER) systems from sequence labeling to span prediction.
Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models
A sequence-to-sequence learning with neural networks has empirically proven to be an effective framework for Chinese Spelling Correction (CSC), which takes a sentence with some spelling errors as input and outputs the corrected one.
Early Exiting with Ensemble Internal Classifiers
In this paper, we show that a novel objective function for the training of the ensemble internal classifiers can be naturally induced from the perspective of ensemble learning and information theory.
Accelerating BERT Inference for Sequence Labeling via Early-Exit
To alleviate this problem, we extend the recent successful early-exit mechanism to accelerate the inference of PTMs for sequence labeling tasks.
Certified Robustness to Text Adversarial Attacks by Randomized [MASK]
Recently, few certified defense methods have been developed to provably guarantee the robustness of a text classifier to adversarial synonym substitutions.
Who Responded to Whom: The Joint Effects of Latent Topics and Discourse in Conversation Structure
Numerous online conversations are produced on a daily basis, resulting in a pressing need to conversation understanding.
Larger-Context Tagging: When and Why Does It Work?
The development of neural networks and pretraining techniques has spawned many sentence-level tagging systems that achieved superior performance on typical benchmarks.
Enhancing Scientific Papers Summarization with Citation Graph
Previous work for text summarization in scientific domain mainly focused on the content of the input document, but seldom considering its citation network.
Mask Attention Networks: Rethinking and Strengthen Transformer
We therefore introduce a new layer named dynamic mask attention network (DMAN) with a learnable mask matrix which is able to model localness adaptively.
Ranked #11 on
Machine Translation
on WMT2014 English-German
Alleviate Exposure Bias in Sequence Prediction \\ with Recurrent Neural Networks
The key idea is that at each time step, the network takes as input a ``bundle'' of similar words predicted at the previous step instead of a single ground truth.
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
To guarantee user acceptability, all the text transformations are linguistically based, and we provide a human evaluation for each one.
An Unsupervised Sampling Approach for Image-Sentence Matching Using Document-Level Structural Information
In this paper, we focus on the problem of unsupervised image-sentence matching.
Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces
The substitutions in the generated adversarial examples are not characters or words but \textit{'pieces'}, which are more natural to Chinese readers.
Uncertainty-Aware Label Refinement for Sequence Labeling
Conditional random fields (CRF) for label decoding has become ubiquitous in sequence labeling tasks.
Unsupervised Summarization for Chat Logs with Topic-Oriented Ranking and Context-Aware Auto-Encoders
Automatic chat summarization can help people quickly grasp important information from numerous chat messages.
Topic-Oriented Spoken Dialogue Summarization for Customer Service with Saliency-Aware Topic Modeling
In a customer service system, dialogue summarization can boost service efficiency by automatically creating summaries for long spoken dialogues in which customers and agents try to address issues about specific topics.
SenSeNet: Neural Keyphrase Generation with Document Structure
Keyphrase Generation (KG) is the task of generating central topics from a given document or literary work, which captures the crucial information necessary to understand the content.
An Enhanced Knowledge Injection Model for Commonsense Generation
Commonsense generation aims at generating plausible everyday scenario description based on a set of provided concepts.
Modeling Evolution of Message Interaction for Rumor Resolution
Previous work for rumor resolution concentrates on exploiting time-series characteristics or modeling topology structure separately.
Text Information Aggregation with Centrality Attention
A lot of natural language processing problems need to encode the text sequence as a fix-length vector, which usually involves aggregation process of combining the representations of all the words, such as pooling or self-attention.
RethinkCWS: Is Chinese Word Segmentation a Solved Task?
The performance of the Chinese Word Segmentation (CWS) systems has gradually reached a plateau with the rapid development of deep neural networks, especially the successful use of large pre-trained models.
Toward Recognizing More Entity Types in NER: An Efficient Implementation using Only Entity Lexicons
In this work, we explore the way to quickly adjust an existing named entity recognition (NER) system to make it capable of recognizing entity types not defined in the system.
Cross-Lingual Dependency Parsing by POS-Guided Word Reordering
The words in each sentence of a source language corpus are rearranged to meet the word order in a target language under the guidance of a part-of-speech based language model (LM).
Automatic Term Name Generation for Gene Ontology: Task and Dataset
Terms contained in Gene Ontology (GO) have been widely used in biology and bio-medicine.
CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems
In this paper, we perform an in-depth analysis of characteristics of different datasets and investigate the performance of different summarization models under a cross-dataset setting, in which a summarizer trained on one corpus will be evaluated on a range of out-of-domain corpora.
CoLAKE: Contextualized Language and Knowledge Embedding
With the emerging branch of incorporating factual knowledge into pre-trained language models such as BERT, most existing models consider shallow, static, and separately pre-trained entity embeddings, which limits the performance gains of these models.
fastHan: A BERT-based Multi-Task Toolkit for Chinese NLP
The joint-model is trained and evaluated on 13 corpora of four tasks, yielding near state-of-the-art (SOTA) performance in dependency parsing and NER, achieving SOTA performance in CWS and POS.
Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis
Based on the SemEval 2014 dataset, we construct the Aspect Robustness Test Set (ARTS) as a comprehensive probe of the aspect robustness of ABSA models.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples
Despite achieving prominent performance on many important tasks, it has been reported that neural networks are vulnerable to adversarial examples.
Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble
Despite neural networks have achieved prominent performance on many natural language processing (NLP) tasks, they are vulnerable to adversarial examples.
Hierarchical Reinforcement Learning for Automatic Disease Diagnosis
Existing approaches usually employ a flat policy structure that treat all symptoms and diseases equally for action making.
Hierarchical Reinforcement Learning
reinforcement-learning
+1
Leveraging Declarative Knowledge in Text and First-Order Logic for Fine-Grained Propaganda Detection
We study the detection of propagandistic text fragments in news articles.
Heterogeneous Graph Neural Networks for Extractive Document Summarization
An intuitive way is to put them in the graph-based neural network, which has a more complex structure for capturing inter-sentence relationships.
FLAT: Chinese NER Using Flat-Lattice Transformer
Recently, the character-word lattice structure has been proved to be effective for Chinese named entity recognition (NER) by incorporating the word information.
Ranked #5 on
Chinese Named Entity Recognition
on MSRA
Chinese Named Entity Recognition
named-entity-recognition
+3
Extractive Summarization as Text Matching
This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems.
Ranked #1 on
Text Summarization
on BBC XSum
Unified Multi-Criteria Chinese Word Segmentation with BERT
Besides, the pre-trained BERT language model has been also introduced into the MCCWS task in a multi-task learning framework.
Storytelling from an Image Stream Using Scene Graphs
In particular, on the within-image level, we employ a Graph Convolution Network (GCN) to enrich local fine-grained region representations of objects on scene graphs.
Ranked #7 on
Visual Storytelling
on VIST
Pre-trained Models for Natural Language Processing: A Survey
Recently, the emergence of pre-trained models (PTMs) has brought natural language processing (NLP) to a new era.
Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation
Fine-tuning pre-trained language models like BERT has become an effective way in NLP and yields state-of-the-art results on many downstream tasks.
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
We study the problem of injecting knowledge into large pre-trained models like BERT and RoBERTa.
Ranked #1 on
Entity Typing
on Open Entity
Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
While neural network-based models have achieved impressive performance on a large body of NLP tasks, the generalization behavior of different models remains poorly understood: Does this excellent performance imply a perfect generalization model, or are there still some limitations?
Chinese Named Entity Recognition Augmented with Lexicon Memory
Inspired by a concept of content-addressable retrieval from cognitive science, we propose a novel fragment-based model augmented with a lexicon-based memory for Chinese NER, in which both the character-level and word-level features are combined to generate better feature representations for possible name candidates.
Chinese Named Entity Recognition
named-entity-recognition
+4
Constructing Multiple Tasks for Augmentation: Improving Neural Image Classification With K-means Features
In order to effectively reduce the impact of non-ideal auxiliary tasks on the main task, we further proposed a novel meta-learning-based multi-task learning approach, which trained the shared hidden layers on auxiliary tasks, while the meta-optimization objective was to minimize the loss on the main task, ensuring that the optimizing direction led to an improvement on the main task.
Learning Sparse Sharing Architectures for Multiple Tasks
Most existing deep multi-task learning models are based on parameter sharing, such as hard sharing, hierarchical sharing, and soft sharing.
Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communication
Visual storytelling aims to generate a narrative paragraph from a sequence of images automatically.
Ranked #9 on
Visual Storytelling
on VIST
Discrete Argument Representation Learning for Interactive Argument Pair Identification
In this paper, we focus on extracting interactive argument pairs from two posts with opposite stances to a certain topic.
A Lexicon-Based Graph Neural Network for Chinese NER
Recurrent neural networks (RNN) used for Chinese named entity recognition (NER) that sequentially track character and word information have achieved great success.
Ranked #13 on
Chinese Named Entity Recognition
on OntoNotes 4
Chinese Named Entity Recognition
named-entity-recognition
+3
Asynchronous Deep Interaction Network for Natural Language Inference
However, in the process of reasoning, the role of the two sentences is obviously different, and the sentence pairs for NLI are asymmetrical corpora.
A Closer Look at Data Bias in Neural Extractive Summarization Models
In this paper, we take stock of the current state of summarization datasets and explore how different factors of datasets influence the generalization behaviour of neural extractive summarization models.
Towards Interpretable Evaluations: A Case Study of Named Entity Recognition
With the proliferation of models for natural language processing (NLP) tasks, it is even harder to understand the differences between models and their relative merits.
Weighed Domain-Invariant Representation Learning for Cross-domain Sentiment Analysis
To address this problem, we propose a modification to DIRL, obtaining a novel weighted domain-invariant representation learning (WDIRL) framework.
Exploring Domain Shift in Extractive Text Summarization
Although domain shift has been well explored in many NLP applications, it still has received little attention in the domain of extractive text summarization.
GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge
Word Sense Disambiguation (WSD) aims to find the exact sense of an ambiguous word in a particular context.
Ranked #3 on
Word Sense Disambiguation
on WiC-TSV
Simplify the Usage of Lexicon in Chinese NER
This method avoids designing a complicated sequence modeling architecture, and for any neural NER model, it requires only subtle adjustment of the character representation layer to introduce the lexicon information.
Ranked #8 on
Chinese Named Entity Recognition
on Resume NER
Chinese Named Entity Recognition
named-entity-recognition
+2
DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks
Variants dropout methods have been designed for the fully-connected layer, convolutional layer and recurrent layer in neural networks, and shown to be effective to avoid overfitting.
Searching for Effective Neural Extractive Summarization: What Works and What's Next
The recent years have seen remarkable success in the use of deep neural networks on text summarization.
Ranked #6 on
Extractive Text Summarization
on CNN / Daily Mail
Generating Responses with a Specific Emotion in Dialog
It is desirable for dialog systems to have capability to express specific emotions during a conversation, which has a direct, quantifiable impact on improvement of their usability and user satisfaction.
Bridging by Word: Image Grounded Vocabulary Construction for Visual Captioning
Existing research usually employs the architecture of CNN-RNN that views the generation as a sequential decision-making process and the entire dataset vocabulary is used as decoding space.
A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder
Multi-criteria Chinese word segmentation (MCCWS) aims to exploit the relations among the multiple heterogeneous segmentation criteria and further improve the performance of each single criterion.
Distantly Supervised Named Entity Recognition using Positive-Unlabeled Learning
In this work, we explore the way to perform named entity recognition (NER) using only unlabeled data and named entity dictionaries.
Learning Task-specific Representation for Novel Words in Sequence Labeling
However, representations of unseen or rare words trained on the end task are usually poor for appreciable performance.
Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation
Disentangling the content and style in the latent space is prevalent in unpaired text style transfer.
How to Fine-Tune BERT for Text Classification?
Language model pre-training has proven to be useful in learning universal language representations.
Ranked #1 on
Text Classification
on Yahoo! Answers
A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing
Our graph-based joint model achieves better performance than previous joint models and state-of-the-art results in both Chinese word segmentation and dependency parsing.
VCWE: Visual Character-Enhanced Word Embeddings
Chinese is a logographic writing system, and the shape of Chinese characters contain rich syntactic and semantic information.
Contextualized Non-local Neural Networks for Sequence Learning
Recently, a large number of neural mechanisms and models have been proposed for sequence learning, of which self-attention, as exemplified by the Transformer model, and graph neural networks (GNNs) have attracted much attention.
Long Short-Term Memory with Dynamic Skip Connections
In recent years, long short-term memory (LSTM) has been successfully used to model sequential data of variable length.
Ranked #33 on
Sentiment Analysis
on IMDb
Incorporating Topic Aspects for Online Comment Convincingness Evaluation
In this paper, we propose to incorporate topic aspects information for online comments convincingness evaluation.
Meta-Learning Multi-task Communication
In this paper, we describe a general framework: Parameters Read-Write Networks (PRaWNs) to systematically analyze current neural models for multi-task learning, in which we find that existing models expect to disentangle features into different spaces while features learned in practice are still entangled in shared space, leaving potential hazards for other training or unseen tasks.
Automatic Essay Scoring Incorporating Rating Schema via Reinforcement Learning
Automatic essay scoring (AES) is the task of assigning grades to essays without human interference.
Convolutional Interaction Network for Natural Language Inference
Attention-based neural models have achieved great success in natural language inference (NLI).
Transferring from Formal Newswire Domain with Hypernet for Twitter POS Tagging
However, from a linguistic perspective, Twitter users not only tend to mimic the formal expressions of traditional media, like news, but they also appear to be developing linguistically informal styles.
Ranked #2 on
Part-Of-Speech Tagging
on Ritter
Deformable Stacked Structure for Named Entity Recognition
Neural architecture for named entity recognition has achieved great success in the field of natural language processing.
Exploring Shared Structures and Hierarchies for Multiple NLP Tasks
Designing shared neural architecture plays an important role in multi-task learning.
Gaussian Word Embedding with a Wasserstein Distance Loss
Therefore, with the aim of representing words in a highly efficient way, we propose to operate a Gaussian word embedding model with a loss function based on the Wasserstein distance.
