Search Results for author:
Found 237 papers, 134 papers with code
SpellBERT: A Lightweight Pretrained Model for Chinese Spelling Check
Chinese Spelling Check (CSC) is to detect and correct Chinese spelling errors.
KNN-Contrastive Learning for Out-of-Domain Intent Classification
The Out-of-Domain (OOD) intent classification is a basic and challenging task for dialogue systems.
Are Factuality Checkers Reliable? Adversarial Meta-evaluation of Factuality in Summarization
In this paper, we present an adversarial meta-evaluation methodology that allows us to (i) diagnose the fine-grained strengths and weaknesses of 6 existing top-performing metrics over 24 diagnostic test datasets, (ii) search for directions for further improvement by data augmentation.
{\mathcal{P}^2}: A Plan-and-Pretrain Approach for Knowledge Graph-to-Text Generation
Text verbalization of knowledge graphs is an important problem with wide application to natural language generation (NLG) systems.
Improving Abstractive Dialogue Summarization with Speaker-Aware Supervised Contrastive Learning
For dialogue summarization, the subdomain of text summarization, utterances are concatenated to flat text before being processed.
“Is Whole Word Masking Always Better for Chinese BERT?”: Probing on Chinese Grammatical Error Correction
The meaning of a word in Chinese is different in that a word is a compositional unit consisting of multiple characters.
DetectiveQA: Evaluating Long-Context Reasoning on Detective Novels
DetectiveQA focuses on evaluating the long-context reasoning ability of LLMs, which not only requires a full understanding of context but also requires extracting important evidences from the context and reasoning according to extracted evidences to answer the given questions.
Farewell to Length Extrapolation, a Training-Free Infinite Context with Finite Attention Scope
Furthermore, we have applied LongCache on mainstream LLMs, including LLaMA3 and Mistral-v0. 3, enabling them to support context lengths of at least 400K in Needle-In-A-Haystack tests.
Case2Code: Learning Inductive Reasoning with Synthetic Data
In this paper, we hope to focus on evaluating and teaching LLMs to conduct inductive reasoning, that is, LLMs are supposed to infer underlying rules by observing examples or sequential transformations.
What's Wrong with Your Code Generated by Large Language Models? An Extensive Study
The increasing development of large language models (LLMs) in code generation has drawn significant attention among researchers.
Scaling Laws for Fact Memorization of Large Language Models
We find that LLMs' fact knowledge capacity has a linear and negative exponential law relationship with model size and training epochs, respectively.
Cross-Modality Safety Alignment
As Artificial General Intelligence (AGI) becomes increasingly integrated into various facets of human life, ensuring the safety and ethical alignment of such systems is paramount.
Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluation
On MMLU, using Inference-time Decontamination can lead to a decrease in the results of Phi3 and Mistral by 6. 7% and 3. 6% respectively.
Unified Active Retrieval for Retrieval Augmented Generation
Experiments on four representative types of user instructions show that UAR significantly outperforms existing work on the retrieval timing judgement and the performance of downstream tasks, which shows the effectiveness of UAR and its helpfulness to downstream tasks.
AgentGym: Evolving Large Language Model-based Agents across Diverse Environments
Building generalist agents that can handle diverse tasks and evolve themselves across different environments is a long-term goal in the AI community.
Automatically Identifying Local and Global Circuits with Linear Computation Graphs
Circuit analysis of any certain model behavior is a central task in mechanistic interpretability.
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models
Recent advancements in Chain-of-Thought prompting have facilitated significant breakthroughs for Large Language Models (LLMs) in complex reasoning tasks.
SpeechAlign: Aligning Speech Generation to Human Preferences
However, the integration of human feedback to align speech outputs to human preferences is often neglected.
Calibrating the Confidence of Large Language Models by Eliciting Fidelity
Large language models optimized with techniques like RLHF have achieved good alignment in being helpful and harmless.
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
Pretraining data of large language models composes multiple domains (e. g., web texts, academic papers, codes), whose mixture proportions crucially impact the competence of outcome models.
A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond
Building on our examination of the developmental trajectories, we further investigate the emerging synergies between code intelligence and broader machine intelligence, uncovering new cross-domain opportunities and illustrating the substantial influence of code intelligence across various domains.
Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem
This paper presents a new method for evaluating LLM hallucination in Question Answering (QA) based on the unanswerable math word problem (MWP).
In-Memory Learning: A Declarative Learning Framework for Large Language Models
The exploration of whether agents can align with their environment without relying on human-labeled data presents an intriguing research topic.
Training-Free Long-Context Scaling of Large Language Models
The ability of Large Language Models (LLMs) to process and generate coherent text is markedly weakened when the number of input tokens exceeds their pretraining length.
Enhancing EEG-to-Text Decoding through Transferable Representations from Pre-trained Contrastive EEG-Text Masked Autoencoder
Reconstructing natural language from non-invasive electroencephalography (EEG) holds great promise as a language decoding technology for brain-computer interfaces (BCIs).
Data-freeWeight Compress and Denoise for Large Language Models
Significantly, our method is characterized by without necessitating additional involvement of any corpus, while simultaneously preserving orthogonality in conjunction with pruning and quantization methods.
GAOKAO-MM: A Chinese Human-Level Benchmark for Multimodal Models Evaluation
The Large Vision-Language Models (LVLMs) have demonstrated great abilities in image perception and language understanding.
Balanced Data Sampling for Language Model Training with Clustering
Data plays a fundamental role in the training of Large Language Models (LLMs).
Hint-before-Solving Prompting: Guiding LLMs to Effectively Utilize Encoded Knowledge
Large Language Models (LLMs) have recently showcased remarkable generalizability in various domains.
LongWanjuan: Towards Systematic Measurement for Long Text Quality
The quality of training data are crucial for enhancing the long-text capabilities of foundation models.
Identifying Semantic Induction Heads to Understand In-Context Learning
Although large language models (LLMs) have demonstrated remarkable performance, the lack of transparency in their inference logic raises concerns about their trustworthiness.
Are LLMs Rational Investors? A Study on Detecting and Reducing the Financial Bias in LLMs
Results show varying degrees of financial irrationality among models, influenced by their design and training.
Code Needs Comments: Enhancing Code LLMs with Comment Augmentation
The programming skill is one crucial ability for Large Language Models (LLMs), necessitating a deep understanding of programming languages (PLs) and their correlation with natural languages (NLs).
Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT
Sparse dictionary learning has been a rapidly growing technique in mechanistic interpretability to attack superposition and extract more human-understandable features from model activations.
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling
We introduce AnyGPT, an any-to-any multimodal language model that utilizes discrete representations for the unified processing of various modalities, including speech, text, images, and music.
LLM can Achieve Self-Regulation via Hyperparameter Aware Generation
HAG extends the current paradigm in the text generation process, highlighting the feasibility of endowing the LLMs with self-regulate decoding strategies.
Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
To address the dropped tokens and padding, we propose the Rectify-Router, comprising the Intra-GPU Rectification and the Fill-in Rectification.
InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning
We further explore how to use LEAN to solve math problems and study its performance under the setting of multi-task learning which shows the possibility of using LEAN as a unified platform for solving and proving in math.
MouSi: Poly-Visual-Expert Vision-Language Models
This technique introduces a fusion network to unify the processing of outputs from different visual experts, while bridging the gap between image encoders and pre-trained LLMs.
Ranked #61 on
Visual Question Answering
on MM-Vet
F-Eval: Assessing Fundamental Abilities with Refined Evaluation Methods
However, these evaluation benchmarks are limited to assessing the instruction-following capabilities, overlooking the fundamental abilities that emerge during the pre-training stage.
Query of CC: Unearthing Large Scale Domain-Specific Knowledge from Public Corpora
Large language models have demonstrated remarkable potential in various tasks, however, there remains a significant scarcity of open-source models and data for specific domains.
SpeechGPT-Gen: Scaling Chain-of-Information Speech Generation
It comprises an autoregressive model based on LLM for semantic information modeling and a non-autoregressive model employing flow matching for perceptual information modeling.
DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learning
These methods regularize the representation space by pulling similar sentence representations closer and pushing away the dissimilar ones and have been proven effective in various NLP tasks, e. g., semantic textual similarity (STS) tasks.
Can AI Assistants Know What They Don't Know?
To answer this question, we construct a model-specific "I don't know" (Idk) dataset for an assistant, which contains its known and unknown questions, based on existing open-domain question answering datasets.
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance
With the rapid development of large language models (LLMs), they are not only used as general-purpose AI assistants but are also customized through further fine-tuning to meet the requirements of different applications.
Secrets of RLHF in Large Language Models Part II: Reward Modeling
We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset and fully leverage high-quality preference data.
Agent Alignment in Evolving Social Norms
Agents based on Large Language Models (LLMs) are increasingly permeating various domains of human production and life, highlighting the importance of aligning them with human values.
SpeechAgents: Human-Communication Simulation with Multi-Modal Multi-Agent Systems
In this paper, we propose SpeechAgents, a multi-modal LLM based multi-agent system designed for simulating human communication.
Pixel-level Semantic Correspondence through Layout-aware Representation Learning and Multi-scale Matching Integration
Establishing precise semantic correspondence across object instances in different images is a fundamental and challenging task in computer vision.
A Survey of Reasoning with Foundation Models
Reasoning, a crucial ability for complex problem-solving, plays a pivotal role in various real-world settings such as negotiation, medical diagnosis, and criminal investigation.
Alignment for Honesty
Recent research has made significant strides in applying alignment techniques to enhance the helpfulness and harmlessness of large language models (LLMs) in accordance with human intentions.
Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication
Large Language Models (LLMs) have recently made significant strides in complex reasoning tasks through the Chain-of-Thought technique.
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way
This paper introduces CoLLiE, an efficient library that facilitates collaborative training of large language models using 3D parallelism, parameter-efficient fine-tuning (PEFT) methods, and optimizers such as Lion, Adan, Sophia, LOMO and AdaLomo.
LLatrieval: LLM-Verified Retrieval for Verifiable Generation
Thus, the LLM can iteratively provide feedback to retrieval and facilitate the retrieval result to fully support verifiable generation.
Flames: Benchmarking Value Alignment of LLMs in Chinese
The widespread adoption of large language models (LLMs) across various regions underscores the urgent need to evaluate their alignment with human values.
$R^3$-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQL
While current tasks of converting natural language to SQL (NL2SQL) using Foundation Models have shown impressive achievements, adapting these approaches for converting natural language to Graph Query Language (NL2GQL) encounters hurdles due to the distinct nature of GQL compared to SQL, alongside the diverse forms of GQL.
Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts
As large language models (LLMs) have shown effectiveness with different prompting methods, such as Chain of Thought, Program of Thought, we find that these methods have formed a great complementarity to each other on math reasoning tasks.
Watermarking LLMs with Weight Quantization
Abuse of large language models reveals high risks as large language models are being deployed at an astonishing speed.
AdaLomo: Low-memory Optimization with Adaptive Learning Rate
Our experiments with instruction-tuning and further pre-training demonstrate that AdaLomo achieves results on par with AdamW, while significantly reducing memory requirements, thereby lowering the hardware barrier to training large language models.
Character-LLM: A Trainable Agent for Role-Playing
Large language models (LLMs) can be used to serve as agents to simulate human behaviors, given the powerful ability to understand human instructions and provide high-quality generated texts.
Efficient Link Prediction via GNN Layers Induced by Negative Sampling
Graph neural networks (GNNs) for link prediction can loosely be divided into two broad categories.
PerturbScore: Connecting Discrete and Continuous Perturbations in NLP
Through experimental results, we find that we can build a connection between discrete and continuous perturbations and use the proposed PerturbScore to learn such correlation, surpassing previous methods used in discrete perturbation measuring.
SeqXGPT: Sentence-Level AI-Generated Text Detection
Therefore, it is important to build strong AI-generated text (AIGT) detectors.
Scaling Laws of RoPE-based Extrapolation
The extrapolation capability of Large Language Models (LLMs) based on Rotary Position Embedding is currently a topic of considerable interest.
Evaluating Hallucinations in Chinese Large Language Models
We analyze the primary types of hallucinations in different types of models and their causes.
Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration
Large Language Models (LLMs) are evolving at an unprecedented pace and have exhibited considerable capability in the realm of natural language processing (NLP) with world knowledge.
The Rise and Potential of Large Language Model Based Agents: A Survey
Many efforts have been made to develop intelligent agents, but they mainly focus on advancement in algorithms or training strategies to enhance specific capabilities or performance on particular tasks.
SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
Therefore, we propose SpeechTokenizer, a unified speech tokenizer for speech large language models.
EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education
Currently, EduChat is available online as an open-source project, with its code, data, and model parameters available on platforms (e. g., GitHub https://github. com/icalk-nlp/EduChat, Hugging Face https://huggingface. co/ecnu-icalk ).
Does Correction Remain A Problem For Large Language Models?
As large language models, such as GPT, continue to advance the capabilities of natural language processing (NLP), the question arises: does the problem of correction still persist?
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
Recently, there has been growing interest in extending the context length of large language models (LLMs), aiming to effectively process long inputs of one turn or conversations with more extensive histories.
Secrets of RLHF in Large Language Models Part I: PPO
Therefore, we explore the PPO-max, an advanced version of PPO algorithm, to efficiently improve the training stability of the policy model.
Distributed Marker Representation for Ambiguous Discourse Markers and Entangled Relations
Discourse analysis is an important task because it models intrinsic semantic structures between sentences in a document.
Full Parameter Fine-tuning for Large Language Models with Limited Resources
Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) but demand massive GPU resources for training.
From Hypergraph Energy Functions to Hypergraph Neural Networks
Hypergraphs are a powerful abstraction for representing higher-order interactions between entities of interest.
An AMR-based Link Prediction Approach for Document-level Event Argument Extraction
Motivated by the fact that all event structures can be inferred from AMR, this work reformulates EAE as a link prediction problem on AMR graphs.
Abstract Meaning Representation
Event Argument Extraction
+2
Do Large Language Models Know What They Don't Know?
Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks.
Multijugate Dual Learning for Low-Resource Task-Oriented Dialogue System
Dialogue data in real scenarios tend to be sparsely available, rendering data-starved end-to-end dialogue systems trained inadequately.
Optimizing Non-Autoregressive Transformers with Contrastive Learning
In this paper, we propose to ease the difficulty of modality learning via sampling from the model distribution instead of the data distribution.
Evaluating the Performance of Large Language Models on GAOKAO Benchmark
Large Language Models(LLMs) have demonstrated remarkable performance across various natural language processing tasks; however, how to comprehensively and accurately assess their performance becomes an urgent issue to be addressed.
PromptNER: A Prompting Method for Few-shot Named Entity Recognition via k Nearest Neighbor Search
We use prompts that contains entity category information to construct label prototypes, which enables our model to fine-tune with only the support set.
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities
Multi-modal large language models are regarded as a crucial step towards Artificial General Intelligence (AGI) and have garnered significant interest with the emergence of ChatGPT.
CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors
A common practice is to recast the task into a text-to-text format such that generative LLMs of natural language (NL-LLMs) like GPT-3 can be prompted to solve it.
MoT: Memory-of-Thought Enables ChatGPT to Self-Improve
Specifically, MoT is divided into two stages: 1. before the test stage, the LLM pre-thinks on the unlabeled dataset and saves the high-confidence thoughts as external memory; 2.
Unified Demonstration Retriever for In-Context Learning
To train UDR, we cast various tasks' training signals into a unified list-wise ranking formulation by language model's feedback.
Improving Contrastive Learning of Sentence Embeddings from AI Feedback
Our method utilizes AI feedback from large pre-trained language models (LLMs) to construct sample pairs with fine-grained sample similarity scores to improve contrastive learning.
Origin Tracing and Detecting of LLMs
The extraordinary performance of large language models (LLMs) heightens the importance of detecting whether the context is generated by an AI system.
Finding Support Examples for In-Context Learning
Additionally, the strong dependency among in-context examples makes it an NP-hard combinatorial optimization problem and enumerating all permutations is infeasible.
Rethinking Label Smoothing on Multi-hop Question Answering
Multi-Hop Question Answering (MHQA) is a significant area in question answering, requiring multiple reasoning components, including document retrieval, supporting sentence prediction, and answer span extraction.
Mitigating Negative Style Transfer in Hybrid Dialogue System
As the functionality of dialogue systems evolves, hybrid dialogue systems that accomplish user-specific goals and participate in open-topic chitchat with users are attracting growing attention.
Investigating Glyph Phonetic Information for Chinese Spell Checking: What Works and What's Next
While pre-trained Chinese language models have demonstrated impressive performance on a wide range of NLP tasks, the Chinese Spell Checking (CSC) task remains a challenge.
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models
We present DiffusionBERT, a new generative masked language model based on discrete diffusion models.
Word-Level Representation From Bytes For Language Modeling
Based on this observation, we rethink the existing character-aware method that takes character-level inputs but makes word-level sequence modeling and prediction.
RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Trees
RLET iteratively performs single step reasoning with sentence selection and deduction generation modules, from which the training signal is accumulated across the tree with elaborately designed aligned reward function that is consistent with the evaluation.
SDCL: Self-Distillation Contrastive Learning for Chinese Spell Checking
To adapt BERT to the CSC task, we propose a token-level self-distillation contrastive learning method.
DORE: Document Ordered Relation Extraction based on Generative Framework
In this paper, we investigate the root cause of the underwhelming performance of the existing generative DocRE models and discover that the culprit is the inadequacy of the training paradigm, instead of the capacities of the models.
Is MultiWOZ a Solved Task? An Interactive TOD Evaluation Framework with User Simulator
Besides, we introduce a sentence-level and a session-level score to measure the sentence fluency and session coherence in the interactive evaluation.
Discovering New Intents Using Latent Variables
In this paper, starting from the intuition that discovering intents could be beneficial to the identification of the known intents, we propose a probabilistic framework for discovering intents where intent assignments are treated as latent variables.
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts
Through extensive experimental results across various tasks and PTMs, we show that LPT can achieve competitive performance to full model tuning and other PETuning methods under both full-data and few-shot scenarios while possessing faster training speed and lower memory cost.
Soft-Labeled Contrastive Pre-training for Function-level Code Representation
In this paper, we present \textbf{SCodeR}, a \textbf{S}oft-labeled contrastive pre-training framework with two positive sample construction methods to learn functional-level \textbf{Code} \textbf{R}epresentation.
Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning
MP2 is a set of combinable prompts pre-trained on 38 Chinese tasks.
BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation
Automatic evaluation metrics are crucial to the development of generative systems.
The Open-World Lottery Ticket Hypothesis for OOD Intent Classification
Most existing methods of Out-of-Domain (OOD) intent classification rely on extensive auxiliary OOD corpora or specific training paradigms.
COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization
Traditional training paradigms for extractive and abstractive summarization systems always only use token-level or sentence-level training objectives.
A Unified Generative Framework based on Prompt Learning for Various Information Extraction Tasks
In this paper, we propose a novel composable prompt-based generative framework, which could be applied to a wide range of tasks in the field of Information Extraction.
Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding
Our experiments on 13 benchmark datasets across five natural language understanding tasks demonstrate the superiority of our method.
An Embarrassingly Easy but Strong Baseline for Nested Named Entity Recognition
In this paper, we propose using Convolutional Neural Network (CNN) to model these spatial relations in the score matrix.
Ranked #3 on
Nested Named Entity Recognition
on ACE 2005
CoNT: Contrastive Neural Text Generation
We validate CoNT on five generation tasks with ten benchmarks, including machine translation, summarization, code comment generation, data-to-text generation and commonsense generation.
What Dense Graph Do You Need for Self-Attention?
Transformers have made progress in miscellaneous tasks, but suffer from quadratic computational and memory complexities.
BBTv2: Towards a Gradient-Free Future with Large Language Models
By contrast, gradient-free methods only require the forward computation of the PTM to tune the prompt, retaining the benefits of efficient tuning and deployment.
Dialogue Meaning Representation for Task-Oriented Dialogue Systems
We propose Dialogue Meaning Representation (DMR), a pliable and easily extendable representation for task-oriented dialogue.
Text Adversarial Purification as Defense against Adversarial Attacks
Adversarial purification is a successful defense mechanism against adversarial attacks without requiring knowledge of the form of the incoming attack.
MarkBERT: Marking Word Boundaries Improves Chinese BERT
We present a Chinese BERT model dubbed MarkBERT that uses word information in this work.
Chinese Named Entity Recognition
named-entity-recognition
+7
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation
Previous works usually adopt heuristic metrics such as the entropy of internal outputs to measure instance difficulty, which suffers from generalization and threshold-tuning.
"Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction
The meaning of a word in Chinese is different in that a word is a compositional unit consisting of multiple characters.
$\mathcal{Y}$-Tuning: An Efficient Tuning Paradigm for Large-Scale Pre-Trained Models via Label Representation Learning
With the success of large-scale pre-trained models (PTMs), how efficiently adapting PTMs to downstream tasks has attracted tremendous attention, especially for PTMs with billions of parameters.
TURNER: The Uncertainty-based Retrieval Framework for Chinese NER
Chinese NER is a difficult undertaking due to the ambiguity of Chinese characters and the absence of word boundaries.
CodeRetriever: Unimodal and Bimodal Contrastive Learning for Code Search
For bimodal contrastive learning, we leverage the documentation and in-line comments of code to build code-text pairs.
Towards Collaborative Question Answering: A Preliminary Study
Knowledge and expertise in the real-world can be disjointedly owned.
Black-Box Tuning for Language-Model-as-a-Service
In such a scenario, which we call Language-Model-as-a-Service (LMaaS), the gradients of PTMs are usually unavailable.
Contrast and Generation Make BART a Good Dialogue Emotion Recognizer
Meanwhile, we utilize an auxiliary response generation task to enhance the model's ability of handling context information, thereby forcing the model to recognize emotions with similar semantics in diverse contexts.
Ranked #13 on
Emotion Recognition in Conversation
on EmoryNLP
Towards More Effective and Economic Sparsely-Activated Model
To increase the number of activated experts without an increase in computational cost, we propose SAM (Switch and Mixture) routing, an efficient hierarchical routing mechanism that activates multiple experts in a same device (GPU).
Towards Efficient NLP: A Standard Evaluation and A Strong Baseline
ELUE is dedicated to depict the Pareto Frontier for various language understanding tasks, such that it can tell whether and how much a method achieves Pareto improvement.
KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier
Pre-trained models are widely used in fine-tuning downstream tasks with linear classifiers optimized by the cross-entropy loss, which might face robustness and stability problems.
RetrievalSum: A Retrieval Enhanced Framework for Abstractive Summarization
Existing summarization systems mostly generate summaries purely relying on the content of the source document.
Learning to Teach with Student Feedback
This one-way interaction leads to the teacher's inability to perceive the characteristics of the student and its training progress.
Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning
\textbf{P}re-\textbf{T}rained \textbf{M}odel\textbf{s} have been widely applied and recently proved vulnerable under backdoor attacks: the released pre-trained weights can be maliciously poisoned with certain triggers.
Pre-Trained Models: Past, Present and Future
Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI).
A Unified Generative Framework for Aspect-Based Sentiment Analysis
Aspect-based Sentiment Analysis (ABSA) aims to identify the aspect terms, their corresponding sentiment polarities, and the opinion terms.
Ranked #1 on
Aspect Sentiment Triplet Extraction
on SemEval
Aspect-Based Sentiment Analysis
Aspect-oriented Opinion Extraction
+2
A Survey of Transformers
X-formers) have been proposed, however, a systematic and comprehensive literature review on these Transformer variants is still missing.
A Unified Generative Framework for Various NER Subtasks
To that end, we propose to formulate the NER subtasks as an entity span sequence generation task, which can be solved by a unified sequence-to-sequence (Seq2Seq) framework.
Ranked #11 on
Nested Named Entity Recognition
on GENIA
Accelerating BERT Inference for Sequence Labeling via Early-Exit
To alleviate this problem, we extend the recent successful early-exit mechanism to accelerate the inference of PTMs for sequence labeling tasks.
Early Exiting with Ensemble Internal Classifiers
In this paper, we show that a novel objective function for the training of the ensemble internal classifiers can be naturally induced from the perspective of ensemble learning and information theory.
Keyphrase Generation with Fine-Grained Evaluation-Guided Reinforcement Learning
In response to this problem, we propose a new fine-grained evaluation metric to improve the RL framework, which considers different granularities: token-level $F_1$ score, edit distance, duplication, and prediction quantities.
QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
As increasing numbers of meetings are recorded and transcribed, meeting summaries have become essential to remind those who may or may not have attended the meetings about the key decisions made and the tasks to be completed.
Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa
In this paper, we firstly compare the induced trees from PTMs and the dependency parsing trees on several popular models for the ABSA task, showing that the induced tree from fine-tuned RoBERTa (FT-RoBERTa) outperforms the parser-provided tree.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+1
Enhancing Scientific Papers Summarization with Citation Graph
Previous work for text summarization in scientific domain mainly focused on the content of the input document, but seldom considering its citation network.
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
To guarantee user acceptability, all the text transformations are linguistically based, and we provide a human evaluation for each one.