Search Results for author:
Found 103 papers, 68 papers with code
Flooding-X: Improving BERT’s Resistance to Adversarial Attacks via Loss-Restricted Fine-Tuning
Adversarial robustness has attracted much attention recently, and the mainstream solution is adversarial training.
LFKQG: A Controlled Generation Framework with Local Fine-tuning for Question Generation over Knowledge Bases
Existing KBQG models still face two main challenges: (1) Most models often focus on the most relevant part of the answer entity, while neglecting the rest of the subgraph.
PlugAT: A Plug and Play Module to Defend against Textual Adversarial Attack
To reduce the potential side effects of using defense modules, we further propose a novel forgetting restricted adversarial training, which filters out bad adversarial examples that impair the performance of original ones.
CQG: A Simple and Effective Controlled Generation Framework for Multi-hop Question Generation
CQG employs a simple method to generate the multi-hop questions that contain key entities in multi-hop reasoning chains, which ensure the complexity and quality of the questions.
Making Parameter-efficient Tuning More Efficient: A Unified Framework for Classification Tasks
Specifically, we re-formulate both token and sentence classification tasks into a unified language modeling task, and map label spaces of different tasks into the same vocabulary space.
Read Extensively, Focus Smartly: A Cross-document Semantic Enhancement Method for Visual Documents NER
To deal with this problem, this work proposes a cross-document semantic enhancement method, which consists of two modules: 1) To prevent distractions from irrelevant regions in the current document, we design a learnable attention mask mechanism, which is used to adaptively filter redundant information in the current document.
Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
Experiments show that the method improves the actor's exploration efficiency and solution diversity, especially on challenging queries, leading to a stronger reasoning model.
Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling
Self-improvement methods enable large language models (LLMs) to generate solutions themselves and iteratively train on filtered, high-quality rationales.
Multi-Programming Language Sandbox for LLMs
We introduce MPLSandbox, an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs).
Distill Visual Chart Reasoning Ability from LLMs to MLLMs
Specifically, we employ text-based synthesizing techniques to construct chart-plotting code and produce ReachQA, a dataset containing 3k reasoning-intensive charts and 20k Q&A pairs to enhance both recognition and reasoning abilities.
Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs
In this paper, we aim to investigate these internal mechanisms within the popular Mixture-of-Expert (MoE)-based LLMs and demonstrate how to improve RAG by examining expert activations in these LLMs.
Have the VLMs Lost Confidence? A Study of Sycophancy in VLMs
Our findings indicate that the ability to prevent sycophancy is predominantly observed in higher layers of the model.
RMB: Comprehensively Benchmarking Reward Models in LLM Alignment
However, the current evaluation of RMs may not directly correspond to their alignment performance due to the limited distribution of evaluation data and evaluation methods that are not closely related to alignment objectives.
Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding
However, we argue that this formulation does not adequately convey the complete reading order information in the layout, which may potentially lead to performance decline in downstream VrD tasks.
Ranked #1 on
Key Information Extraction
on CORD
Empirical Insights on Fine-Tuning Large Language Models for Question-Answering
Large language models (LLMs) encode extensive world knowledge through pre-training on massive datasets, which can then be fine-tuned for the question-answering (QA) task.
TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities
Task-oriented dialogue (TOD) systems aim to efficiently handle task-oriented conversations, including information collection.
What's Wrong with Your Code Generated by Large Language Models? An Extensive Study
The increasing development of large language models (LLMs) in code generation has drawn significant attention among researchers.
SafeAligner: Safety Alignment against Jailbreak Attacks via Response Disparity Guidance
As the development of large language models (LLMs) rapidly advances, securing these models effectively without compromising their utility has become a pivotal area of research.
Beyond Boundaries: Learning a Universal Entity Taxonomy across Datasets and Languages for Open Named Entity Recognition
Open Named Entity Recognition (NER), which involves identifying arbitrary types of entities from arbitrary domains, remains challenging for Large Language Models (LLMs).
SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model
To address these limitations, we propose a Safety Preference Alignment dataset for Vision Language Models named SPA-VL.
Toward Optimal LLM Alignments Using Two-Player Games
We theoretically demonstrate that this iterative reinforcement learning optimization converges to a Nash Equilibrium for the game induced by the agents.
AgentGym: Evolving Large Language Model-based Agents across Diverse Environments
Building generalist agents that can handle diverse tasks and evolve themselves across different environments is a long-term goal in the AI community.
MetaRM: Shifted Distributions Alignment via Meta-Learning
These two issues can be united as a challenge posed by the shifted distribution of the environment.
Length Generalization of Causal Transformers without Position Encoding
Generalizing to longer sentences is important for recent Transformer-based language models.
Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models
The generated demos strategically interpolate between existing demos and the given query, transforming the query from OOD to ID.
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals
We first empirically show that the features of either clean signals or adversarial perturbations are redundant and span in low-dimensional linear subspaces respectively with minimal overlap, and the classical low-dimensional subspace projection can suppress perturbation features out of the subspace of clean signals.
EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models
This paper introduces EasyJailbreak, a unified framework simplifying the construction and evaluation of jailbreak attacks against LLMs.
RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions
In this paper, drawing inspiration from recent works that LLMs are sensitive to the design of the instructions, we utilize instructions in code style, which are more structural and less ambiguous, to replace typically natural language instructions.
CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models
Adversarial misuse, particularly through `jailbreaking' that circumvents a model's safety and ethical protocols, poses a significant challenge for Large Language Models (LLMs).
LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition
To overcome the limitations of existing data augmentation methods that compromise semantic integrity and address the uncertainty inherent in LLM-generated text, we leverage the distinctive characteristics of the NER task by augmenting the original data at both the contextual and entity levels.
Unveiling Linguistic Regions in Large Language Models
Furthermore, this core region exhibits significant dimensional dependence, perturbations to even a single parameter on specific dimensions leading to a loss of linguistic competence.
Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysis
Domain adaption has been widely adapted for cross-domain sentiment analysis to transfer knowledge from the source domain to the target domain.
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling
We introduce AnyGPT, an any-to-any multimodal language model that utilizes discrete representations for the unified processing of various modalities, including speech, text, images, and music.
LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration
Large language models (LLMs) have demonstrated impressive performance in understanding language and executing complex reasoning tasks.
Advancing Translation Preference Modeling with RLHF: A Step Towards Cost-Effective Solution
To address this issue, we propose a cost-effective preference learning strategy, optimizing reward models by distinguishing between human and machine translations.
LongHeads: Multi-Head Attention is Secretly a Long Context Processor
Instead of allowing each head to attend to the full sentence, which struggles with generalizing to longer sequences due to out-of-distribution (OOD) issues, we allow each head to process in-distribution length by selecting and attending to important context chunks.
ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages
Tool learning is widely acknowledged as a foundational approach or deploying large language models (LLMs) in real-world scenarios.
Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
In this paper, we propose R$^3$: Learning Reasoning through Reverse Curriculum Reinforcement Learning (RL), a novel method that employs only outcome supervision to achieve the benefits of process supervision for large language models.
Are Large Language Models Good Prompt Optimizers?
Despite the success, the underlying mechanism of this approach remains unexplored, and the true effectiveness of LLMs as Prompt Optimizers requires further validation.
StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
The advancement of large language models (LLMs) has significantly propelled the field of code generation.
Navigating the OverKill in Large Language Models
Large language models are meticulously aligned to be both helpful and harmless.
MouSi: Poly-Visual-Expert Vision-Language Models
This technique introduces a fusion network to unify the processing of outputs from different visual experts, while bridging the gap between image encoders and pre-trained LLMs.
Ranked #93 on
Visual Question Answering
on MM-Vet
From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalities
Multi-modal Large Language Models (MLLMs) have shown impressive abilities in generating reasonable responses with respect to multi-modal contents.
RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning
To bridge this gap, we introduce RoTBench, a multi-level benchmark for evaluating the robustness of LLMs in tool learning.
Secrets of RLHF in Large Language Models Part II: Reward Modeling
We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset and fully leverage high-quality preference data.
LLaMA Beyond English: An Empirical Study on Language Capability Transfer
In recent times, substantial advancements have been witnessed in large language models (LLMs), exemplified by ChatGPT, showcasing remarkable proficiency across a range of complex tasks.
ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios
Furthermore, a sole emphasis on outcomes disregards the intricate capabilities essential for LLMs to effectively utilize tools.
LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks.
LLMEval: A Preliminary Study on How to Evaluate Large Language Models
The three crucial questions for LLM evaluation are ``what, where, and how to evaluate''.
Rescue: Ranking LLM Responses with Partial Ordering to Improve Response Generation
This method trains the model to prioritize the best responses from a pool of candidates created for a particular task.
Making Harmful Behaviors Unlearnable for Large Language Models
Specifically, we introduce ``security vectors'', a few new parameters that can be separated from the LLM, to ensure LLM's responses are consistent with the harmful behavior.
Unveiling A Core Linguistic Region in Large Language Models
We have discovered a core region in LLMs that corresponds to linguistic competence, accounting for approximately 1% of the total model parameters.
Orthogonal Subspace Learning for Language Model Continual Learning
In this paper, we propose orthogonal low-rank adaptation (O-LoRA), a simple and efficient approach for continual learning in language models, effectively mitigating catastrophic forgetting while learning new tasks.
Improving Generalization of Alignment with Human Preferences through Group Invariant Learning
In this work, we propose a novel approach that can learn a consistent policy via RL across various data groups or domains.
Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction
However, BIO-tagging scheme relies on the correct order of model inputs, which is not guaranteed in real-world NER on scanned VrDs where text are recognized and arranged by OCR systems.
Ranked #1 on
Reading Order Detection
on ReadingBank
RealBehavior: A Framework for Faithfully Characterizing Foundation Models' Human-like Behavior Mechanisms
Reports of human-like behaviors in foundation models are growing, with psychological theories providing enduring tools to investigate these behaviors.
RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification
Recently, Target-oriented Multimodal Sentiment Classification (TMSC) has gained significant attention among scholars.
TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
In this paper, we introduce TRACE, a novel benchmark designed to evaluate continual learning in LLMs.
Universal Multi-modal Entity Alignment via Iteratively Fusing Modality Similarity Paths
The objective of Entity Alignment (EA) is to identify equivalent entity pairs from multiple Knowledge Graphs (KGs) and create a more comprehensive and unified KG.
Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback serves as a crucial bridge, aligning large language models with human and societal values.
The Rise and Potential of Large Language Model Based Agents: A Survey
Many efforts have been made to develop intelligent agents, but they mainly focus on advancement in algorithms or training strategies to enhance specific capabilities or performance on particular tasks.
Secrets of RLHF in Large Language Models Part I: PPO
Therefore, we explore the PPO-max, an advanced version of PPO algorithm, to efficiently improve the training stability of the policy model.
Open Set Relation Extraction via Unknown-Aware Training
Inspired by text adversarial attacks, we adaptively apply small but critical perturbations to original training instances and thus synthesizing negative instances that are more likely to be mistaken by the model as known relations.
Actively Supervised Clustering for Open Relation Extraction
The key to the setting is selecting which instances to label.
RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction
However, general matching methods lack explicit modeling of the above matching pattern.
Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement
To enhance the multi-step reasoning capabilities of large language models, researchers have extensively explored prompting methods, notably the Chain-of-Thought (CoT) method which explicitly elicits human-like rationales.
Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model
Pretrained language models have achieved remarkable success in various natural language processing tasks.
A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition
Particularly, we propose a Confidence-based Partial Label Learning (CPLL) method to integrate the prior confidence (given by annotators) and posterior confidences (learned by models) for crowd-annotated NER.
Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization
Models trained with empirical risk minimization (ERM) are revealed to easily rely on spurious correlations, resulting in poor generalization.
Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild
The principle of continual relation extraction~(CRE) involves adapting to emerging novel relations while preserving od knowledge.
InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction
Large language models have unlocked strong multi-task capabilities from reading instructive prompts.
Ranked #3 on
Zero-shot Named Entity Recognition (NER)
on CrossNER
(using extra training data)
A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models
GPT series models, such as GPT-3, CodeX, InstructGPT, ChatGPT, and so on, have gained considerable attention due to their exceptional natural language processing capabilities.
How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks
The GPT-3. 5 models have demonstrated impressive performance in various Natural Language Processing (NLP) tasks, showcasing their strong understanding and reasoning capabilities.
Natural Language Inference
Natural Language Understanding
+1
Correspondence Transformers With Asymmetric Feature Learning and Matching Flow Super-Resolution
The asymmetric feature learning module exploits a biased cross-attention mechanism to encode token features of source images with their target counterparts.
Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer?
We demonstrate that the distance between the distributions of different languages is highly consistent with the syntactic difference in terms of linguistic formalisms.
Efficient Adversarial Training with Robust Early-Bird Tickets
Adversarial training is one of the most powerful methods to improve the robustness of pre-trained language models (PLMs).
Towards Understanding Omission in Dialogue Summarization
By analyzing this dataset, we find that a large improvement in summarization quality can be achieved by providing ground-truth omission labels for the summarization model to recover omission information, which demonstrates the importance of omission detection for omission mitigation in dialogue summarization.
Robust Lottery Tickets for Pre-trained Language Models
Recent works on Lottery Ticket Hypothesis have shown that pre-trained language models (PLMs) contain smaller matching subnetworks(winning tickets) which are capable of reaching accuracy comparable to the original models.
Learning "O" Helps for Learning More: Handling the Concealed Entity Problem for Class-incremental NER
In this work, we conduct an empirical study on the "Unlabeled Entity Problem" and find that it leads to severe confusion between "O" and entities, decreasing class discrimination of old classes and declining the model's ability to learn new classes.
Less is Better: Recovering Intended-Feature Subspace to Robustify NLU Models
When delving into a lower manifold to remove redundancies, RISK reveals that an extremely low-dimensional subspace with intended features can robustly represent the highly biased dataset.
Causal Intervention Improves Implicit Sentiment Analysis
In this work, we propose a causal intervention model for Implicit Sentiment Analysis using Instrumental Variable (ISAIV).
Searching for Optimal Subword Tokenization in Cross-domain NER
Input distribution shift is one of the vital problems in unsupervised domain adaptation (UDA).
MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective
NER model has achieved promising performance on standard NER benchmarks.
Ranked #9 on
Named Entity Recognition (NER)
on WNUT 2017
Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents
Text semantic matching is a fundamental task that has been widely used in various scenarios, such as community question answering, information retrieval, and recommendation.
Plug-Tagger: A Pluggable Sequence Labeling Framework Using Language Models
Specifically, for each task, a label word set is first constructed by selecting a high-frequency word for each class respectively, and then, task-specific vectors are inserted into the inputs and optimized to manipulate the model predictions towards the corresponding label words.
Template-free Prompt Tuning for Few-shot NER
Prompt-based methods have been successfully applied in sentence-level few-shot learning tasks, mostly owing to the sophisticated design of templates and label words.
A Relation-Oriented Clustering Method for Open Relation Extraction
The clustering-based unsupervised relation discovery method has gradually become one of the important methods of open relation extraction (OpenRE).
Ranked #1 on
Relation Extraction
on FewRel
Heterogeneous Graph Neural Networks for Keyphrase Generation
The encoder-decoder framework achieves state-of-the-art results in keyphrase generation (KG) tasks by predicting both present keyphrases that appear in the source document and absent keyphrases that do not.
Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining
With the rapid increase in the volume of dialogue data from daily life, there is a growing demand for dialogue summarization.
SENT: Sentence-level Distant Relation Extraction via Negative Training
In this work, we propose the use of negative training (NT), in which a model is trained using complementary labels regarding that ``the instance does not belong to these complementary labels".
A Unified Generative Framework for Various NER Subtasks
To that end, we propose to formulate the NER subtasks as an entity span sequence generation task, which can be solved by a unified sequence-to-sequence (Seq2Seq) framework.
Ranked #11 on
Nested Named Entity Recognition
on GENIA
One2Set: Generating Diverse Keyphrases as a Set
In this work, we propose a new training paradigm One2Set without predefining an order to concatenate the keyphrases.
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
To guarantee user acceptability, all the text transformations are linguistically based, and we provide a human evaluation for each one.
Uncertainty-Aware Label Refinement for Sequence Labeling
Conditional random fields (CRF) for label decoding has become ubiquitous in sequence labeling tasks.
Constructing Multiple Tasks for Augmentation: Improving Neural Image Classification With K-means Features
In order to effectively reduce the impact of non-ideal auxiliary tasks on the main task, we further proposed a novel meta-learning-based multi-task learning approach, which trained the shared hidden layers on auxiliary tasks, while the meta-optimization objective was to minimize the loss on the main task, ensuring that the optimizing direction led to an improvement on the main task.
A Lexicon-Based Graph Neural Network for Chinese NER
Recurrent neural networks (RNN) used for Chinese named entity recognition (NER) that sequentially track character and word information have achieved great success.
Ranked #13 on
Chinese Named Entity Recognition
on OntoNotes 4
Learning Task-specific Representation for Novel Words in Sequence Labeling
However, representations of unseen or rare words trained on the end task are usually poor for appreciable performance.
Switch-LSTMs for Multi-Criteria Chinese Word Segmentation
With these auto-switched LSTMs, our model provides a more flexible solution for multi-criteria CWS, which is also easy to transfer the learned knowledge to new criteria.
Long Short-Term Memory with Dynamic Skip Connections
In recent years, long short-term memory (LSTM) has been successfully used to model sequential data of variable length.
Ranked #35 on
Sentiment Analysis
on IMDb
Transferring from Formal Newswire Domain with Hypernet for Twitter POS Tagging
However, from a linguistic perspective, Twitter users not only tend to mimic the formal expressions of traditional media, like news, but they also appear to be developing linguistically informal styles.
Ranked #2 on
Part-Of-Speech Tagging
on Ritter
A Lexicon-Based Supervised Attention Model for Neural Sentiment Analysis
Attention mechanisms have been leveraged for sentiment classification tasks because not all words have the same importance.
Part-of-Speech Tagging for Twitter with Adversarial Neural Networks
In this work, we study the problem of part-of-speech tagging for Tweets.
Ranked #3 on
Part-Of-Speech Tagging
on Ritter
