Search Results for author:
Found 98 papers, 42 papers with code
A Spectral Method for Unsupervised Multi-Document Summarization
Multi-document summarization (MDS) aims at producing a good-quality summary for several related documents.
Large Language Models Are Unconscious of Unreasonability in Math Problems
Large language models (LLMs) demonstrate substantial capabilities in solving math problems.
Reducing Hallucinations in Entity Abstract Summarization with Facts-Template Decomposition
Entity abstract summarization aims to generate a coherent description of a given entity based on a set of relevant Internet documents.
ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors
The safety of Large Language Models (LLMs) has gained increasing attention in recent years, but there still lacks a comprehensive approach for detecting safety issues within LLMs' responses in an aligned, customizable and explainable manner.
PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization
Although LoRA fine-tuning is effective, there is still a performance gap compared to full fine-tuning, since its weight update is limited to low-rank matrices.
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
We introduce Generalized Instruction Tuning (called GLAN), a general and scalable method for instruction tuning of Large Language Models (LLMs).
Can Large Multimodal Models Uncover Deep Semantics Behind Images?
Understanding the deep semantics of images is essential in the era dominated by social media.
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding
To mitigate the high inference latency stemming from autoregressive decoding in Large Language Models (LLMs), Speculative Decoding has emerged as a novel decoding paradigm for LLM inference.
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Subsequently, we scale up DeepSeekMoE to 16B parameters and show that it achieves comparable performance with LLaMA2 7B, with only about 40% of computations.
Language Models Understand Numbers, at Least Partially
Large language models (LLMs) have exhibited impressive competence in various tasks, but their opaque internal mechanisms hinder their use in mathematical problems.
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
In this paper, we present an innovative process-oriented math process reward model called \textbf{Math-Shepherd}, which assigns a reward score to each step of math problem solutions.
Ranked #13 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Guiding AMR Parsing with Reverse Graph Linearization
Abstract Meaning Representation (AMR) parsing aims to extract an abstract semantic graph from a given sentence.
Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning
To assess the quality of weights in the absence of additional validation data, we design a masked self-prediction (MSP) score that exhibits a strong correlation with the final ICL performance.
InfoCL: Alleviating Catastrophic Forgetting in Continual Text Classification from An Information Theoretic Perspective
Continual learning (CL) aims to constantly learn new knowledge over time while avoiding catastrophic forgetting on old tasks.
Large Language Model for Science: A Study on P vs. NP
In this work, we use large language models (LLMs) to augment and accelerate research on the P versus NP problem, one of the most important open problems in theoretical computer science and mathematics.
Making Large Language Models Better Reasoners with Alignment
To address this problem, we introduce an \textit{Alignment Fine-Tuning (AFT)} paradigm, which involves three steps: 1) fine-tuning LLMs with COT training data; 2) generating multiple COT responses for each question, and categorizing them into positive and negative ones based on whether they achieve the correct answer; 3) calibrating the scores of positive and negative responses given by LLMs with a novel constraint alignment loss.
Large Language Models are not Fair Evaluators
In this paper, we uncover a systematic bias in the evaluation paradigm of adopting large language models~(LLMs), e. g., GPT-4, as a referee to score and compare the quality of responses generated by candidate models.
Learn to Not Link: Exploring NIL Prediction in Entity Linking
We conduct a series of experiments with the widely used bi-encoder and cross-encoder entity linking models, results show that both types of NIL mentions in training data have a significant influence on the accuracy of NIL prediction.
Denoising Bottleneck with Mutual Information Maximization for Video Multimodal Fusion
However, unlike other image-text multimodal tasks, video has longer multimodal sequences with more redundancy and noise in both visual and audio modalities.
ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories
Recently, Large Language Models (LLMs) have been serving as general-purpose interfaces, posing a significant demand for comprehensive visual knowledge.
Bi-Drop: Enhancing Fine-tuning Generalization via Synchronous sub-net Estimation and Optimization
Also, Bi-Drop needs only one mini-batch to estimate the sub-net so it achieves higher utility of training data.
RepCL: Exploring Effective Representation for Continual Text Classification
Continual learning (CL) aims to constantly learn new knowledge over time while avoiding catastrophic forgetting on old tasks.
Enhancing Continual Relation Extraction via Classifier Decomposition
In this work, we point out that there exist two typical biases after training of this vanilla strategy: classifier bias and representation bias, which causes the previous knowledge that the model learned to be shaded.
A Survey on In-context Learning
With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few examples.
Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers
We comprehensively compare the behaviors of in-context learning and explicit finetuning on real tasks to provide empirical evidence that supports our understanding.
Statistical Dataset Evaluation: Reliability, Difficulty, and Validity
Taking the Named Entity Recognition (NER) datasets as a case study, we introduce $9$ statistical metrics for a statistical dataset evaluation framework.
DialogQAE: N-to-N Question Answer Pair Extraction from Customer Service Chatlog
Harvesting question-answer (QA) pairs from customer service chatlog in the wild is an efficient way to enrich the knowledge base for customer service chatbots in the cold start or continuous integration scenarios.
DialogUSR: Complex Dialogue Utterance Splitting and Reformulation for Multiple Intent Detection
While interacting with chatbots, users may elicit multiple intents in a single dialogue utterance.
Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentation
In this paper, through empirical studies we argue that this assumption may not hold, and an important reason for catastrophic forgetting is that the learned representations do not have good robustness against the appearance of analogous relations in the subsequent learning process.
Calibrating Factual Knowledge in Pretrained Language Models
However, we find that facts stored in the PLMs are not always correct.
Less is More: Rethinking State-of-the-art Continual Relation Extraction Models with a Frustratingly Easy but Effective Approach
2) Balanced Tuning (BT) finetunes the model on the balanced memory data.
Neural Knowledge Bank for Pretrained Transformers
The ability of pretrained Transformers to remember factual knowledge is essential but still limited for existing models.
Robust Fine-tuning via Perturbation and Interpolation from In-batch Instances
For each instance in a batch, we involve other instances in the same batch to interact with it.
A Two-Stream AMR-enhanced Model for Document-level Event Argument Extraction
In this paper, we focus on extracting event arguments from an entire document, which mainly faces two critical problems: a) the long-distance dependency between trigger and arguments over sentences; b) the distracting context towards an event in the document.
Document-level Event Extraction
Event Argument Extraction
+2
HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classification
However, in this paradigm, there exists a huge gap between the classification tasks with sophisticated label hierarchy and the masked language model (MLM) pretraining tasks of PLMs and thus the potentials of PLMs can not be fully tapped.
ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs
As Abstract Meaning Representation (AMR) implicitly involves compound semantic annotations, we hypothesize auxiliary tasks which are semantically or formally related can better enhance AMR parsing.
Ranked #7 on
AMR Parsing
on LDC2020T02
(using extra training data)
StableMoE: Stable Routing Strategy for Mixture of Experts
We point out that existing learning-to-route MoE methods suffer from the routing fluctuation issue, i. e., the target expert of the same input may change along with training, but only one expert will be activated for the input during inference.
Mixture of Experts for Biomedical Question Answering
In this paper, in order to alleviate the parameter competition problem, we propose a Mixture-of-Expert (MoE) based question answering method called MoEBQA that decouples the computation for different types of questions by sparse routing.
Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation
We propose Speculative Decoding (SpecDec), for the first time ever, to formally study exploiting the idea of speculative execution to accelerate autoregressive (AR) decoding.
A Roadmap for Big Model
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
CUGE: A Chinese Language Understanding and Generation Evaluation Benchmark
We argue that for general-purpose language intelligence evaluation, the benchmark itself needs to be comprehensive and systematic.
Hierarchical Curriculum Learning for AMR Parsing
Abstract Meaning Representation (AMR) parsing aims to translate sentences to semantic representation with a hierarchical structure, and is recently empowered by pretrained sequence-to-sequence models.
An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labeling
Few-Shot Sequence Labeling (FSSL) is a canonical paradigm for the tagging models, e. g., named entity recognition and slot filling, to generalize on an emerging, resource-scarce domain.
Ranked #6 on
Few-shot NER
on Few-NERD (INTER)
Behind the Scenes: An Exploration of Trigger Biases Problem in Few-Shot Event Classification
However, we find they suffer from trigger biases that signify the statistical homogeneity between some trigger words and target event types, which we summarize as trigger overlapping and trigger separability.
Explicit Interaction Network for Aspect Sentiment Triplet Extraction
Table encoder extracts sentiment at token-pair level, so that the compositional feature between targets and opinions can be easily captured.
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark
Artificial Intelligence (AI), along with the recent progress in biomedical language understanding, is gradually changing medical practice.
Ranked #1 on
Semantic Similarity
on CHIP-STS
Decompose, Fuse and Generate: A Formation-Informed Method for Chinese Definition Generation
In this paper, we tackle the task of Definition Generation (DG) in Chinese, which aims at automatically generating a definition for a word.
Problems and Countermeasures in Natural Language Processing Evaluation
Starting from the concept, com-position, development and meaning of natural language evaluation, this article classifies and summarizes the tasks and char-acteristics of mainstream natural language evaluation, and then summarizes the problems and causes of natural language pro-cessing evaluation.
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation
Large pretrained generative models like GPT-3 often suffer from hallucinating non-existent or incorrect content, which undermines their potential merits in real applications.
Knowledge Neurons in Pretrained Transformers
In this paper, we present preliminary studies on how factual knowledge is stored in pretrained Transformers by introducing the concept of knowledge neurons.
Incorporating Connections Beyond Knowledge Embeddings: A Plug-and-Play Module to Enhance Commonsense Reasoning in Machine Reading Comprehension
Conventional Machine Reading Comprehension (MRC) has been well-addressed by pattern matching, but the ability of commonsense reasoning remains a gap between humans and machines.
Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View
In open domain table-to-text generation, we notice that the unfaithful generation usually contains hallucinated content which can not be aligned to any input table record.
Coarse-to-Fine Entity Representations for Document-level Relation Extraction
In classification, we combine the entity representations from both two levels into more comprehensive representations for relation extraction.
Ranked #34 on
Relation Extraction
on DocRED
An Anchor-Based Automatic Evaluation Metric for Document Summarization
The widespread adoption of reference-based automatic evaluation metrics such as ROUGE has promoted the development of document summarization.
Discriminatively-Tuned Generative Classifiers for Robust Natural Language Inference
We explore training objectives for discriminative fine-tuning of our generative classifiers, showing improvements over log loss fine-tuning from prior work .
An Empirical Study on Model-agnostic Debiasing Strategies for Robust Natural Language Inference
The prior work on natural language inference (NLI) debiasing mainly targets at one or few known biases while not necessarily making the models more robust.
Inductively Representing Out-of-Knowledge-Graph Entities by Optimal Estimation Under Translational Assumptions
Conventional Knowledge Graph Completion (KGC) assumes that all test entities appear during training.
HypoNLI: Exploring the Artificial Patterns of Hypothesis-only Bias in Natural Language Inference
Many recent studies have shown that for models trained on datasets for natural language inference (NLI), it is possible to make correct predictions by merely looking at the hypothesis while completely ignoring the premise.
XGPT: Cross-modal Generative Pre-Training for Image Captioning
While many BERT-based cross-modal pre-trained models produce excellent results on downstream understanding tasks like image-text retrieval and VQA, they cannot be applied to generation tasks directly.
Multi-View Learning for Vision-and-Language Navigation
Learning to navigate in a visual environment following natural language instructions is a challenging task because natural language instructions are highly variable, ambiguous, and under-specified.
Pun-GAN: Generative Adversarial Network for Pun Generation
It consists of a generator to produce pun sentences, and a discriminator to distinguish between the generated pun sentences and the real sentences with specific word senses.
Towards Fine-grained Text Sentiment Transfer
In this paper, we focus on the task of fine-grained text sentiment transfer (FGST).
Towards Comprehensive Description Generation from Factual Attribute-value Tables
To relieve these problems, we first propose force attention (FA) method to encourage the generator to pay more attention to the uncovered attributes to avoid potential key attributes missing.
Learning to Control the Fine-grained Sentiment for Story Ending Generation
Therefore, we propose a generic and novel framework which consists of a sentiment analyzer and a sentimental generator, respectively addressing the two challenges.
A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer
Therefore, in this paper, we propose a dual reinforcement learning framework to directly transfer the style of the text via a one-step mapping model, without any separation of content and style.
Ranked #1 on
Unsupervised Text Style Transfer
on GYAFC
Fine-grained Coordinated Cross-lingual Text Stream Alignment for Endless Language Knowledge Acquisition
This paper proposes to study fine-grained coordinated cross-lingual text stream alignment through a novel information network decipherment paradigm.
Leveraging Gloss Knowledge in Neural Word Sense Disambiguation by Hierarchical Co-Attention
The goal of Word Sense Disambiguation (WSD) is to identify the correct meaning of a word in the particular context.
Incorporating Glosses into Neural Word Sense Disambiguation
GAS models the semantic relationship between the context and the gloss in an improved memory network framework, which breaks the barriers of the previous supervised methods and knowledge-based methods.
Ranked #3 on
Word Sense Disambiguation
on SemEval 2015 Task 13
Table-to-text Generation by Structure-aware Seq2seq Learning
In the decoding phase, dual attention mechanism which contains word level attention and field level attention is proposed to model the semantic relevance between the generated description and the table.
Ranked #1 on
Table-to-Text Generation
on WikiBio
A Soft-label Method for Noise-tolerant Distantly Supervised Relation Extraction
Distant-supervised relation extraction inevitably suffers from wrong labeling problems because it heuristically labels relational facts with knowledge bases.
Affinity-Preserving Random Walk for Multi-Document Summarization
Multi-document summarization provides users with a short text that summarizes the information in a set of related documents.
Order-Planning Neural Text Generation From Structured Data
Generating texts from structured data (e. g., a table) is important for various natural language processing tasks such as question answering and dialog systems.
A Progressive Learning Approach to Chinese SRL Using Heterogeneous Data
But the training data of single corpus is often limited.
Improving Chinese SRL with Heterogeneous Annotations
Previous studies on Chinese semantic role labeling (SRL) have concentrated on single semantically annotated corpus.
Reading and Thinking: Re-read LSTM Unit for Textual Entailment Recognition
After read the premise again, the model can get a better understanding of the premise, which can also affect the understanding of the hypothesis.
Ranked #42 on
Natural Language Inference
on SNLI
Towards Time-Aware Knowledge Graph Completion
In this paper, we present a novel time-aware knowledge graph completion model that is able to predict links in a KG using both the existing facts and the temporal information of the facts.
Event Detection with Burst Information Networks
Retrospective event detection is an important task for discovering previously unidentified events in a text stream.
Aligning Coordinated Text Streams through Burst Information Network Construction and Decipherment
We introduce a novel Burst Information Network (BINet) representation that can display the most important information and illustrate the connections among bursty entities, events and keywords in the corpus.
Implicit Discourse Relation Classification via Multi-Task Neural Networks
Without discourse connectives, classifying implicit discourse relations is a challenging task and a bottleneck for building a practical discourse parser.
Joint Learning Templates and Slots for Event Schema Induction
Automatic event schema induction (AESI) means to extract meta-event from raw text, in other words, to find out what types (templates) of event may exist in the raw text and what roles (slots) may exist in each event type.
