Search Results for author:
Found 18 papers, 10 papers with code
Gradient-Based Adversarial Factual Consistency Evaluation for Abstractive Summarization
Based on the artificial dataset, we train an evaluation model that can not only make accurate and robust factual consistency discrimination but is also capable of making interpretable factual errors tracing by backpropagated gradient distribution on token embeddings.
Revisit Overconfidence for OOD Detection: Reassigned Contrastive Learning with Adaptive Class-dependent Threshold
Detecting Out-of-Domain (OOD) or unknown intents from user queries is essential in a task-oriented dialog system.
Disentangled Knowledge Transfer for OOD Intent Discovery with Unified Contrastive Learning
Discovering Out-of-Domain(OOD) intents is essential for developing new skills in a task-oriented dialogue system.
WanJuan-CC: A Safe and High-Quality Open-sourced English Webtext Dataset
To evaluate the quality and utility of the dataset, we trained 1B-parameter and 3B-parameter models using WanJuan-CC and another dataset, RefinedWeb.
Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
To address the dropped tokens and padding, we propose the Rectify-Router, comprising the Intra-GPU Rectification and the Fill-in Rectification.
Query of CC: Unearthing Large Scale Domain-Specific Knowledge from Public Corpora
Large language models have demonstrated remarkable potential in various tasks, however, there remains a significant scarcity of open-source models and data for specific domains.
Evaluating Large Language Models at Evaluating Instruction Following
As research in large language models (LLMs) continues to accelerate, LLM-based evaluation has emerged as a scalable and cost-effective alternative to human evaluations for comparing the ever increasing list of models.
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
In this work, we study structured pruning as an effective means to develop smaller LLMs from pre-trained, larger models.
Ranked #40 on
Question Answering
on PIQA
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Experimental results on three knowledge-driven NLP tasks show that existing injection methods are not suitable for the new paradigm, while map-tuning effectively improves the performance of downstream models.
Emergent Modularity in Pre-trained Transformers
In analogy to human brains, we consider two main characteristics of modularity: (1) functional specialization of neurons: we evaluate whether each neuron is mainly specialized in a certain function, and find that the answer is yes.
KNIFE: Distilling Reasoning Knowledge From Free-Text Rationales
First, KNIFE finetunes a teacher LM (given task input and FTR) to predict the task output, transferring reasoning knowledge from the FTRs to the teacher's hidden states.
Disentangling Confidence Score Distribution for Out-of-Domain Intent Detection with Energy-Based Learning
In this paper, we propose a simple but strong energy-based score function to detect OOD where the energy scores of OOD samples are higher than IND samples.
Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation
Out-of-Domain (OOD) detection is a key component in a task-oriented dialog system, which aims to identify whether a query falls outside the predefined supported intent set.
Unsupervised and Few-shot Parsing from Pretrained Language Models
We therefore extend the unsupervised models to few-shot parsing models (FPOA, FPIO) that use a few annotated trees to learn better linear projection matrices for parsing.
An Empirical Study on Adversarial Attack on NMT: Languages and Positions Matter
For autoregressive NMT models that generate target words from left to right, we observe that adversarial attack on the source language is more effective than on the target language, and that attacking front positions of target sentences or positions of source sentences aligned to the front positions of corresponding target sentences is more effective than attacking other positions.
Adversarial Self-Supervised Learning for Out-of-Domain Detection
Detecting out-of-domain (OOD) intents is crucial for the deployed task-oriented dialogue system.
Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System
Existing slot filling models can only recognize pre-defined in-domain slot types from a limited slot set.
Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning
Detecting Out-of-Domain (OOD) or unknown intents from user queries is essential in a task-oriented dialog system.
