Search Results for author:
Found 49 papers, 21 papers with code
Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs
Our exploration demonstrates that Ascend NPUs are capable of efficiently and effectively training dense models with more than 100 billion parameters.
DAST: Difficulty-Aware Self-Training on Large Language Models
Present Large Language Models (LLM) self-training methods always under-sample on challenging queries, leading to inadequate learning on difficult problems which limits LLMs' ability.
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models
Despite demonstrating impressive capabilities, Large Language Models (LLMs) still often struggle to accurately express the factual knowledge they possess, especially in cases where the LLMs' knowledge boundaries are ambiguous.
CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference
To the best of our knowledge, we are the first to study LLM safety in multi-turn dialogue coreference.
Dynamic Stochastic Decoding Strategy for Open-Domain Dialogue Generation
In DDS, both sequence-level and token-level adaptive search can be achieved to adjust the decoding process in a unified framework.
Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment
As the capabilities of large language models (LLMs) have expanded dramatically, aligning these models with human values presents a significant challenge.
Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Models
2) Role Prompting assigns a central prompt to the general domain and a unique role prompt to each specific domain to minimize inter-domain confusion during training.
UniRetriever: Multi-task Candidates Selection for Various Context-Adaptive Conversational Retrieval
Conversational retrieval refers to an information retrieval system that operates in an iterative and interactive manner, requiring the retrieval of various external resources, such as persona, knowledge, and even response, to effectively engage with the user and successfully complete the dialogue.
YODA: Teacher-Student Progressive Learning for Language Models
With the teacher's guidance, the student learns to iteratively refine its answer with feedback, and forms a robust and comprehensive understanding of the posed questions.
UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems
To better plan and incorporate the use of multiple sources in generating personalized response, we firstly decompose it into three sub-tasks: Knowledge Source Selection, Knowledge Retrieval, and Response Generation.
Data Management For Training Large Language Models: A Survey
This survey aims to provide a comprehensive overview of current research in data management within both the pretraining and supervised fine-tuning stages of LLMs, covering various aspects of data management strategy design.
Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization
While significant attention has been dedicated to exploiting weaknesses in LLMs through jailbreaking attacks, there remains a paucity of effort in defending against these attacks.
FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models
To fill this research gap, in this paper, we propose FollowBench, a Multi-level Fine-grained Constraints Following Benchmark for LLMs.
Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis
The rapid development of large language models (LLMs) has not only provided numerous opportunities but also presented significant challenges.
Large Language Models as Source Planner for Personalized Knowledge-grounded Dialogue
Open-domain dialogue system usually requires different sources of knowledge to generate more informative and evidential responses.
Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignment
Inspired by previous work which identified that feed-forward networks (FFNs) within Transformers are responsible for factual knowledge expressions, we investigate two methods to efficiently improve the factual expression capability {of FFNs} by knowledge enhancement and alignment respectively.
SELF: Self-Evolution with Language Feedback
SELF initiates with a meta-skill learning process that equips the LLMs with capabilities for self-feedback and self-refinement.
TPE: Towards Better Compositional Reasoning over Conceptual Tools with Multi-persona Collaboration
Large language models (LLMs) have demonstrated exceptional performance in planning the use of various functional tools, such as calculators and retrievers, particularly in question-answering tasks.
Aligning Large Language Models with Human: A Survey
(2) Training methodologies: a detailed review of the prevailing training methods employed for LLM alignment.
DecompEval: Evaluating Generated Texts as Unsupervised Decomposed Question Answering
Existing evaluation metrics for natural language generation (NLG) tasks face the challenges on generalization ability and interpretability.
ReSee: Responding through Seeing Fine-grained Visual Knowledge in Open-domain Dialogue
To demonstrate the superiority and universality of the provided visual knowledge, we propose a simple but effective framework ReSee to add visual representation into vanilla dialogue models by modality concatenations.
Enhancing Large Language Models Against Inductive Instructions with Dual-critique Prompting
Numerous works are proposed to align large language models (LLMs) with human intents to better fulfill instructions, ensuring they are trustful and helpful.
Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs
Large Language Models (LLMs), such as \texttt{ChatGPT}, greatly empower dialogue systems with strong language understanding and generation capabilities.
MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions
In this paper, we propose a framework, MoralDial to train and evaluate moral dialogue systems.
KPT: Keyword-guided Pre-training for Grounded Dialog Generation
For the former, the grounding knowledge consists of keywords extracted from the response.
Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation
In order to detect such toxic generations, existing methods rely on templates, real-world data extraction, crowdsourcing workers, or automatic generation to construct adversarial contexts that are likely to induce toxic generations.
Towards Diverse, Relevant and Coherent Open-Domain Dialogue Generation via Hybrid Latent Variables
Specifically, HLV constrains the global semantics of responses through discrete latent variables and enriches responses with continuous latent variables.
Modeling Complex Dialogue Mappings via Sentence Semantic Segmentation Guided Conditional Variational Auto-Encoder
Complex dialogue mappings (CDM), including one-to-many and many-to-one mappings, tend to make dialogue models generate incoherent or dull responses, and modeling these mappings remains a huge challenge for neural dialogue systems.
Exploring Effective Information Utilization in Multi-Turn Topic-Driven Conversations
In order to expand the information that PLMs can utilize, we encode topic and dialogue history information using certain prompts with multiple channels of Fusion-in-Decoder (FiD) and explore the influence of three different channel settings.
PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained Language Model
We investigate different aspects of responses generated by PanGu-Bot, including response quality, knowledge, and safety.
Continual Prompt Tuning for Dialog State Tracking
A desirable dialog system should be able to continually learn new skills without forgetting old ones, and thereby adapt to new domains or tasks in its life cycle.
Compilable Neural Code Generation with Compiler Feedback
Automatically generating compilable programs with (or without) natural language descriptions has always been a touchstone problem for computational linguistics and automated software engineering.
Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarks
The research of open-domain dialog systems has been greatly prospered by neural models trained on large-scale corpora, however, such corpora often introduce various safety problems (e. g., offensive languages, biases, and toxic behaviors) that significantly hinder the deployment of dialog systems in practice.
Pan More Gold from the Sand: Refining Open-domain Dialogue Training with Noisy Self-Retrieval Generation
Experiments over publicly available datasets demonstrate that our method can help models generate better responses, even such training data are usually impressed as low-quality data.
COLD: A Benchmark for Chinese Offensive Language Detection
To this end, we propose a benchmark --COLD for Chinese offensive language analysis, including a Chinese Offensive Language Dataset --COLDATASET and a baseline detector --COLDETECTOR which is trained on the dataset.
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework
We propose LMTurk, a novel approach that treats few-shot learners as crowdsourcing workers.
UniDS: A Unified Dialogue System for Chit-Chat and Task-oriented Dialogues
With the advances in deep learning, tremendous progress has been made with chit-chat dialogue systems and task-oriented dialogue systems.
CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems
As labeling cost for different modules in task-oriented dialog (ToD) systems is high, a major challenge in practice is to learn different tasks with the least amount of labeled data.
Representation Memorization for Fast Learning New Knowledge without Forgetting
In this paper, we consider scenarios that require learning new classes or data distributions quickly and incrementally over time, as it often occurs in real-world dynamic environments.
Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems
In this paper, we devise a self-training approach to utilize the abundant unlabeled dialog data to further improve state-of-the-art pre-trained models in few-shot learning scenarios for ToD systems.
SLIM: Explicit Slot-Intent Mapping with BERT for Joint Multi-Intent Detection and Slot Filling
Utterance-level intent detection and token-level slot filling are two key tasks for natural language understanding (NLU) in task-oriented systems.
Ranked #3 on
Slot Filling
on MixSNIPS
SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation
Code representation learning, which aims to encode the semantics of source code into distributed vectors, plays an important role in recent deep-learning-based models for code intelligence.
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems
Natural language generation (NLG) is an essential component of task-oriented dialog systems.
ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation
In this case, the recommender is updated continually and periodically with new data that arrives in each update cycle, and the updated model needs to provide recommendations for user activities before the next model update.
Memory Augmented Neural Model for Incremental Session-based Recommendation
We empirically show that MAN is well-suited for the incremental SR task, and it consistently outperforms state-of-the-art neural and nonparametric methods.
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
We present an efficient method of utilizing pretrained language models, where we learn selective binary masks for pretrained weights in lieu of modifying them through finetuning.
Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systems
Natural language generation (NLG) is an essential component of task-oriented dialogue systems.
Context Tree for Adaptive Session-based Recommendation
Therefore, recommendations need to be adaptive to such frequent changes.
Information Retrieval
Personalization in Goal-Oriented Dialog
The main goal of modeling human conversation is to create agents which can interact with people in both open-ended and goal-oriented scenarios.
