Search Results for author:
Found 30 papers, 26 papers with code
A Large-Scale Chinese Short-Text Conversation Dataset
The cleaned dataset and the pre-training models will facilitate the research of short-text conversation modeling.
CPM: A Large-scale Generative Chinese Pre-trained Language Model
However, applying GPT-3 to address Chinese NLP tasks is still challenging, as the training corpus of GPT-3 is primarily English, and the parameters are not publicly available.
CPM-2: Large-scale Cost-effective Pre-trained Language Models
We present a suite of cost-effective techniques for the use of PLMs to deal with the efficiency issues of pre-training, fine-tuning, and inference.
EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training
Although pre-trained language models have remarkably enhanced the generation ability of dialogue systems, open-domain Chinese dialogue systems are still limited by the dialogue data and the model size compared with English ones.
EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training
Large-scale pre-training has shown remarkable performance in building open-domain dialogue systems.
Black-Box Prompt Optimization: Aligning Large Language Models without Model Training
However, these models are often not well aligned with human intents, which calls for additional treatments on them, that is, the alignment problem.
AlignBench: Benchmarking Chinese Alignment of Large Language Models
We will provide public APIs for evaluating AlignBench with CritiqueLLM to facilitate the evaluation of LLMs' Chinese alignment.
CritiqueLLM: Scaling LLM-as-Critic for Effective and Explainable Evaluation of Large Language Model Generation
Since the natural language processing (NLP) community started to make large language models (LLMs), such as GPT-4, act as a critic to evaluate the quality of generated texts, most of them only train a critique generation model of a specific scale on specific datasets.
CoTK: An Open-Source Toolkit for Fast Development and Fair Evaluation of Text Generation
In text generation evaluation, many practical issues, such as inconsistent experimental settings and metric implementations, are often ignored but lead to unfair evaluation and untenable conclusions.
Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph
Despite the success of generative pre-trained language models on a series of text generation tasks, they still suffer in cases where reasoning over underlying commonsense knowledge is required during generation.
Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generation
Non-AutoRegressive (NAR) text generation models have drawn much attention because of their significantly faster decoding speed and good generation quality in machine translation.
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge
To benefit the downstream tasks in sentiment analysis, we propose a novel language representation model called SentiLARE, which introduces word-level linguistic knowledge including part-of-speech tag and sentiment polarity (inferred from SentiWordNet) into pre-trained models.
JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs
Existing pre-trained models for knowledge-graph-to-text (KG-to-text) generation simply fine-tune text-to-text pre-trained models such as BART or T5 on KG-to-text datasets, which largely ignore the graph structure during encoding and lack elaborate pre-training tasks to explicitly model graph-text alignments.
Ranked #1 on
KG-to-Text Generation
on WebQuestions
ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors
The safety of Large Language Models (LLMs) has gained increasing attention in recent years, but there still lacks a comprehensive approach for detecting safety issues within LLMs' responses in an aligned, customizable and explainable manner.
ARAML: A Stable Adversarial Training Framework for Text Generation
The generator is optimized with maximum likelihood estimation augmented by the discriminator's rewards instead of policy gradient.
CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation
Existing reference-free metrics have obvious limitations for evaluating controlled text generation models.
Towards Efficient and Exact Optimization of Language Model Alignment
We prove that EXO is guaranteed to optimize in the same direction as the RL algorithms asymptotically for arbitary parametrization of the policy, while enables efficient optimization by circumventing the complexities associated with RL algorithms.
Tailoring Language Generation Models under Total Variation Distance
The standard paradigm of neural language generation adopts maximum likelihood estimation (MLE) as the optimizing method.
Click: Controllable Text Generation with Sequence Likelihood Contrastive Learning
It has always been an important yet challenging problem to control language models to avoid generating texts with undesirable attributes, such as toxic language and unnatural repetition.
Learning Instructions with Unlabeled Data for Zero-Shot Cross-Task Generalization
Recently, instruction tuning (IT), which fine-tunes a pre-trained language model on a massive collection of tasks described via human-craft instructions, has been shown effective in instruction learning for unseen tasks.
Unveiling the Implicit Toxicity in Large Language Models
While recent studies primarily focus on probing toxic outputs that can be easily detected with existing toxicity classifiers, we show that LLMs can generate diverse implicit toxic outputs that are exceptionally difficult to detect via simply zero-shot prompting.
DecompEval: Evaluating Generated Texts as Unsupervised Decomposed Question Answering
Existing evaluation metrics for natural language generation (NLG) tasks face the challenges on generalization ability and interpretability.
Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization
We hope our work could contribute to the comprehension of jailbreaking attacks and defenses, and shed light on the relationship between LLMs' capability and safety.
Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation
Despite the success of text-to-text pre-trained models in various natural language generation (NLG) tasks, the generation performance is largely restricted by the number of labeled data in downstream tasks, particularly in data-to-text generation tasks.
Rethinking and Refining the Distinct Metric
We provide both empirical and theoretical evidence to show that our method effectively removes the biases existing in the original distinct score.
Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Paths
Commonsense explanation generation aims to empower the machine's sense-making capability by generating plausible explanations to statements against commonsense.
A Text GAN for Language Generation with Non-Autoregressive Generator
Despite the great success of Generative Adversarial Networks (GANs) in generating high-quality images, GANs for text generation still face two major challenges: first, most text GANs are unstable in training mainly due to ineffective optimization of the generator, and they heavily rely on maximum likelihood pretraining; second, most text GANs adopt autoregressive generators without latent variables, which largely limits the ability to learn latent representations for natural language text.
Semantic-Enhanced Explainable Finetuning for Open-Domain Dialogues
This paper propose to combine pretrained language models with the modular dialogue paradigm for open-domain dialogue modeling.
Language Model Decoding as Direct Metrics Optimization
And most importantly, we prove that this induced distribution is guaranteed to improve the perplexity on human texts, which suggests a better approximation to the underlying distribution of human texts.
