Search Results for author:
Found 47 papers, 22 papers with code
NEZHA: Neural Contextualized Representation for Chinese Language Understanding
The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized information in text by pre-training on large-scale corpora.
GPT-based Generation for Classical Chinese Poetry
We present a simple yet effective method for generating high quality classical Chinese poetry with Generative Pre-trained Language Model (GPT).
JABER and SABER: Junior and Senior Arabic BERt
Language-specific pre-trained models have proven to be more accurate than multilingual ones in a monolingual evaluation setting, Arabic is no exception.
PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained Language Model
We investigate different aspects of responses generated by PanGu-Bot, including response quality, knowledge, and safety.
Unified Mandarin TTS Front-end Based on Distilled BERT Model
The front-end module in a typical Mandarin text-to-speech system (TTS) is composed of a long pipeline of text processing components, which requires extensive efforts to build and is prone to large accumulative model size and cascade errors.
Multi-channel Reverse Dictionary Model
A reverse dictionary takes the description of a target word as input and outputs the target word together with other words that match the description.
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
As far as we know, almost all existing textual backdoor attack methods insert additional contents into normal samples as triggers, which causes the trigger-embedded samples to be detected and the backdoor attacks to be blocked without much effort.
Sub-Character Tokenization for Chinese Pretrained Language Models
2) Pronunciation-based SubChar tokenizers can encode Chinese homophones into the same transliteration sequences and produce the same tokenization output, hence being robust to homophone typos.
Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attacks
In this work, we demonstrate the universal vulnerability of PTMs, where fine-tuned PTMs can be easily controlled by backdoor attacks in arbitrary downstream tasks.
Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarks
The research of open-domain dialog systems has been greatly prospered by neural models trained on large-scale corpora, however, such corpora often introduce various safety problems (e. g., offensive languages, biases, and toxic behaviors) that significantly hinder the deployment of dialog systems in practice.
Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
In this work, we propose a simple and effective method to cover a much larger proportion of the attack search space, called Adversarial and Mixup Data Augmentation (AMDA).
Source Code Summarization with Structural Relative Position Guided Transformer
We further show that how the proposed SCRIPT captures the structural relative dependencies.
Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios
The recent trend of using Large Language Models (LLMs) as tool agents in real-world applications underscores the necessity for comprehensive evaluations of their capabilities, particularly in complex scenarios involving planning, creating, and using tools.
Improving Sequence Modeling Ability of Recurrent Neural Networks via Sememes
Sememes, the minimum semantic units of human languages, have been successfully utilized in various natural language processing applications.
Sparse Structure Search for Delta Tuning
Generally, DT methods exquisitely design delta modules (DT modules) which could be applied to arbitrary fine-grained positions inside PTMs.
Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation
In order to detect such toxic generations, existing methods rely on templates, real-world data extraction, crowdsourcing workers, or automatic generation to construct adversarial contexts that are likely to induce toxic generations.
DecompEval: Evaluating Generated Texts as Unsupervised Decomposed Question Answering
Existing evaluation metrics for natural language generation (NLG) tasks face the challenges on generalization ability and interpretability.
PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models
LLMs are prompted to generate extensive content in response to these meta-questions.
MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions
In this paper, we propose a framework, MoralDial to train and evaluate moral dialogue systems.
PanGu-Coder: Program Synthesis with Function-Level Language Modeling
We present PanGu-Coder, a pretrained decoder-only language model adopting the PanGu-Alpha architecture for text-to-code generation, i. e. the synthesis of programming language solutions given a natural language problem description.
Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignment
Inspired by previous work which identified that feed-forward networks (FFNs) within Transformers are responsible for factual knowledge expressions, we investigate two methods to efficiently improve the factual expression capability {of FFNs} by knowledge enhancement and alignment respectively.
Sentiment Lexicon Expansion Based on Neural PU Learning, Double Dictionary Lookup, and Polarity Association
Although many sentiment lexicons in different languages exist, most are not comprehensive.
SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation
Code representation learning, which aims to encode the semantics of source code into distributed vectors, plays an important role in recent deep-learning-based models for code intelligence.
CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems
As labeling cost for different modules in task-oriented dialog (ToD) systems is high, a major challenge in practice is to learn different tasks with the least amount of labeled data.
UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation
Specially, we adopt knowledge distillation from a vision-language pretrained model to improve image selection, which avoids any requirement on the existence and quality of image captions.
UniDS: A Unified Dialogue System for Chit-Chat and Task-oriented Dialogues
With the advances in deep learning, tremendous progress has been made with chit-chat dialogue systems and task-oriented dialogue systems.
CoCA-MDD: A Coupled Cross-Attention based Framework for Streaming Mispronunciation Detection and Diagnosis
We utilize conv-transformer structure to encode input speech in a streaming manner.
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework
We propose LMTurk, a novel approach that treats few-shot learners as crowdsourcing workers.
Pan More Gold from the Sand: Refining Open-domain Dialogue Training with Noisy Self-Retrieval Generation
Experiments over publicly available datasets demonstrate that our method can help models generate better responses, even such training data are usually impressed as low-quality data.
HyperPELT: Unified Parameter-Efficient Language Model Tuning for Both Language and Vision-and-Language Tasks
In this paper, we design a novel unified parameter-efficient transfer learning framework that works effectively on both pure language and V&L tasks.
Compilable Neural Code Generation with Compiler Feedback
Automatically generating compilable programs with (or without) natural language descriptions has always been a touchstone problem for computational linguistics and automated software engineering.
CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Training
Specifically, we first extract multiple code views using compiler tools, and learn the complementary information among them under a contrastive learning framework.
Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding
There is a growing body of work in recent years to develop pre-trained language models (PLMs) for the Arabic language.
Sparse Structure Search for Parameter-Efficient Tuning
The searched structures preserve more than 99\% fine-tuning performance with 0. 01\% trainable parameters.
Lexicon-injected Semantic Parsing for Task-Oriented Dialog
Then we present a novel lexicon-injected semantic parser, which collects slot labels of tree representation as a lexicon, and injects lexical features to the span representation of parser.
KPT: Keyword-guided Pre-training for Grounded Dialog Generation
For the former, the grounding knowledge consists of keywords extracted from the response.
Momentum Contrastive Pre-training for Question Answering
In order to address this problem, we propose a novel Momentum Contrastive pRe-training fOr queStion anSwering (MCROSS) method for extractive QA.
MultiCoder: Multi-Programming-Lingual Pre-Training for Low-Resource Code Completion
On the other hand, there are few studies exploring the effects of multi-programming-lingual (MultiPL) pre-training for the code completion, especially the impact on low-resource programming languages.
Learning Summary-Worthy Visual Representation for Abstractive Summarization in Video
Multimodal abstractive summarization for videos (MAS) requires generating a concise textual summary to describe the highlights of a video according to multimodal resources, in our case, the video content and its transcript.
Large Language Models as Source Planner for Personalized Knowledge-grounded Dialogue
Open-domain dialogue system usually requires different sources of knowledge to generate more informative and evidential responses.
YODA: Teacher-Student Progressive Learning for Language Models
With the teacher's guidance, the student learns to iteratively refine its answer with feedback, and forms a robust and comprehensive understanding of the posed questions.
Understanding the planning of LLM agents: A survey
As Large Language Models (LLMs) have shown significant intelligence, the progress to leverage LLMs as planning modules of autonomous agents has attracted more attention.
UniRetriever: Multi-task Candidates Selection for Various Context-Adaptive Conversational Retrieval
Conversational retrieval refers to an information retrieval system that operates in an iterative and interactive manner, requiring the retrieval of various external resources, such as persona, knowledge, and even response, to effectively engage with the user and successfully complete the dialogue.
Evaluating Robustness of Generative Search Engine on Adversarial Factual Questions
To this end, we propose evaluating the robustness of generative search engines in the realistic and high-risk setting, where adversaries have only black-box system access and seek to deceive the model into returning incorrect responses.
Improving Language Model Reasoning with Self-motivated Learning
The framework motivates the model itself to automatically generate rationales on existing datasets.
WESE: Weak Exploration to Strong Exploitation for LLM Agents
Concretely, WESE involves decoupling the exploration and exploitation process, employing a cost-effective weak agent to perform exploration tasks for global knowledge.
