Search Results for author:
Found 164 papers, 81 papers with code
Minimum Risk Training for Neural Machine Translation
We propose minimum risk training for end-to-end neural machine translation.
Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation
The attentional mechanism has proven to be effective in improving end-to-end neural machine translation.
Question Answering over Knowledge Base with Neural Attention Combining Global Knowledge Information
With the rapid growth of knowledge bases (KBs) on the web, how to take full advantage of them becomes increasingly important.
Semi-Supervised Learning for Neural Machine Translation
While end-to-end neural machine translation (NMT) has made remarkable progress recently, NMT systems only rely on parallel corpora for parameter estimation.
Chinese Poetry Generation with Planning based Neural Network
Chinese poetry generation is a very challenging task in natural language processing.
Latent Topic Embedding
Topic modeling and word embedding are two important techniques for deriving latent semantics from data.
An End-to-End Model for Question Answering over Knowledge Base with Cross-Attention Combining Global Knowledge
This simple representation strategy is not easy to express the proper information in the question.
DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
Experiments show that human performance is well above current state-of-the-art baseline systems, leaving plenty of room for the community to make improvements.
Multi-channel Encoder for Neural Machine Translation
This design of encoder yields relatively uniform composition on source sentence, despite the gating mechanism employed in encoding RNN.
A New Method of Region Embedding for Text Classification
In this paper, we propose a new method of learning and utilizing task-specific distributed representations of n-grams, referred to as “region embeddings”.
Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification
Machine reading comprehension (MRC) on real web data usually requires the machine to answer a question by analyzing multiple passages retrieved by search engine.
Ranked #3 on
Question Answering
on MS MARCO
Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network
Human generates responses relying on semantic and functional dependencies, including coreference relation, among dialogue elements and their context.
Ranked #6 on
Conversational Response Selection
on RRS
Familia: A Configurable Topic Modeling Framework for Industrial Text Engineering
In order to relieve burdens of software engineers without knowledge of Bayesian networks, Familia is able to conduct automatic parameter inference for a variety of topic models.
Addressing Troublesome Words in Neural Machine Translation
One of the weaknesses of Neural Machine Translation (NMT) is in handling lowfrequency and ambiguous words, which we refer as troublesome words.
STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework
Simultaneous translation, which translates sentences before they are finished, is useful in many scenarios but is notoriously difficult due to word-order differences.
Modeling Coherence for Discourse Neural Machine Translation
Discourse coherence plays an important role in the translation of one text.
Learning to Select Knowledge for Response Generation in Dialog Systems
Specifically, a posterior distribution over knowledge is inferred from both utterances and responses, and it ensures the appropriate selection of knowledge during the training process.
Knowledge Aware Conversation Generation with Explainable Reasoning over Augmented Graphs
Two types of knowledge, triples from knowledge graphs and texts from documents, have been studied for knowledge aware open-domain conversation generation, in which graph paths can narrow down vertex candidates for knowledge selection decision, and texts can provide rich information for response generation.
End-to-End Speech Translation with Knowledge Distillation
End-to-end speech translation (ST), which directly translates from source language speech into target language text, has attracted intensive attentions in recent years.
ERNIE: Enhanced Representation through Knowledge Integration
We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration).
Ranked #3 on
Natural Language Inference
on XNLI Chinese Dev
Chinese Named Entity Recognition
Chinese Sentence Pair Classification
+8
Know More about Each Other: Evolving Dialogue Strategy via Compound Assessment
In this paper, a novel Generation-Evaluation framework is developed for multi-turn conversations with the objective of letting both participants know more about each other.
Generating Multiple Diverse Responses with Multi-Mapping and Posterior Mapping Selection
In this paper, we propose a multi-mapping mechanism to better capture the one-to-many relationship, where multiple mapping modules are employed as latent mechanisms to model the semantic mappings from an input post to its diverse responses.
Proactive Human-Machine Conversation with Explicit Conversation Goals
DuConv enables a very challenging task as the model needs to both understand dialogue and plan over the given knowledge graph.
ARNOR: Attention Regularization based Noise Reduction for Distant Supervision Relation Classification
In this paper, we propose ARNOR, a novel Attention Regularization based NOise Reduction framework for distant supervision relation classification.
Enhancing Pre-Trained Language Representations with Rich Knowledge for Machine Reading Comprehension
In this work, we investigate the potential of leveraging external knowledge bases (KBs) to further improve BERT for MRC.
Proactive Human-Machine Conversation with Explicit Conversation Goal
Konv enables a very challenging task as the model needs to both understand dialogue and plan over the given knowledge graph.
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
Recently, pre-trained models have achieved state-of-the-art results in various language understanding tasks, which indicates that pre-training on large-scale corpora may play a crucial role in natural language processing.
Ranked #1 on
Chinese Sentence Pair Classification
on LCQMC Dev
Chinese Named Entity Recognition
Chinese Reading Comprehension
+8
DuTongChuan: Context-aware Translation Model for Simultaneous Interpreting
In this paper, we present DuTongChuan, a novel context-aware translation model for simultaneous interpreting.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Baidu Neural Machine Translation Systems for WMT19
In this paper we introduce the systems Baidu submitted for the WMT19 shared task on Chinese{\textless}-{\textgreater}English news translation.
Multi-agent Learning for Neural Machine Translation
Conventional Neural Machine Translation (NMT) models benefit from the training with an additional agent, e. g., dual learning, and bidirectional decoding with one agent decoding from left to right and the other decoding in the opposite direction.
PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable
Pre-training models have been proved effective for a wide range of natural language processing tasks.
Enhancing Local Feature Extraction with Global Representation for Neural Text Classification
For text classification, traditional local feature driven models learn long dependency by deeply stacking or hybrid modeling.
D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension
In this paper, we introduce a simple system Baidu submitted for MRQA (Machine Reading for Question Answering) 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models.
CoKE: Contextualized Knowledge Graph Embedding
This work presents Contextualized Knowledge Graph Embedding (CoKE), a novel paradigm that takes into account such contextual nature, and learns dynamic, flexible, and fully contextualized entity and relation embeddings.
Synchronous Speech Recognition and Speech-to-Text Translation with Interactive Decoding
Speech-to-text translation (ST), which translates source language speech into target language text, has attracted intensive attention in recent years.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation
Current pre-training works in natural language generation pay little attention to the problem of exposure bias on downstream tasks.
Ranked #1 on
Question Generation
on SQuAD1.1
(using extra training data)
DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications
Machine reading comprehension (MRC) is a crucial task in natural language processing and has achieved remarkable advancements.
Towards Conversational Recommendation over Multi-Type Dialogs
We propose a new task of conversational recommendation over multi-type dialogs, where the bots can proactively and naturally lead a conversation from a non-recommendation dialog (e. g., QA) to a recommendation dialog, taking into account user's interests and feedback.
SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis
In particular, the prediction of aspect-sentiment pairs is converted into multi-label classification, aiming to capture the dependency between words in a pair.
Ranked #14 on
Stock Market Prediction
on Astock
Leveraging Graph to Improve Abstractive Multi-Document Summarization
Graphs that capture relations between textual units have great benefits for detecting salient information from multiple documents and generating overall coherent summaries.
ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph
Thus, ERNIE-ViL can learn the joint representations characterizing the alignments of the detailed semantics across vision and language.
Ranked #2 on
Visual Question Answering (VQA)
on VCR (QA-R) test
PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning
To build a high-quality open-domain chatbot, we introduce the effective training process of PLATO-2 via curriculum learning.
Conversational Graph Grounded Policy Learning for Open-Domain Conversation Generation
To address the challenge of policy learning in open-domain multi-turn conversation, we propose to represent prior information about dialog transitions as a graph and learn a graph grounded dialog policy, aimed at fostering a more coherent and controllable dialog.
Two-dimensional ferromagnetic semiconductor VBr3 with tunable anisotropy
Two-dimensional (2D) ferromagnets (FMs) have attracted widespread attention due to their prospects in spintronic applications.
Materials Science Strongly Correlated Electrons
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering
In open-domain question answering, dense passage retrieval has become a new paradigm to retrieve relevant passages for finding answers.
Ranked #4 on
Passage Retrieval
on Natural Questions
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding
We argue that such contiguously masking method neglects to model the intra-dependencies and inter-relation of coarse-grained linguistic information.
Discovering Dialog Structure Graph for Open-Domain Dialog Generation
Learning interpretable dialog structure from human-human dialogs yields basic insights into the structure of conversation, and also provides background knowledge to facilitate dialog generation.
UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning
Existed pre-training methods either focus on single-modal tasks or multi-modal tasks, and cannot effectively adapt to each other.
Ranked #3 on
Image Captioning
on MS COCO
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora
In this paper, we propose ERNIE-M, a new training method that encourages the model to align the representation of multiple languages with monolingual corpora, to overcome the constraint that the parallel corpus size places on the model performance.
Ranked #14 on
Zero-Shot Cross-Lingual Transfer
on XTREME
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer
Transformers are not suited for processing long documents, due to their quadratically increasing memory and time consumption.
Ranked #1000000000 on
Text Classification
on IMDb
Knowledge Distillation based Ensemble Learning for Neural Machine Translation
Under this framework, we introduce word-level ensemble learning and sequence-level ensemble learning for neural machine translation, where sequence-level ensemble learning is capable of aggregating translation models with different decoding strategies.
Learning to Select External Knowledge with Multi-Scale Negative Sampling
The Track-1 of DSTC9 aims to effectively answer user requests or questions during task-oriented dialogues, which are out of the scope of APIs/DB.
Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsing
For better distribution matching, we require that at least 80% of SQL patterns in the training data are covered by generated queries.
Syntactic and Semantic-driven Learning for Open Information Extraction
One of the biggest bottlenecks in building accurate, high coverage neural open IE systems is the need for large labelled corpora.
BSTC: A Large-Scale Chinese-English Speech Translation Dataset
This corpus is expected to promote the research of automatic simultaneous translation as well as the development of practical systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A Unified Pre-training Framework for Conversational AI
In this work, we explore the application of PLATO-2 on various dialogue systems, including open-domain conversation, knowledge grounded dialogue, and task-oriented conversation.
Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching
This paper proposes an approach to Dense Video Captioning (DVC) without pairwise event-sentence annotation.
BASS: Boosting Abstractive Summarization with Unified Semantic Graph
Abstractive summarization for long-document or multi-document remains challenging for the Seq2Seq architecture, as Seq2Seq is not good at analyzing long-distance relations in text.
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression
Specifically, the first stage, General Distillation, performs distillation with guidance from pretrained teacher, gerenal data and latent distillation loss.
ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction
Recent advances in graph neural networks (GNNs) have shown great promise in applying GNNs for molecular representation learning.
Ranked #2 on
Molecular Property Prediction
on ToxCast
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
We trained the model with 10 billion parameters on a 4TB corpus consisting of plain texts and a large-scale knowledge graph.
DuReader\_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications
Machine reading comprehension (MRC) is a crucial task in natural language processing and has achieved remarkable advancements.
Discovering Dialog Structure Graph for Coherent Dialog Generation
Learning discrete dialog structure graph from human-human dialogs yields basic insights into the structure of conversation, and also provides background knowledge to facilitate dialog generation.
PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval
Recently, dense passage retrieval has become a mainstream approach to finding relevant information in various natural language processing tasks.
DuTrust: A Sentiment Analysis Dataset for Trustworthiness Evaluation
Therefore, in order to systematically evaluate the factors for building trustworthy systems, we propose a novel and well-annotated sentiment analysis dataset to evaluate robustness and interpretability.
Evolving Decomposed Plasticity Rules for Information-Bottlenecked Meta-Learning
In contrast, biological neural networks (BNNs) can adapt to various new tasks by continually updating the neural connections based on the inputs, which is aligned with the paradigm of learning effective learning rules in addition to static parameters, e. g., meta-learning.
Mixup Decoding for Diverse Machine Translation
To further improve the faithfulness and diversity of the translations, we propose two simple but effective approaches to select diverse sentence pairs in the training corpus and adjust the interpolation weight for each pair correspondingly.
Fine-grained Entity Typing via Label Reasoning
Conventional entity typing approaches are based on independent classification paradigms, which make them difficult to recognize inter-dependent, long-tailed and fine-grained entity types.
Ranked #8 on
Entity Typing
on Open Entity
Controllable Dialogue Generation with Disentangled Multi-grained Style Specification and Attribute Consistency Reward
Controllable text generation is an appealing but challenging task, which allows users to specify particular attributes of the generated outputs.
A Multimodal Sentiment Dataset for Video Recommendation
Based on this, we propose a multimodal sentiment analysis dataset, named baiDu Video Sentiment dataset (DuVideoSenti), and introduce a new sentiment system which is designed to describe the sentimental style of a video on recommendation scenery.
DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendation
In this paper, we provide a bilingual parallel human-to-human recommendation dialog dataset (DuRecDial 2. 0) to enable researchers to explore a challenging task of multilingual and cross-lingual conversational recommendation.
PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation
To explore the limit of dialogue generation pre-training, we present the models of PLATO-XL with up to 11 billion parameters, trained on both Chinese and English social media conversations.
ERNIE-SPARSE: Robust Efficient Transformer Through Hierarchically Unifying Isolated Information
The first factor is information bottleneck sensitivity, which is caused by the key feature of Sparse Transformer — only a small number of global tokens can attend to all other tokens.
Do What Nature Did To Us: Evolving Plastic Recurrent Neural Networks For Generalized Tasks
While artificial neural networks (ANNs) have been widely adopted in machine learning, researchers are increasingly obsessed by the gaps between ANNs and natural neural networks (NNNs).
Building Chinese Biomedical Language Models via Multi-Level Text Discrimination
In this work we introduce eHealth, a Chinese biomedical PLM built from scratch with a new pre-training framework.
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking
In this paper, we propose a novel joint training approach for dense passage retrieval and passage re-ranking.
SgSum: Transforming Multi-document Summarization into Sub-graph Selection
Comparing with traditional methods, our method has two main advantages: (1) the relations between sentences are captured by modeling both the graph structure of the whole document set and the candidate sub-graphs; (2) directly outputs an integrate summary in the form of sub-graph which is more informative and coherent.
Amendable Generation for Dialogue State Tracking
In this paper, we propose a novel Amendable Generation for Dialogue State Tracking (AG-DST), which contains a two-pass generation process: (1) generating a primitive dialogue state based on the dialogue of the current turn and the previous dialogue state, and (2) amending the primitive dialogue state from the first pass.
Ranked #1 on
Dialogue State Tracking
on Wizard-of-Oz
Dialogue State Tracking
Multi-domain Dialogue State Tracking
+1
Docking-based Virtual Screening with Multi-Task Learning
Machine learning shows great potential in virtual screening for drug discovery.
HelixMO: Sample-Efficient Molecular Optimization in Scene-Sensitive Latent Space
Efficient exploration of the chemical space to search the candidate drugs that satisfy various constraints is a fundamental task of drug discovery.
Learning with Noisy Correspondence for Cross-modal Matching
Based on this observation, we reveal and study a latent and challenging direction in cross-modal matching, named noisy correspondence, which could be regarded as a new paradigm of noisy labels.
DuQM: A Chinese Dataset of Linguistically Perturbed Natural Questions for Evaluating the Robustness of Question Matching Models
For this purpose, we create a Chinese dataset namely DuQM which contains natural questions with linguistic perturbations to evaluate the robustness of question matching models.
TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue Modeling on Spoken Conversations
Task-oriented dialogue systems have been plagued by the difficulties of obtaining large-scale and high-quality annotated conversations.
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
A unified framework named ERNIE 3. 0 was recently proposed for pre-training large-scale knowledge enhanced models and trained a model with 10 billion parameters.
ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation
To explore the landscape of large-scale pre-training for bidirectional text-image generation, we train a 10-billion parameter ERNIE-ViLG model on a large-scale dataset of 145 million (Chinese) image-text pairs which achieves state-of-the-art performance for both text-to-image and image-to-text tasks, obtaining an FID of 7. 9 on MS-COCO for text-to-image synthesis and best results on COCO-CN and AIC-ICC for image captioning.
Ranked #42 on
Text-to-Image Generation
on MS COCO
Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods
In this survey, we provide a systematic overview of the research progress on the faithfulness problem of NLG, including problem analysis, evaluation metrics and optimization methods.
Long Time No See! Open-Domain Conversation with Long-Term Persona Memory
Most of the open-domain dialogue models tend to perform poorly in the setting of long-term human-bot conversations.
UNIMO-2: End-to-End Unified Vision-Language Grounded Learning
In particular, we propose to conduct grounded learning on both images and texts via a sharing grounded space, which helps bridge unaligned images and texts, and align the visual and textual semantic spaces on different types of corpora.
PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation
Despite recent progress of pre-trained language models on generating fluent text, existing methods still suffer from incoherence problems in long-form text generation tasks that require proper content control and planning to form a coherent high-level logical flow.
DU-VLG: Unifying Vision-and-Language Generation via Dual Sequence-to-Sequence Pre-training
To bridge the gap between image understanding and generation, we further design a novel commitment loss.
DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine
In this paper, we present DuReader_retrieval, a large-scale Chinese dataset for passage retrieval.
ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention
We argue that two factors, information bottleneck sensitivity and inconsistency between different attention topologies, could affect the performance of the Sparse Transformer.
Unified Structure Generation for Universal Information Extraction
Information extraction suffers from its varying targets, heterogeneous structures, and demand-specific schemas.
Ranked #4 on
Aspect-Based Sentiment Analysis (ABSA)
on ASTE
(using extra training data)
Where to Go for the Holidays: Towards Mixed-Type Dialogs for Clarification of User Goals
For example, users have determined the departure, the destination, and the travel time for booking a flight.
Towards Multi-Turn Empathetic Dialogs with Positive Emotion Elicitation
In this task, the agent conducts empathetic responses along with the target of eliciting the user's positive emotions in the multi-turn dialog.
A Thorough Examination on Zero-shot Dense Retrieval
Recent years have witnessed the significant advance in dense retrieval (DR) based on powerful pre-trained language models (PLM).
HelixADMET: a robust and endpoint extensible ADMET system incorporating self-supervised knowledge transfer
Additionally, the pre-trained model provided by H-ADMET can be fine-tuned to generate new and customised ADMET endpoints, meeting various demands of drug research and development requirements.
ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrieval
Our method 1) introduces a self on-the-fly distillation method that can effectively distill late interaction (i. e., ColBERT) to vanilla dual-encoder, and 2) incorporates a cascade distillation process to further improve the performance with a cross-encoder teacher.
A Fine-grained Interpretability Evaluation Benchmark for Neural NLP
Based on this benchmark, we conduct experiments on three typical models with three saliency methods, and unveil their strengths and weakness in terms of interpretability.
Less Learn Shortcut: Analyzing and Mitigating Learning of Spurious Feature-Label Correlation
In this study, we focus on the spurious correlation between word features and labels that models learn from the biased data distribution of training data.
Bi-SimCut: A Simple Strategy for Boosting Neural Machine Translation
We introduce Bi-SimCut: a simple but effective training strategy to boost neural machine translation (NMT) performance.
Ranked #1 on
Machine Translation
on WMT2014 German-English
Link the World: Improving Open-domain Conversation with Dynamic Spatiotemporal-aware Knowledge
Making chatbots world aware in a conversation like a human is a crucial challenge, where the world may contain dynamic knowledge and spatiotemporal state.
HelixFold-Single: MSA-free Protein Structure Prediction by Using Protein Language Model as an Alternative
Our proposed method, HelixFold-Single, first pre-trains a large-scale protein language model (PLM) with thousands of millions of primary sequences utilizing the self-supervised learning paradigm, which will be used as an alternative to MSAs for learning the co-evolution information.
An Interpretability Evaluation Benchmark for Pre-trained Language Models
To fill in the gap, we propose a novel evaluation benchmark providing with both English and Chinese annotated data.
GEM-2: Next Generation Molecular Property Prediction Network by Modeling Full-range Many-body Interactions
Full-range many-body interactions between electrons have been proven effective in obtaining an accurate solution of the Schr"odinger equation by classical computational chemistry methods, although modeling such interactions consumes an expensive computational cost.
SeSQL: Yet Another Large-scale Session-level Chinese Text-to-SQL Dataset
As the first session-level Chinese dataset, CHASE contains two separate parts, i. e., 2, 003 sessions manually constructed from scratch (CHASE-C), and 3, 456 sessions translated from English SParC (CHASE-T).
Towards Boosting the Open-Domain Chatbot with Human Feedback
Many open-domain dialogue models pre-trained with social media comments can generate coherent replies but have difficulties producing engaging responses when interacting with real users.
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training
They attempt to learn cross-modal representation using contrastive learning on image-text pairs, however, the built inter-modal correlations only rely on a single view for each modality.
Ranked #1 on
Image Retrieval
on AIC-ICC
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding
Recent years have witnessed the rise and success of pre-training techniques in visually-rich document understanding.
Ranked #2 on
Semantic entity labeling
on FUNSD
Q-TOD: A Query-driven Task-oriented Dialogue System
Firstly, as the query is in the form of natural language and not confined to the schema of the knowledge base, the issue of domain adaption is alleviated remarkably in Q-TOD.
CDConv: A Benchmark for Contradiction Detection in Chinese Conversations
Dialogue contradiction is a critical issue in open-domain dialogue systems.
Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards
Derivative-free prompt learning has emerged as a lightweight alternative to prompt tuning, which only requires model inference to optimize the prompts.
ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts
Recent progress in diffusion models has revolutionized the popular technology of text-to-image generation.
Ranked #12 on
Text-to-Image Generation
on MS COCO
UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance
Diffusion generative models have recently greatly improved the power of text-conditioned image generation.
FRSUM: Towards Faithful Abstractive Summarization via Enhancing Factual Robustness
We first measure a model's factual robustness by its success rate to defend against adversarial attacks when generating factual information.
PLATO-K: Internal and External Knowledge Enhanced Dialogue Generation
Recently, the practical deployment of open-domain dialogue systems has been plagued by the knowledge issue of information deficiency and factual inaccuracy.
CLOP: Video-and-Language Pre-Training with Knowledge Regularizations
To this end, we propose a Cross-modaL knOwledge-enhanced Pre-training (CLOP) method with Knowledge Regularizations.
ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech
In this paper, we extend the pretraining method for cross-lingual multi-speaker speech synthesis tasks, including cross-lingual multi-speaker voice cloning and cross-lingual multi-speaker speech editing.
ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation
Recent cross-lingual cross-modal works attempt to extend Vision-Language Pre-training (VLP) models to non-English inputs and achieve impressive performance.
Ranked #1 on
Multimodal Machine Translation
on Multi30K
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages
Extensive results show that ERNIE-Code outperforms previous multilingual LLMs for PL or NL across a wide range of end tasks of code intelligence, including multilingual code-to-text, text-to-code, code-to-code, and text-to-text generation.
Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modeling
In this paper, we propose an unsupervised query enhanced approach for knowledge-intensive conversations, namely QKConv.
ERNIE 3.0 Tiny: Frustratingly Simple Method to Improve Task-Agnostic Distillation Generalization
Experimental results demonstrate that our method yields a student with much better generalization, significantly outperforms existing baselines, and establishes a new state-of-the-art result on in-domain, out-domain, and low-resource datasets in the setting of task-agnostic distillation.
Universal Information Extraction as Unified Semantic Matching
Based on this paradigm, we propose to universally model various IE tasks with Unified Semantic Matching (USM) framework, which introduces three unified token linking operations to model the abilities of structuring and conceptualizing.
ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Models
In response to this lacuna, this paper introduces a pioneering contribution in the form of a text-to-waveform music generation model, underpinned by the utilization of diffusion models.
SMoA: Sparse Mixture of Adapters to Mitigate Multiple Dataset Biases
Recent studies reveal that various biases exist in different NLP tasks, and over-reliance on biases results in models' poor generalization ability and low adversarial robustness.
Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization
The experimental analysis also proves that CrossConST could close the sentence representation gap and better align the representation space.
TOME: A Two-stage Approach for Model-based Retrieval
Recently, model-based retrieval has emerged as a new paradigm in text retrieval that discards the index in the traditional retrieval model and instead memorizes the candidate corpora using model parameters.
Learning In-context Learning for Named Entity Recognition
M}$, and a new entity extractor can be implicitly constructed by applying new instruction and demonstrations to PLMs, i. e., $\mathcal{ (\lambda .
A Simple yet Effective Self-Debiasing Framework for Transformer Models
In this way, the top-layer sentence representation will be trained to ignore the common biased features encoded by the low-layer sentence representation and focus on task-relevant unbiased features.
Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularization
Multilingual sentence representations are the foundation for similarity-based bitext mining, which is crucial for scaling multilingual neural machine translation (NMT) system to more languages.
Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation
In this study, we present an initial analysis of the factual knowledge boundaries of LLMs and how retrieval augmentation affects LLMs on open-domain QA.
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation
Consistency regularization methods, such as R-Drop (Liang et al., 2021) and CrossConST (Gao et al., 2023), have achieved impressive supervised and zero-shot performance in the neural machine translation (NMT) field.
GLS-CSC: A Simple but Effective Strategy to Mitigate Chinese STM Models' Over-Reliance on Superficial Clue
To address this issue, it is crucial to analyze and mitigate the influence of superficial clues on STM models.
Tool-Augmented Reward Modeling
We validate our approach across a wide range of domains, incorporating seven distinct external tools.
IBADR: an Iterative Bias-Aware Dataset Refinement Framework for Debiasing NLU models
Then, we pair each sample with a bias indicator representing its bias degree, and use these extended samples to train a sample generator.
Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Models
The training paradigm for machine translation has gradually shifted, from learning neural machine translation (NMT) models with extensive parallel corpora to instruction finetuning on multilingual large language models (LLMs) with high-quality translation pairs.
DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing
Graph Neural Networks (GNNs) have shown great power for learning and mining on graphs, and Graph Structure Learning (GSL) plays an important role in boosting GNNs with a refined graph.
BASES: Large-scale Web Search User Simulation with Large Language Model based Agents
Due to the excellent capacities of large language models (LLMs), it becomes feasible to develop LLM-based agents for reliable user simulation.
On Training Data Influence of GPT Models
This paper presents GPTfluence, a novel approach that leverages a featurized simulation to assess the impact of training examples on the training dynamics of GPT models.
Dual Modalities of Text: Visual and Textual Generative Pre-training
Harnessing visual texts represents a burgeoning frontier in the evolution of language modeling.
\textrm{DuReader}_{\textrm{vis}}: A Chinese Dataset for Open-domain Document Visual Question Answering
Open-domain question answering has been used in a wide range of applications, such as web search and enterprise search, which usually takes clean texts extracted from various formats of documents (e. g., web pages, PDFs, or Word documents) as the information source.
PLATO-KAG: Unsupervised Knowledge-Grounded Conversation via Joint Modeling
Large-scale conversation models are turning to leveraging external knowledge to improve the factual accuracy in response generation.
DuSQL: A Large-Scale and Pragmatic Chinese Text-to-SQL Dataset
This paper describes in detail the construction process and data statistics of DuSQL.
Non-Autoregressive Chinese ASR Error Correction with Phonological Training
Automatic Speech Recognition (ASR) is an efficient and widely used input method that transcribes speech signals into text.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Diversified Multiple Instance Learning for Document-Level Multi-Aspect Sentiment Classification
To this end, we propose a novel Diversified Multiple Instance Learning Network (D-MILN), which is able to achieve aspect-level sentiment classification with only document-level weak supervision.
Syntax-guided Contrastive Learning for Pre-trained Language Model
Syntactic information has been proved to be useful for transformer-based pre-trained language models.
Learning Adaptive Segmentation Policy for End-to-End Simultaneous Translation
End-to-end simultaneous speech-to-text translation aims to directly perform translation from streaming source speech to target text with high translation quality and low latency.
Learning Adaptive Segmentation Policy for Simultaneous Translation
The policy learns to segment the source text by considering possible translations produced by the translation model, maintaining consistency between the segmentation and translation.
Findings of the Third Workshop on Automatic Simultaneous Translation
This paper reports the results of the shared task we hosted on the Third Workshop of Automatic Simultaneous Translation (AutoSimTrans).
SgSum:Transforming Multi-document Summarization into Sub-graph Selection
Comparing with traditional methods, our method has two main advantages: (1) the relations between sentences are captured by modeling both the graph structure of the whole document set and the candidate sub-graphs; (2) directly outputs an integrate summary in the form of sub-graph which is more informative and coherent.
Findings of the Second Workshop on Automatic Simultaneous Translation
This paper presents the results of the shared task of the 2nd Workshop on Automatic Simultaneous Translation (AutoSimTrans).
