Search Results for author:
Found 177 papers, 87 papers with code
Findings of the Third Workshop on Automatic Simultaneous Translation
This paper reports the results of the shared task we hosted on the Third Workshop of Automatic Simultaneous Translation (AutoSimTrans).
Non-Autoregressive Chinese ASR Error Correction with Phonological Training
Automatic Speech Recognition (ASR) is an efficient and widely used input method that transcribes speech signals into text.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Learning Adaptive Segmentation Policy for End-to-End Simultaneous Translation
End-to-end simultaneous speech-to-text translation aims to directly perform translation from streaming source speech to target text with high translation quality and low latency.
Syntax-guided Contrastive Learning for Pre-trained Language Model
Syntactic information has been proved to be useful for transformer-based pre-trained language models.
\textrm{DuReader}_{\textrm{vis}}: A Chinese Dataset for Open-domain Document Visual Question Answering
Open-domain question answering has been used in a wide range of applications, such as web search and enterprise search, which usually takes clean texts extracted from various formats of documents (e. g., web pages, PDFs, or Word documents) as the information source.
Findings of the Second Workshop on Automatic Simultaneous Translation
This paper presents the results of the shared task of the 2nd Workshop on Automatic Simultaneous Translation (AutoSimTrans).
PLATO-KAG: Unsupervised Knowledge-Grounded Conversation via Joint Modeling
Large-scale conversation models are turning to leveraging external knowledge to improve the factual accuracy in response generation.
Diversified Multiple Instance Learning for Document-Level Multi-Aspect Sentiment Classification
To this end, we propose a novel Diversified Multiple Instance Learning Network (D-MILN), which is able to achieve aspect-level sentiment classification with only document-level weak supervision.
DuSQL: A Large-Scale and Pragmatic Chinese Text-to-SQL Dataset
This paper describes in detail the construction process and data statistics of DuSQL.
Learning Adaptive Segmentation Policy for Simultaneous Translation
The policy learns to segment the source text by considering possible translations produced by the translation model, maintaining consistency between the segmentation and translation.
SgSum:Transforming Multi-document Summarization into Sub-graph Selection
Comparing with traditional methods, our method has two main advantages: (1) the relations between sentences are captured by modeling both the graph structure of the whole document set and the candidate sub-graphs; (2) directly outputs an integrate summary in the form of sub-graph which is more informative and coherent.
Shall Your Data Strategy Work? Perform a Swift Study
This work presents a swift method to assess the efficacy of particular types of instruction-tuning data, utilizing just a handful of probe examples and eliminating the need for model retraining.
Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking
Large language models (LLMs) face inherent performance bottlenecks under parameter constraints, particularly in processing critical tokens that demand complex reasoning.
BeamLoRA: Beam-Constraint Low-Rank Adaptation
Due to the demand for efficient fine-tuning of large language models, Low-Rank Adaptation (LoRA) has been widely adopted as one of the most effective parameter-efficient fine-tuning methods.
Curiosity-Driven Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) has proven effective in aligning large language models (LLMs) with human preferences, but often at the cost of reduced output diversity.
Mixture of Hidden-Dimensions Transformer
MOHD offers a new perspective for scaling the model, showcasing the potential of hidden dimension sparsity to boost efficiency
MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions
However, token-level RLHF suffers from the credit assignment problem over long sequences, where delayed rewards make it challenging for the model to discern which actions contributed to successful outcomes.
Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging
To ensure that each specialized expert in the MoE model works as expected, we select a small amount of seed data that each expert excels to pre-optimize the router.
S$^2$AG-Vid: Enhancing Multi-Motion Alignment in Video Diffusion Models via Spatial and Syntactic Attention-Based Guidance
To enhance the motion-subject binding, we implement a syntax-guided contrastive constraint in the subsequent denoising phase, aimed at improving the correlations between the CA maps of verbs and their corresponding nouns. Both qualitative and quantitative evaluations demonstrate that the proposed framework significantly outperforms baseline approaches, producing higher-quality videos with improved subject-motion consistency.
Orthogonal Finetuning for Direct Preference Optimization
Hence, we introduce orthogonal finetuning for DPO via a weight-Rotated Preference Optimization (RoPO) method, which merely conducts rotational and magnitude-stretching updates on the weight parameters to maintain the hyperspherical energy invariant, thereby preserving the knowledge encoded in the angle between neurons.
NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
Large Language Models (LLMs) have ignited an innovative surge of AI applications, marking a new era of exciting possibilities equipped with extended context windows.
Exploring the Causality of End-to-End Autonomous Driving
Deep learning-based models are widely deployed in autonomous driving areas, especially the increasingly noticed end-to-end solutions.
BEVWorld: A Multimodal World Model for Autonomous Driving via Unified BEV Latent Space
The world model consists of two parts: the multi-modal tokenizer and the latent BEV sequence diffusion model.
HFT: Half Fine-Tuning for Large Language Models
Large language models (LLMs) with one or more fine-tuning phases have become a necessary step to unlock various capabilities, enabling LLMs to follow natural language instructions or align with human preferences.
Autoregressive Pre-Training on Pixels and Texts
Our extensive evaluation across a wide range of benchmarks shows that incorporating both visual and textual data significantly improves the performance of pixel-based language models.
On Training Data Influence of GPT Models
This paper presents GPTfluence, a novel approach that leverages a featurized simulation to assess the impact of training examples on the training dynamics of GPT models.
BASES: Large-scale Web Search User Simulation with Large Language Model based Agents
Due to the excellent capacities of large language models (LLMs), it becomes feasible to develop LLM-based agents for reliable user simulation.
DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing
Graph Neural Networks (GNNs) have shown great power for learning and mining on graphs, and Graph Structure Learning (GSL) plays an important role in boosting GNNs with a refined graph.
Towards Boosting Many-to-Many Multilingual Machine Translation with Large Language Models
The training paradigm for machine translation has gradually shifted, from learning neural machine translation (NMT) models with extensive parallel corpora to instruction finetuning on multilingual large language models (LLMs) with high-quality translation pairs.
IBADR: an Iterative Bias-Aware Dataset Refinement Framework for Debiasing NLU models
Then, we pair each sample with a bias indicator representing its bias degree, and use these extended samples to train a sample generator.
Tool-Augmented Reward Modeling
We validate our approach across a wide range of domains, incorporating seven distinct external tools.
GLS-CSC: A Simple but Effective Strategy to Mitigate Chinese STM Models' Over-Reliance on Superficial Clue
To address this issue, it is crucial to analyze and mitigate the influence of superficial clues on STM models.
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation
Consistency regularization methods, such as R-Drop (Liang et al., 2021) and CrossConST (Gao et al., 2023), have achieved impressive supervised and zero-shot performance in the neural machine translation (NMT) field.
Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation
In this study, we present the first analysis on the factual knowledge boundaries of LLMs and how retrieval augmentation affects LLMs on open-domain question answering (QA), with a bunch of important findings.
Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularization
Multilingual sentence representations are the foundation for similarity-based bitext mining, which is crucial for scaling multilingual neural machine translation (NMT) system to more languages.
A Simple yet Effective Self-Debiasing Framework for Transformer Models
In this way, the top-layer sentence representation will be trained to ignore the common biased features encoded by the low-layer sentence representation and focus on task-relevant unbiased features.
Learning In-context Learning for Named Entity Recognition
M}$, and a new entity extractor can be implicitly constructed by applying new instruction and demonstrations to PLMs, i. e., $\mathcal{ (\lambda .
TOME: A Two-stage Approach for Model-based Retrieval
Recently, model-based retrieval has emerged as a new paradigm in text retrieval that discards the index in the traditional retrieval model and instead memorizes the candidate corpora using model parameters.
Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization
The experimental analysis also proves that CrossConST could close the sentence representation gap and better align the representation space.
SMoA: Sparse Mixture of Adapters to Mitigate Multiple Dataset Biases
Recent studies reveal that various biases exist in different NLP tasks, and over-reliance on biases results in models' poor generalization ability and low adversarial robustness.
ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Models
In response to this lacuna, this paper introduces a pioneering contribution in the form of a text-to-waveform music generation model, underpinned by the utilization of diffusion models.
Universal Information Extraction as Unified Semantic Matching
Based on this paradigm, we propose to universally model various IE tasks with Unified Semantic Matching (USM) framework, which introduces three unified token linking operations to model the abilities of structuring and conceptualizing.
ERNIE 3.0 Tiny: Frustratingly Simple Method to Improve Task-Agnostic Distillation Generalization
Experimental results demonstrate that our method yields a student with much better generalization, significantly outperforms existing baselines, and establishes a new state-of-the-art result on in-domain, out-domain, and low-resource datasets in the setting of task-agnostic distillation.
Query Enhanced Knowledge-Intensive Conversation via Unsupervised Joint Modeling
In this paper, we propose an unsupervised query enhanced approach for knowledge-intensive conversations, namely QKConv.
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages
Extensive results show that ERNIE-Code outperforms previous multilingual LLMs for PL or NL across a wide range of end tasks of code intelligence, including multilingual code-to-text, text-to-code, code-to-code, and text-to-text generation.
ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation
Recent cross-lingual cross-modal works attempt to extend Vision-Language Pre-training (VLP) models to non-English inputs and achieve impressive performance.
Ranked #1 on
Multimodal Machine Translation
on Multi30K
ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech
In this paper, we extend the pretraining method for cross-lingual multi-speaker speech synthesis tasks, including cross-lingual multi-speaker voice cloning and cross-lingual multi-speaker speech editing.
CLOP: Video-and-Language Pre-Training with Knowledge Regularizations
To this end, we propose a Cross-modaL knOwledge-enhanced Pre-training (CLOP) method with Knowledge Regularizations.
PLATO-K: Internal and External Knowledge Enhanced Dialogue Generation
Recently, the practical deployment of open-domain dialogue systems has been plagued by the knowledge issue of information deficiency and factual inaccuracy.
FRSUM: Towards Faithful Abstractive Summarization via Enhancing Factual Robustness
We first measure a model's factual robustness by its success rate to defend against adversarial attacks when generating factual information.
UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal Guidance
Diffusion generative models have recently greatly improved the power of text-conditioned image generation.
ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts
Recent progress in diffusion models has revolutionized the popular technology of text-to-image generation.
Ranked #12 on
Text-to-Image Generation
on MS COCO
Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards
Derivative-free prompt learning has emerged as a lightweight alternative to prompt tuning, which only requires model inference to optimize the prompts.
CDConv: A Benchmark for Contradiction Detection in Chinese Conversations
Dialogue contradiction is a critical issue in open-domain dialogue systems.
Q-TOD: A Query-driven Task-oriented Dialogue System
Firstly, as the query is in the form of natural language and not confined to the schema of the knowledge base, the issue of domain adaption is alleviated remarkably in Q-TOD.
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding
Recent years have witnessed the rise and success of pre-training techniques in visually-rich document understanding.
Ranked #2 on
Semantic entity labeling
on FUNSD
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training
They attempt to learn cross-modal representation using contrastive learning on image-text pairs, however, the built inter-modal correlations only rely on a single view for each modality.
Ranked #1 on
Image Retrieval
on AIC-ICC
Towards Boosting the Open-Domain Chatbot with Human Feedback
Many open-domain dialogue models pre-trained with social media comments can generate coherent replies but have difficulties producing engaging responses when interacting with real users.
SeSQL: Yet Another Large-scale Session-level Chinese Text-to-SQL Dataset
As the first session-level Chinese dataset, CHASE contains two separate parts, i. e., 2, 003 sessions manually constructed from scratch (CHASE-C), and 3, 456 sessions translated from English SParC (CHASE-T).
GEM-2: Next Generation Molecular Property Prediction Network by Modeling Full-range Many-body Interactions
Full-range many-body interactions between electrons have been proven effective in obtaining an accurate solution of the Schr"odinger equation by classical computational chemistry methods, although modeling such interactions consumes an expensive computational cost.
HelixFold-Single: MSA-free Protein Structure Prediction by Using Protein Language Model as an Alternative
Our proposed method, HelixFold-Single, first pre-trains a large-scale protein language model (PLM) with thousands of millions of primary sequences utilizing the self-supervised learning paradigm, which will be used as an alternative to MSAs for learning the co-evolution information.
An Interpretability Evaluation Benchmark for Pre-trained Language Models
To fill in the gap, we propose a novel evaluation benchmark providing with both English and Chinese annotated data.
Link the World: Improving Open-domain Conversation with Dynamic Spatiotemporal-aware Knowledge
Making chatbots world aware in a conversation like a human is a crucial challenge, where the world may contain dynamic knowledge and spatiotemporal state.
Bi-SimCut: A Simple Strategy for Boosting Neural Machine Translation
We introduce Bi-SimCut: a simple but effective training strategy to boost neural machine translation (NMT) performance.
Ranked #1 on
Machine Translation
on WMT2014 German-English
Less Learn Shortcut: Analyzing and Mitigating Learning of Spurious Feature-Label Correlation
In this study, we focus on the spurious correlation between word features and labels that models learn from the biased data distribution of training data.
A Fine-grained Interpretability Evaluation Benchmark for Neural NLP
Based on this benchmark, we conduct experiments on three typical models with three saliency methods, and unveil their strengths and weakness in terms of interpretability.
ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrieval
Our method 1) introduces a self on-the-fly distillation method that can effectively distill late interaction (i. e., ColBERT) to vanilla dual-encoder, and 2) incorporates a cascade distillation process to further improve the performance with a cross-encoder teacher.
HelixADMET: a robust and endpoint extensible ADMET system incorporating self-supervised knowledge transfer
Additionally, the pre-trained model provided by H-ADMET can be fine-tuned to generate new and customised ADMET endpoints, meeting various demands of drug research and development requirements.
A Thorough Examination on Zero-shot Dense Retrieval
Recent years have witnessed the significant advance in dense retrieval (DR) based on powerful pre-trained language models (PLM).
Towards Multi-Turn Empathetic Dialogs with Positive Emotion Elicitation
In this task, the agent conducts empathetic responses along with the target of eliciting the user's positive emotions in the multi-turn dialog.
Where to Go for the Holidays: Towards Mixed-Type Dialogs for Clarification of User Goals
For example, users have determined the departure, the destination, and the travel time for booking a flight.
ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention
We argue that two factors, information bottleneck sensitivity and inconsistency between different attention topologies, could affect the performance of the Sparse Transformer.
Unified Structure Generation for Universal Information Extraction
Information extraction suffers from its varying targets, heterogeneous structures, and demand-specific schemas.
Ranked #5 on
Aspect-Based Sentiment Analysis (ABSA)
on ASTE
(using extra training data)
DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine
In this paper, we present DuReader_retrieval, a large-scale Chinese dataset for passage retrieval.
PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation
Despite recent progress of pre-trained language models on generating fluent text, existing methods still suffer from incoherence problems in long-form text generation tasks that require proper content control and planning to form a coherent high-level logical flow.
DU-VLG: Unifying Vision-and-Language Generation via Dual Sequence-to-Sequence Pre-training
To bridge the gap between image understanding and generation, we further design a novel commitment loss.
UNIMO-2: End-to-End Unified Vision-Language Grounded Learning
In particular, we propose to conduct grounded learning on both images and texts via a sharing grounded space, which helps bridge unaligned images and texts, and align the visual and textual semantic spaces on different types of corpora.
Long Time No See! Open-Domain Conversation with Long-Term Persona Memory
Most of the open-domain dialogue models tend to perform poorly in the setting of long-term human-bot conversations.
Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods
In this survey, we provide a systematic overview of the research progress on the faithfulness problem of NLG, including problem analysis, evaluation metrics and optimization methods.
ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation
To explore the landscape of large-scale pre-training for bidirectional text-image generation, we train a 10-billion parameter ERNIE-ViLG model on a large-scale dataset of 145 million (Chinese) image-text pairs which achieves state-of-the-art performance for both text-to-image and image-to-text tasks, obtaining an FID of 7. 9 on MS-COCO for text-to-image synthesis and best results on COCO-CN and AIC-ICC for image captioning.
Ranked #41 on
Text-to-Image Generation
on MS COCO
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
A unified framework named ERNIE 3. 0 was recently proposed for pre-training large-scale knowledge enhanced models and trained a model with 10 billion parameters.
TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue Modeling on Spoken Conversations
Task-oriented dialogue systems have been plagued by the difficulties of obtaining large-scale and high-quality annotated conversations.
DuQM: A Chinese Dataset of Linguistically Perturbed Natural Questions for Evaluating the Robustness of Question Matching Models
For this purpose, we create a Chinese dataset namely DuQM which contains natural questions with linguistic perturbations to evaluate the robustness of question matching models.
Learning with Noisy Correspondence for Cross-modal Matching
Based on this observation, we reveal and study a latent and challenging direction in cross-modal matching, named noisy correspondence, which could be regarded as a new paradigm of noisy labels.
Cross-modal retrieval with noisy correspondence
Image-text matching
+3
HelixMO: Sample-Efficient Molecular Optimization in Scene-Sensitive Latent Space
Efficient exploration of the chemical space to search the candidate drugs that satisfy various constraints is a fundamental task of drug discovery.
Docking-based Virtual Screening with Multi-Task Learning
Machine learning shows great potential in virtual screening for drug discovery.
Amendable Generation for Dialogue State Tracking
In this paper, we propose a novel Amendable Generation for Dialogue State Tracking (AG-DST), which contains a two-pass generation process: (1) generating a primitive dialogue state based on the dialogue of the current turn and the previous dialogue state, and (2) amending the primitive dialogue state from the first pass.
Ranked #1 on
Dialogue State Tracking
on Wizard-of-Oz
Dialogue State Tracking
Multi-domain Dialogue State Tracking
+1
SgSum: Transforming Multi-document Summarization into Sub-graph Selection
Comparing with traditional methods, our method has two main advantages: (1) the relations between sentences are captured by modeling both the graph structure of the whole document set and the candidate sub-graphs; (2) directly outputs an integrate summary in the form of sub-graph which is more informative and coherent.
Building Chinese Biomedical Language Models via Multi-Level Text Discrimination
In this work we introduce eHealth, a Chinese biomedical PLM built from scratch with a new pre-training framework.
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking
In this paper, we propose a novel joint training approach for dense passage retrieval and passage re-ranking.
Do What Nature Did To Us: Evolving Plastic Recurrent Neural Networks For Generalized Tasks
While artificial neural networks (ANNs) have been widely adopted in machine learning, researchers are increasingly obsessed by the gaps between ANNs and natural neural networks (NNNs).
ERNIE-SPARSE: Robust Efficient Transformer Through Hierarchically Unifying Isolated Information
The first factor is information bottleneck sensitivity, which is caused by the key feature of Sparse Transformer — only a small number of global tokens can attend to all other tokens.
PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation
To explore the limit of dialogue generation pre-training, we present the models of PLATO-XL with up to 11 billion parameters, trained on both Chinese and English social media conversations.
DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendation
In this paper, we provide a bilingual parallel human-to-human recommendation dialog dataset (DuRecDial 2. 0) to enable researchers to explore a challenging task of multilingual and cross-lingual conversational recommendation.
A Multimodal Sentiment Dataset for Video Recommendation
Based on this, we propose a multimodal sentiment analysis dataset, named baiDu Video Sentiment dataset (DuVideoSenti), and introduce a new sentiment system which is designed to describe the sentimental style of a video on recommendation scenery.
Fine-grained Entity Typing via Label Reasoning
Conventional entity typing approaches are based on independent classification paradigms, which make them difficult to recognize inter-dependent, long-tailed and fine-grained entity types.
Ranked #8 on
Entity Typing
on Open Entity
Mixup Decoding for Diverse Machine Translation
To further improve the faithfulness and diversity of the translations, we propose two simple but effective approaches to select diverse sentence pairs in the training corpus and adjust the interpolation weight for each pair correspondingly.
Evolving Decomposed Plasticity Rules for Information-Bottlenecked Meta-Learning
In contrast, biological neural networks (BNNs) can adapt to various new tasks by continually updating the neural connections based on the inputs, which is aligned with the paradigm of learning effective learning rules in addition to static parameters, e. g., meta-learning.
DuTrust: A Sentiment Analysis Dataset for Trustworthiness Evaluation
Therefore, in order to systematically evaluate the factors for building trustworthy systems, we propose a novel and well-annotated sentiment analysis dataset to evaluate robustness and interpretability.
PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval
Recently, dense passage retrieval has become a mainstream approach to finding relevant information in various natural language processing tasks.
Discovering Dialog Structure Graph for Coherent Dialog Generation
Learning discrete dialog structure graph from human-human dialogs yields basic insights into the structure of conversation, and also provides background knowledge to facilitate dialog generation.
DuReader\_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications
Machine reading comprehension (MRC) is a crucial task in natural language processing and has achieved remarkable advancements.
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
We trained the model with 10 billion parameters on a 4TB corpus consisting of plain texts and a large-scale knowledge graph.
ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Prediction
Recent advances in graph neural networks (GNNs) have shown great promise in applying GNNs for molecular representation learning.
Ranked #2 on
Molecular Property Prediction
on QM9
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression
Specifically, the first stage, General Distillation, performs distillation with guidance from pretrained teacher, gerenal data and latent distillation loss.
BASS: Boosting Abstractive Summarization with Unified Semantic Graph
Abstractive summarization for long-document or multi-document remains challenging for the Seq2Seq architecture, as Seq2Seq is not good at analyzing long-distance relations in text.
Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching
This paper proposes an approach to Dense Video Captioning (DVC) without pairwise event-sentence annotation.
A Unified Pre-training Framework for Conversational AI
In this work, we explore the application of PLATO-2 on various dialogue systems, including open-domain conversation, knowledge grounded dialogue, and task-oriented conversation.
BSTC: A Large-Scale Chinese-English Speech Translation Dataset
This corpus is expected to promote the research of automatic simultaneous translation as well as the development of practical systems.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Syntactic and Semantic-driven Learning for Open Information Extraction
One of the biggest bottlenecks in building accurate, high coverage neural open IE systems is the need for large labelled corpora.
Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsing
For better distribution matching, we require that at least 80% of SQL patterns in the training data are covered by generated queries.
Learning to Select External Knowledge with Multi-Scale Negative Sampling
The Track-1 of DSTC9 aims to effectively answer user requests or questions during task-oriented dialogues, which are out of the scope of APIs/DB.
Knowledge Distillation based Ensemble Learning for Neural Machine Translation
Under this framework, we introduce word-level ensemble learning and sequence-level ensemble learning for neural machine translation, where sequence-level ensemble learning is capable of aggregating translation models with different decoding strategies.
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora
In this paper, we propose ERNIE-M, a new training method that encourages the model to align the representation of multiple languages with monolingual corpora, to overcome the constraint that the parallel corpus size places on the model performance.
Ranked #14 on
Zero-Shot Cross-Lingual Transfer
on XTREME
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer
Transformers are not suited for processing long documents, due to their quadratically increasing memory and time consumption.
Ranked #1000000000 on
Text Classification
on IMDb
UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning
Existed pre-training methods either focus on single-modal tasks or multi-modal tasks, and cannot effectively adapt to each other.
Ranked #4 on
Image Captioning
on MS COCO
Discovering Dialog Structure Graph for Open-Domain Dialog Generation
Learning interpretable dialog structure from human-human dialogs yields basic insights into the structure of conversation, and also provides background knowledge to facilitate dialog generation.
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding
We argue that such contiguously masking method neglects to model the intra-dependencies and inter-relation of coarse-grained linguistic information.
RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering
In open-domain question answering, dense passage retrieval has become a new paradigm to retrieve relevant passages for finding answers.
Ranked #4 on
Passage Retrieval
on Natural Questions
Two-dimensional ferromagnetic semiconductor VBr3 with tunable anisotropy
Two-dimensional (2D) ferromagnets (FMs) have attracted widespread attention due to their prospects in spintronic applications.
Materials Science Strongly Correlated Electrons
Conversational Graph Grounded Policy Learning for Open-Domain Conversation Generation
To address the challenge of policy learning in open-domain multi-turn conversation, we propose to represent prior information about dialog transitions as a graph and learn a graph grounded dialog policy, aimed at fostering a more coherent and controllable dialog.
ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph
Thus, ERNIE-ViL can learn the joint representations characterizing the alignments of the detailed semantics across vision and language.
Ranked #2 on
Visual Question Answering (VQA)
on VCR (QA-R) test
PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning
To build a high-quality open-domain chatbot, we introduce the effective training process of PLATO-2 via curriculum learning.
Leveraging Graph to Improve Abstractive Multi-Document Summarization
Graphs that capture relations between textual units have great benefits for detecting salient information from multiple documents and generating overall coherent summaries.
SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis
In particular, the prediction of aspect-sentiment pairs is converted into multi-label classification, aiming to capture the dependency between words in a pair.
Ranked #14 on
Stock Market Prediction
on Astock
Towards Conversational Recommendation over Multi-Type Dialogs
We propose a new task of conversational recommendation over multi-type dialogs, where the bots can proactively and naturally lead a conversation from a non-recommendation dialog (e. g., QA) to a recommendation dialog, taking into account user's interests and feedback.
DuReader_robust: A Chinese Dataset Towards Evaluating Robustness and Generalization of Machine Reading Comprehension in Real-World Applications
Machine reading comprehension (MRC) is a crucial task in natural language processing and has achieved remarkable advancements.
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation
Current pre-training works in natural language generation pay little attention to the problem of exposure bias on downstream tasks.
Ranked #1 on
Question Generation
on SQuAD1.1
(using extra training data)
Synchronous Speech Recognition and Speech-to-Text Translation with Interactive Decoding
Speech-to-text translation (ST), which translates source language speech into target language text, has attracted intensive attention in recent years.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
CoKE: Contextualized Knowledge Graph Embedding
This work presents Contextualized Knowledge Graph Embedding (CoKE), a novel paradigm that takes into account such contextual nature, and learns dynamic, flexible, and fully contextualized entity and relation embeddings.
D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension
In this paper, we introduce a simple system Baidu submitted for MRQA (Machine Reading for Question Answering) 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models.
Enhancing Local Feature Extraction with Global Representation for Neural Text Classification
For text classification, traditional local feature driven models learn long dependency by deeply stacking or hybrid modeling.
PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable
Pre-training models have been proved effective for a wide range of natural language processing tasks.
Multi-agent Learning for Neural Machine Translation
Conventional Neural Machine Translation (NMT) models benefit from the training with an additional agent, e. g., dual learning, and bidirectional decoding with one agent decoding from left to right and the other decoding in the opposite direction.
Baidu Neural Machine Translation Systems for WMT19
In this paper we introduce the systems Baidu submitted for the WMT19 shared task on Chinese{\textless}-{\textgreater}English news translation.
DuTongChuan: Context-aware Translation Model for Simultaneous Interpreting
In this paper, we present DuTongChuan, a novel context-aware translation model for simultaneous interpreting.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
Recently, pre-trained models have achieved state-of-the-art results in various language understanding tasks, which indicates that pre-training on large-scale corpora may play a crucial role in natural language processing.
Ranked #1 on
Chinese Sentence Pair Classification
on LCQMC Dev
Chinese Named Entity Recognition
Chinese Reading Comprehension
+8
Proactive Human-Machine Conversation with Explicit Conversation Goal
Konv enables a very challenging task as the model needs to both understand dialogue and plan over the given knowledge graph.
Enhancing Pre-Trained Language Representations with Rich Knowledge for Machine Reading Comprehension
In this work, we investigate the potential of leveraging external knowledge bases (KBs) to further improve BERT for MRC.
ARNOR: Attention Regularization based Noise Reduction for Distant Supervision Relation Classification
In this paper, we propose ARNOR, a novel Attention Regularization based NOise Reduction framework for distant supervision relation classification.
Proactive Human-Machine Conversation with Explicit Conversation Goals
DuConv enables a very challenging task as the model needs to both understand dialogue and plan over the given knowledge graph.
Generating Multiple Diverse Responses with Multi-Mapping and Posterior Mapping Selection
In this paper, we propose a multi-mapping mechanism to better capture the one-to-many relationship, where multiple mapping modules are employed as latent mechanisms to model the semantic mappings from an input post to its diverse responses.
Know More about Each Other: Evolving Dialogue Strategy via Compound Assessment
In this paper, a novel Generation-Evaluation framework is developed for multi-turn conversations with the objective of letting both participants know more about each other.
ERNIE: Enhanced Representation through Knowledge Integration
We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration).
Ranked #3 on
Natural Language Inference
on XNLI Chinese Dev
Chinese Named Entity Recognition
Chinese Sentence Pair Classification
+8
End-to-End Speech Translation with Knowledge Distillation
End-to-end speech translation (ST), which directly translates from source language speech into target language text, has attracted intensive attentions in recent years.
Knowledge Aware Conversation Generation with Explainable Reasoning over Augmented Graphs
Two types of knowledge, triples from knowledge graphs and texts from documents, have been studied for knowledge aware open-domain conversation generation, in which graph paths can narrow down vertex candidates for knowledge selection decision, and texts can provide rich information for response generation.
Learning to Select Knowledge for Response Generation in Dialog Systems
Specifically, a posterior distribution over knowledge is inferred from both utterances and responses, and it ensures the appropriate selection of knowledge during the training process.
Modeling Coherence for Discourse Neural Machine Translation
Discourse coherence plays an important role in the translation of one text.
STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework
Simultaneous translation, which translates sentences before they are finished, is useful in many scenarios but is notoriously difficult due to word-order differences.
Addressing Troublesome Words in Neural Machine Translation
One of the weaknesses of Neural Machine Translation (NMT) is in handling lowfrequency and ambiguous words, which we refer as troublesome words.
Familia: A Configurable Topic Modeling Framework for Industrial Text Engineering
In order to relieve burdens of software engineers without knowledge of Bayesian networks, Familia is able to conduct automatic parameter inference for a variety of topic models.
Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network
Human generates responses relying on semantic and functional dependencies, including coreference relation, among dialogue elements and their context.
Ranked #6 on
Conversational Response Selection
on RRS
Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification
Machine reading comprehension (MRC) on real web data usually requires the machine to answer a question by analyzing multiple passages retrieved by search engine.
Ranked #3 on
Question Answering
on MS MARCO
A New Method of Region Embedding for Text Classification
In this paper, we propose a new method of learning and utilizing task-specific distributed representations of n-grams, referred to as “region embeddings”.
Multi-channel Encoder for Neural Machine Translation
This design of encoder yields relatively uniform composition on source sentence, despite the gating mechanism employed in encoding RNN.
DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
Experiments show that human performance is well above current state-of-the-art baseline systems, leaving plenty of room for the community to make improvements.
An End-to-End Model for Question Answering over Knowledge Base with Cross-Attention Combining Global Knowledge
This simple representation strategy is not easy to express the proper information in the question.
Latent Topic Embedding
Topic modeling and word embedding are two important techniques for deriving latent semantics from data.
Chinese Poetry Generation with Planning based Neural Network
Chinese poetry generation is a very challenging task in natural language processing.
Semi-Supervised Learning for Neural Machine Translation
While end-to-end neural machine translation (NMT) has made remarkable progress recently, NMT systems only rely on parallel corpora for parameter estimation.
Question Answering over Knowledge Base with Neural Attention Combining Global Knowledge Information
With the rapid growth of knowledge bases (KBs) on the web, how to take full advantage of them becomes increasingly important.
Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation
The attentional mechanism has proven to be effective in improving end-to-end neural machine translation.
Minimum Risk Training for Neural Machine Translation
We propose minimum risk training for end-to-end neural machine translation.
