Search Results for author:
Found 98 papers, 41 papers with code
Tencent Translation System for the WMT21 News Translation Task
Based on our success in the last WMT, we continuously employed advanced techniques such as large batch training, data selection and data filtering.
RAST: Domain-Robust Dialogue Rewriting as Sequence Tagging
The task of dialogue rewriting aims to reconstruct the latest dialogue utterance by copying the missing content from the dialogue context.
Tencent Neural Machine Translation Systems for the WMT20 News Translation Task
This paper describes Tencent Neural Machine Translation systems for the WMT 2020 news translation tasks.
Redistributing Low-Frequency Words: Making the Most of Monolingual Data in Non-Autoregressive Translation
In this work, we provide an appealing alternative for NAT – monolingual KD, which trains NAT student on external monolingual data with AT teacher trained on the original bilingual data.
Tencent AI Lab Machine Translation Systems for WMT20 Chat Translation Task
This paper describes the Tencent AI Lab’s submission of the WMT 2020 shared task on chat translation in English-German.
Tencent AI Lab Machine Translation Systems for the WMT20 Biomedical Translation Task
This paper describes the Tencent AI Lab submission of the WMT2020 shared task on biomedical translation in four language directions: German<->English, English<->German, Chinese<->English and English<->Chinese.
Tencent AI Lab Machine Translation Systems for the WMT21 Biomedical Translation Task
This paper describes the Tencent AI Lab submission of the WMT2021 shared task on biomedical translation in eight language directions: English-German, English-French, English-Spanish and English-Russian.
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
To address the DoT problem, we propose a Multi-Agent Debate (MAD) framework, in which multiple agents express their arguments in the state of "tit for tat" and a judge manages the debate process to obtain a final solution.
Revisiting Non-Autoregressive Translation at Scale
Experimental results on the large-scale WMT20 En-De show that the asymmetric architecture (e. g. bigger encoder and smaller decoder) can achieve comparable performance with the scaling model, while maintaining the superiority of decoding speed with standard NAT models.
Cross-modality Data Augmentation for End-to-End Sign Language Translation
It has been a challenging task due to the modality gap between sign videos and texts and the data scarcity of labeled data.
Ranked #3 on
Sign Language Translation
on CSL-Daily
A Survey on Zero Pronoun Translation
Zero pronouns (ZPs) are frequently omitted in pro-drop languages (e. g. Chinese, Hungarian, and Hindi), but should be recalled in non-pro-drop languages (e. g. English).
Exploring Human-Like Translation Strategy with Large Language Models
Large language models (LLMs) have demonstrated impressive capabilities in general scenarios, exhibiting a level of aptitude that approaches, in some aspects even surpasses, human-level intelligence.
Document-Level Machine Translation with Large Language Models
Large language models (LLMs) such as Chat-GPT can produce coherent, cohesive, relevant, and fluent answers for various natural language processing (NLP) tasks.
ParroT: Translating During Chat Using Large Language Models
Therefore, we propose the $\mathbf{ParroT}$ framework to enhance and regulate the translation abilities during chat based on open-sourced LLMs (i. e., LLaMA-7b, BLOOMZ-7b-mt) and human written translation and evaluation data.
Search-Engine-augmented Dialogue Response Generation with Cheaply Supervised Query Production
We propose a dialogue model that can access the vast and dynamic information from any search engine for response generation.
Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine
By evaluating on a number of benchmark test sets, we find that ChatGPT performs competitively with commercial translation products (e. g., Google Translate) on high-resource European languages but lags behind significantly on low-resource or distant languages.
Adapters for Enhanced Modeling of Multilingual Knowledge and Text
Specifically, we introduce a lightweight adapter set to enhance MLLMs with cross-lingual entity alignment and facts from MLKGs for many languages.
Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languages
This paper describes Tencent's multilingual machine translation systems for the WMT22 shared task on Large-Scale Machine Translation Evaluation for African Languages.
Tencent AI Lab - Shanghai Jiao Tong University Low-Resource Translation System for the WMT22 Translation Task
Finally, our unconstrained system achieves BLEU scores of 17. 0 and 30. 4 for English to/from Livonian.
Scaling Back-Translation with Domain Text Generation for Sign Language Gloss Translation
In this paper, to overcome the limitation, we propose a Prompt based domain text Generation (PGEN) approach to produce the large-scale in-domain spoken language text data.
ngram-OAXE: Phrase-Based Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation
Recently, a new training oaxe loss has proven effective to ameliorate the effect of multimodality for non-autoregressive translation (NAT), which removes the penalty of word order errors in the standard cross-entropy loss.
A Template-based Method for Constrained Neural Machine Translation
In this work, we propose a template-based method that can yield results with high translation quality and match accuracy and the inference speed of our method is comparable with unconstrained NMT models.
Understanding and Mitigating the Uncertainty in Zero-Shot Translation
Zero-shot translation is a promising direction for building a comprehensive multilingual neural machine translation (MNMT) system.
Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation
In this paper, we present a substantial step in better understanding the SOTA sequence-to-sequence (Seq2Seq) pretraining for neural machine translation~(NMT).
Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translation
By carefully designing experiments, we identify two representative characteristics of the data gap in source: (1) style gap (i. e., translated vs. natural text style) that leads to poor generalization capability; (2) content gap that induces the model to produce hallucination content biased towards the target language.
On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation
Pre-training (PT) and back-translation (BT) are two simple and powerful methods to utilize monolingual data for improving the model performance of neural machine translation (NMT).
On the Copying Behaviors of Pre-Training for Neural Machine Translation
In response to this problem, we propose a simple and effective method named copying penalty to control the copying behaviors in decoding.
Language Models are Good Translators
Inspired by the recent progress of large-scale pre-trained language models on machine translation in a limited scenario, we firstly demonstrate that a single language model (LM4MT) can achieve comparable performance with strong encoder-decoder NMT models on standard machine translation benchmarks, using the same training data and similar amount of model parameters.
Progressive Multi-Granularity Training for Non-Autoregressive Translation
Non-autoregressive translation (NAT) significantly accelerates the inference process via predicting the entire target sequence.
Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation
We propose a new training objective named order-agnostic cross entropy (OaXE) for fully non-autoregressive translation (NAT) models.
On the Language Coverage Bias for Neural Machine Translation
Language coverage bias, which indicates the content-dependent differences between sentence pairs originating from the source and target languages, is important for neural machine translation (NMT) because the target-original training data is not well exploited in current practice.
Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation
In this work, we propose to improve the sampling procedure by selecting the most informative monolingual sentences to complement the parallel data.
Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation
Results demonstrate that the proposed approach can significantly and universally improve translation quality by reducing translation errors on low-frequency words.
TranSmart: A Practical Interactive Machine Translation System
Automatic machine translation is super efficient to produce translations yet their quality is not guaranteed.
Understanding and Improving Lexical Choice in Non-Autoregressive Translation
To this end, we introduce an extra Kullback-Leibler divergence term derived by comparing the lexical choice of NAT model and that embedded in the raw data.
Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning
Encoder layer fusion (EncoderFusion) is a technique to fuse all the encoder layers (instead of the uppermost layer) for sequence-to-sequence (Seq2Seq) models, which has proven effective on various NLP tasks.
Robust Dialogue Utterance Rewriting as Sequence Tagging
The task of dialogue rewriting aims to reconstruct the latest dialogue utterance by copying the missing content from the dialogue context.
Emotion Classification by Jointly Learning to Lexiconize and Classify
Emotion lexicons have been shown effective for emotion classification (Baziotis et al., 2018).
EmpDG: Multi-resolution Interactive Empathetic Dialogue Generation
In response to this problem, we propose a multi-resolution adversarial model {--} EmpDG, to generate more empathetic responses.
Rethinking the Value of Transformer Components
Transformer becomes the state-of-the-art translation model, while it is not well studied how each intermediate component contributes to the model performance, which poses significant challenges for designing optimal architectures.
Context-Aware Cross-Attention for Non-Autoregressive Translation
Non-autoregressive translation (NAT) significantly accelerates the inference process by predicting the entire target sequence.
Multi-Task Learning with Shared Encoder for Non-Autoregressive Machine Translation
In addition, experimental results demonstrate that our Multi-Task NAT is complementary to knowledge distillation, the standard knowledge transfer method for NAT.
On the Sparsity of Neural Machine Translation Models
Modern neural machine translation (NMT) models employ a large number of parameters, which leads to serious over-parameterization and typically causes the underutilization of computational resources.
Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translation
First, we train an identification model on the original training data, and use it to distinguish inactive examples and active examples by their sentence-level output probabilities.
On the Sub-Layer Functionalities of Transformer Decoder
There have been significant efforts to interpret the encoder of Transformer-based encoder-decoder architectures for neural machine translation (NMT); meanwhile, the decoder remains largely unexamined despite its critical role.
On the Inference Calibration of Neural Machine Translation
Confidence calibration, which aims to make model predictions equal to the true correctness measures, is important for neural machine translation (NMT) because it is able to offer useful indicators of translation errors in the generated output.
How Does Selective Mechanism Improve Self-Attention Networks?
Self-attention networks (SANs) with selective mechanism has produced substantial improvements in various NLP tasks by concentrating on a subset of input words.
Assessing the Bilingual Knowledge Learned by Neural Machine Translation Models
In this paper, we bridge the gap by assessing the bilingual knowledge learned by NMT models with phrase table -- an interpretable table of bilingual lexicons.
Neuron Interaction Based Representation Composition for Neural Machine Translation
Starting from this intuition, we propose a novel approach to compose representations learned by different components in neural machine translation (e. g., multi-layer networks or multi-head attention), based on modeling strong interactions among neurons in the representation vectors.
Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks
The key challenge of multi-domain translation lies in simultaneously encoding both the general knowledge shared across domains and the particular knowledge distinctive to each domain in a unified model.
EmpDG: Multiresolution Interactive Empathetic Dialogue Generation
In response to this problem, we propose a multi-resolution adversarial model -- EmpDG, to generate more empathetic responses.
Retrieval-guided Dialogue Response Generation via a Matching-to-Generation Framework
End-to-end sequence generation is a popular technique for developing open domain dialogue systems, though they suffer from the \textit{safe response problem}.
Multi-Granularity Self-Attention for Neural Machine Translation
Current state-of-the-art neural machine translation (NMT) uses a deep multi-head self-attention network with no explicit phrase information.
Towards Better Modeling Hierarchical Structure for Self-Attention with Ordered Neurons
Recent studies have shown that a hybrid of self-attention networks (SANs) and recurrent neural networks (RNNs) outperforms both individual architectures, while not much is known about why the hybrid models work.
Towards Understanding Neural Machine Translation with Word Importance
Although neural machine translation (NMT) has advanced the state-of-the-art on various language pairs, the interpretability of NMT remains unsatisfactory.
One Model to Learn Both: Zero Pronoun Prediction and Translation
In this paper, we propose a unified and discourse-aware ZP translation approach for neural MT models.
Self-Attention with Structural Position Representations
Although self-attention networks (SANs) have advanced the state-of-the-art on various NLP tasks, one criticism of SANs is their ability of encoding positions of input words (Shaw et al., 2018).
Exploiting Sentential Context for Neural Machine Translation
In this work, we present novel approaches to exploit sentential context for neural machine translation (NMT).
Assessing the Ability of Self-Attention Networks to Learn Word Order
Self-attention networks (SAN) have attracted a lot of interests due to their high parallelization and strong performance on a variety of NLP tasks, e. g. machine translation.
Dynamic Past and Future for Neural Machine Translation
Previous studies have shown that neural machine translation (NMT) models can benefit from explicitly modeling translated (Past) and untranslated (Future) to groups of translated and untranslated contents through parts-to-wholes assignment.
Convolutional Self-Attention Networks
Self-attention networks (SANs) have drawn increasing interest due to their high parallelization in computation and flexibility in modeling dependencies.
Modeling Recurrence for Transformer
In addition to the standard recurrent neural network, we introduce a novel attentive recurrent network to leverage the strengths of both attention and recurrent networks.
Information Aggregation for Multi-Head Attention with Routing-by-Agreement
Multi-head attention is appealing for its ability to jointly extract different types of information from multiple representation subspaces.
Context-Aware Self-Attention Networks
Self-attention model have shown its flexibility in parallel computation and the effectiveness on modeling both long- and short-term dependencies.
Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement
With the promising progress of deep neural networks, layer aggregation has been used to fuse information across layers in various fields, such as computer vision and machine translation.
Learning to Refine Source Representations for Neural Machine Translation
Neural machine translation (NMT) models generally adopt an encoder-decoder architecture for modeling the entire translation process.
Neural Machine Translation with Adequacy-Oriented Learning
Although Neural Machine Translation (NMT) models have advanced state-of-the-art performance in machine translation, they face problems like the inadequate translation.
Ranked #34 on
Machine Translation
on WMT2014 English-German
Convolutional Self-Attention Network
Self-attention network (SAN) has recently attracted increasing interest due to its fully parallelized computation and flexibility in modeling dependencies.
Exploiting Deep Representations for Neural Machine Translation
Advanced neural machine translation (NMT) models generally implement encoder and decoder as multiple layers, which allows systems to model complex functions and capture complicated linguistic structures.
Modeling Localness for Self-Attention Networks
Self-attention networks have proven to be of profound value for its strength of capturing global dependencies.
Ranked #28 on
Machine Translation
on WMT2014 English-German
Multi-Head Attention with Disagreement Regularization
Multi-head attention is appealing for the ability to jointly attend to information from different representation subspaces at different positions.
Learning to Jointly Translate and Predict Dropped Pronouns with a Shared Reconstruction Mechanism
Pronouns are frequently omitted in pro-drop languages, such as Chinese, generally leading to significant challenges with respect to the production of complete translations.
Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory
Such models rely on insufficient information for generating a specific response since a certain query could be answered in multiple ways.
Neural Machine Translation with Key-Value Memory-Augmented Attention
Although attention-based Neural Machine Translation (NMT) has achieved remarkable progress in recent years, it still suffers from issues of repeating and dropping translations.
Target Foresight Based Attention for Neural Machine Translation
In neural machine translation, an attention model is used to identify the aligned source words for a target word (target foresight word) in order to select translation context, but it does not make use of any information of this target foresight word at all.
Towards Robust Neural Machine Translation
Small perturbations in the input can severely distort intermediate representations and thus impact translation quality of neural machine translation (NMT) models.
Generative Stock Question Answering
We study the problem of stock related question answering (StockQA): automatically generating answers to stock related questions, just like professional stock analysts providing action recommendations to stocks upon user's requests.
Translating Pro-Drop Languages with Reconstruction Models
Next, the annotated source sentence is reconstructed from hidden representations in the NMT model.
Modeling Past and Future for Neural Machine Translation
The Past and Future contents are fed to both the attention model and the decoder states, which offers NMT systems the knowledge of translated and untranslated contents.
Learning to Remember Translation History with a Continuous Cache
Existing neural machine translation (NMT) models generally translate sentences in isolation, missing the opportunity to take advantage of document-level information.
Semantics-Enhanced Task-Oriented Dialogue Translation: A Case Study on Hotel Booking
We showcase TODAY, a semantics-enhanced task-oriented dialogue translation system, whose novelties are: (i) task-oriented named entity (NE) definition and a hybrid strategy for NE recognition and translation; and (ii) a novel grounded semantic method for dialogue understanding and task-order management.
Translating Phrases in Neural Machine Translation
Otherwise, the NMT decoder generates a word from the vocabulary as the general NMT decoder does.
Chunk-Based Bi-Scale Decoder for Neural Machine Translation
In typical neural machine translation~(NMT), the decoder generates a sentence word by word, packing all linguistic granularities in the same time-scale of RNN.
Modeling Source Syntax for Neural Machine Translation
Even though a linguistics-free sequence to sequence model in neural machine translation (NMT) has certain capability of implicitly learning syntactic information of source sentences, this paper shows that source syntax can be explicitly incorporated into NMT effectively to provide further improvements.
Exploiting Cross-Sentence Context for Neural Machine Translation
In translation, considering the document as a whole can help to resolve ambiguities and inconsistencies.
Neural Machine Translation with Reconstruction
Although end-to-end Neural Machine Translation (NMT) has achieved remarkable progress in the past two years, it suffers from a major drawback: translations generated by NMT systems often lack of adequacy.
Neural Machine Translation Advised by Statistical Machine Translation
Neural Machine Translation (NMT) is a new approach to machine translation that has made great progress in recent years.
Context Gates for Neural Machine Translation
In neural machine translation (NMT), generation of a target word depends on both source and target contexts.
Automatic Construction of Discourse Corpora for Dialogue Translation
Then tags such as speaker and discourse boundary from the script data are projected to its subtitle data via an information retrieval approach in order to map monolingual discourse to bilingual texts.
A Novel Approach to Dropped Pronoun Translation
Finally, we integrate the above outputs into our translation system to recall missing pronouns by both extracting rules from the DP-labelled training data and translating the DP-generated input sentences.
Modeling Coverage for Neural Machine Translation
Attention mechanism has enhanced state-of-the-art Neural Machine Translation (NMT) by jointly learning to align and translate.
A Deep Memory-based Architecture for Sequence-to-Sequence Learning
We propose DEEPMEMORY, a novel deep architecture for sequence-to-sequence learning, which performs the task through a series of nonlinear transformations from the representation of the input sequence (e. g., a Chinese sentence) to the final output sequence (e. g., translation to English).
Context-Dependent Translation Selection Using Convolutional Neural Network
We propose a novel method for translation selection in statistical machine translation, in which a convolutional neural network is employed to judge the similarity between a phrase pair in two languages.
