Search Results for author:
Found 92 papers, 28 papers with code
RoBLEURT Submission for WMT2021 Metrics Task
After investigating the recent advances of trainable metrics, we conclude several aspects of vital importance to obtain a well-performed metric model by: 1) jointly leveraging the advantages of source-included model and reference-only model, 2) continuously pre-training the model with massive synthetic data pairs, and 3) fine-tuning the model with data denoising strategy.
Bilingual Methods for Adaptive Training Data Selection for Machine Translation
In this paper, we propose a new data selection method which uses semi-supervised convolutional neural networks based on bitokens (Bi-SSCNNs) for training machine translation systems from a large bilingual corpus.
Mutual-Learning Improves End-to-End Speech Translation
In this paper, we propose an alternative–a trainable mutual-learning scenario, where the MT and the ST models are collaboratively trained and are considered as peers, rather than teacher/student.
TermMind: Alibaba’s WMT21 Machine Translation Using Terminologies Task Submission
We used tags to mark and add the term translations into the matched sentences.
QEMind: Alibaba’s Submission to the WMT21 Quality Estimation Shared Task
Quality Estimation, as a crucial step of quality control for machine translation, has been explored for years.
Challenges of Neural Machine Translation for Short Texts
After empirically investigating the rationale behind this, we summarize two challenges in NMT for STs associated with translation error types above, respectively: (1) the imbalanced length distribution in training set intensifies model inference calibration over STs, leading to more over-translation cases on STs; and (2) the lack of contextual information forces NMT to have higher data uncertainty on short sentences, and thus NMT model is troubled by considerable mistranslation errors.
Alibaba Speech Translation Systems for IWSLT 2018
This work describes the En→De Alibaba speech translation system developed for the evaluation campaign of the International Workshop on Spoken Language Translation (IWSLT) 2018.
GCPG: A General Framework for Controllable Paraphrase Generation
Under GCPG, we reconstruct commonly adopted lexical condition (i. e., Keywords) and syntactical conditions (i. e., Part-Of-Speech sequence, Constituent Tree, Masked Template and Sentential Exemplar) and study the combination of the two types.
Balcony: A Lightweight Approach to Dynamic Inference of Generative Language Models
By freezing the pretrained LLM and inserting additional transformer layers at selected exit points, Balcony maintains the full model's performance while enabling real-time adaptation to different computational budgets.
R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning
Despite recent breakthroughs in reasoning-enhanced large language models (LLMs) like DeepSeek-R1, incorporating inference-time reasoning into machine translation (MT), where human translators naturally employ structured, multi-layered reasoning chain-of-thoughts (CoTs), is yet underexplored.
ZETA: Leveraging Z-order Curves for Efficient Top-k Attention
However, causal masks require the current query token to only attend to past tokens, preventing the existing top-$k$ attention method from efficiently searching for the most relevant tokens in parallel, thereby limiting training efficiency.
Rethinking Post-Training Quantization: Introducing a Statistical Pre-Calibration Approach
State-of-the-art Post-training Quantization (PTQ) techniques often rely on calibration processes to maintain the accuracy of these models.
Batch-Max: Higher LLM Throughput using Larger Batch Sizes and KV Cache Compression
Several works have developed eviction policies to remove key-value (KV) pairs from the KV cache for more efficient inference.
Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge
The increasing complexity of computer systems necessitates innovative approaches to fault and error management, going beyond traditional manual log analysis.
Do Robot Snakes Dream like Electric Sheep? Investigating the Effects of Architectural Inductive Biases on Hallucination
Do they affect how and where they occur?
LogLM: From Task-based to Instruction-based Automated Log Analysis
Automatic log analysis is essential for the efficient Operation and Maintenance (O&M) of software systems, providing critical insights into system behaviors.
Draft on the Fly: Adaptive Self-Speculative Decoding using Cosine Similarity
We present a simple on the fly method for faster inference of large language models.
EchoAtt: Attend, Copy, then Adjust for More Efficient Large Language Models
Large Language Models (LLMs), with their increasing depth and number of parameters, have demonstrated outstanding performance across a variety of natural language processing tasks.
What Do You Want? User-centric Prompt Generation for Text-to-image Synthesis via Multi-turn Guidance
To achieve this, we mined 15 essential dimensions for high-quality prompts from advanced users and curated a multi-turn dataset.
S2D: Sorted Speculative Decoding For More Efficient Deployment of Nested Large Language Models
In SD, usually, one draft model is used to serve a specific target model; however, in practice, LLMs are diverse, and we might need to deal with many target models or more than one target model simultaneously.
Single Parent Family: A Spectrum of Family Members from a Single Pre-Trained Foundation Model
This paper introduces a novel method of Progressive Low Rank Decomposition (PLRD) tailored for the compression of large language models.
EWEK-QA: Enhanced Web and Efficient Knowledge Graph Retrieval for Citation-based Question Answering Systems
To mitigate these issues, we propose our enhanced web and efficient knowledge graph (KG) retrieval solution (EWEK-QA) to enrich the content of the extracted knowledge fed to the system.
CHIQ: Contextual History Enhancement for Improving Query Rewriting in Conversational Search
In this paper, we study how open-source large language models (LLMs) can be effectively deployed for improving query rewriting in conversational search, especially for ambiguous queries.
OTTAWA: Optimal TransporT Adaptive Word Aligner for Hallucination and Omission Translation Errors Detection
In this work, we introduce OTTAWA, a novel Optimal Transport (OT)-based word aligner specifically designed to enhance the detection of hallucinations and omissions in MT systems.
CHARP: Conversation History AwaReness Probing for Knowledge-grounded Dialogue Systems
In this work, we dive deep into one of the popular knowledge-grounded dialogue benchmarks that focus on faithfulness, FaithDial.
OAC: Output-adaptive Calibration for Accurate Post-training Quantization
The Hessian is also used for detecting the most salient weights to quantization.
AdpQ: A Zero-shot Calibration Free Adaptive Post Training Quantization Method for LLMs
This paper presents AdpQ, a novel zero-shot adaptive PTQ method for LLMs that achieves the state-of-the-art performance in low-precision quantization (e. g. 3-bit) without requiring any calibration data.
An Efficient End-to-End Approach to Noise Invariant Speech Features via Multi-Task Learning
Lastly, we show that the proposed recipe can be applied to other distillation methodologies, such as the recent DPWavLM.
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation
Given the significant resource allocation required for training and evaluating models, it is advantageous to have an efficient method for selecting high-quality IT data.
QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
Finetuning large language models requires huge GPU memory, restricting the choice to acquire Larger models.
On the importance of Data Scale in Pretraining Arabic Language Models
Pretraining monolingual language models have been proven to be vital for performance in Arabic Natural Language Processing (NLP) tasks.
"Knowing When You Don't Know": A Multilingual Relevance Assessment Dataset for Robust Retrieval-Augmented Generation
NoMIRACL includes both a non-relevant and a relevant subset.
Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models
The benefit of using integers for outlier values is that it enables us to use operator tiling to avoid performing 16-bit integer matrix multiplication to address this problem effectively.
Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference
We extend SortedNet to generative NLP tasks, making large language models dynamic without any Pre-Training and by only replacing Standard Fine-Tuning (SFT) with Sorted Fine-Tuning (SoFT).
SortedNet: A Scalable and Generalized Framework for Training Modular Deep Neural Networks
Deep neural networks (DNNs) must cater to a variety of users with different performance needs and budgets, leading to the costly practice of training, storing, and maintaining numerous user/task-specific models.
On the Transferability of Whisper-based Representations for "In-the-Wild" Cross-Task Downstream Speech Applications
Large self-supervised pre-trained speech models have achieved remarkable success across various speech-processing tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Translate the Beauty in Songs: Jointly Learning to Align Melody and Translate Lyrics
Song translation requires both translation of lyrics and alignment of music notes so that the resulting verse can be sung to the accompanying melody, which is a challenging problem that has attracted some interests in different aspects of the translation process.
One Adapter for All Programming Languages? Adapter Tuning for Code Search and Summarization
To alleviate the potentially catastrophic forgetting issue in multilingual models, we fix all pre-trained model parameters, insert the parameter-efficient structure adapter, and fine-tune it.
Mathematical Challenges in Deep Learning
Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012.
Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inference
A popular approach to streaming speech translation is to employ a single offline model with a wait-k policy to support different latency requirements, which is simpler than training multiple online models with different latency constraints.
RobustDistiller: Compressing Universal Speech Representations for Enhanced Environment Robustness
The proposed layer-wise distillation recipe is evaluated on top of three well-established universal representations, as well as with three downstream tasks.
Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation
Existing zero-shot methods fail to align the two modalities of speech and text into a shared semantic space, resulting in much worse performance compared to the supervised ST methods.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Non-Parametric Domain Adaptation for End-to-End Speech Translation
End-to-End Speech Translation (E2E-ST) has received increasing attention due to the potential of its less error propagation, lower latency, and fewer parameters.
UniTE: Unified Translation Evaluation
Translation quality evaluation plays a crucial role in machine translation.
RoBLEURT Submission for the WMT2021 Metrics Task
After investigating the recent advances of trainable metrics, we conclude several aspects of vital importance to obtain a well-performed metric model by: 1) jointly leveraging the advantages of source-included model and reference-only model, 2) continuously pre-training the model with massive synthetic data pairs, and 3) fine-tuning the model with data denoising strategy.
Attention Mechanism with Energy-Friendly Operations
Attention mechanism has become the dominant module in natural language processing models.
Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation
We experimentally find that these prompts can be simply concatenated as a whole to multi-attribute CTG without any re-training, yet raises problems of fluency decrease and position sensitivity.
Efficient Cluster-Based k-Nearest-Neighbor Machine Translation
k-Nearest-Neighbor Machine Translation (kNN-MT) has been recently proposed as a non-parametric solution for domain adaptation in neural machine translation (NMT).
PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization
Few-shot abstractive summarization has become a challenging task in natural language generation.
DePA: Improving Non-autoregressive Machine Translation with Dependency-Aware Decoder
We propose a novel and general Dependency-Aware Decoder (DePA) to enhance target dependency modeling in the decoder of fully NAT models from two perspectives: decoder self-attention and decoder input.
QEMind: Alibaba's Submission to the WMT21 Quality Estimation Shared Task
Quality Estimation, as a crucial step of quality control for machine translation, has been explored for years.
Regularizing End-to-End Speech Translation with Triangular Decomposition Agreement
In this paper, we attempt to model the joint probability of transcription and translation based on the speech input to directly leverage such triplet data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Unifying Cross-lingual Summarization and Machine Translation with Compression Rate
Through introducing compression rate, the information ratio between the source and the target text, we regard the MT task as a special CLS task with a compression rate of 100%.
Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation
Recently, $k$NN-MT has shown the promising capability of directly incorporating the pre-trained neural machine translation (NMT) model with domain-specific token-level $k$-nearest-neighbor ($k$NN) retrieval to achieve domain adaptation without retraining.
Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variables
However, it usually suffers from capturing spurious correlations between the output language and language invariant semantics due to the maximum likelihood training objective, leading to poor transfer performance on zero-shot translation.
Task-Oriented Dialogue System as Natural Language Generation
In this paper, we propose to formulate the task-oriented dialogue system as the purely natural language generation task, so as to fully leverage the large-scale pre-trained models like GPT-2 and simplify complicated delexicalization prepossessing.
Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training
To this end, we propose two pre-training tasks.
Context-Interactive Pre-Training for Document Machine Translation
Document machine translation aims to translate the source sentence into the target language in the presence of additional contextual information.
Continual Learning for Neural Machine Translation
In practical applications, NMT models are usually trained on a general domain corpus and then fine-tuned by continuing training on the in-domain corpus.
G-Transformer for Document-level Machine Translation
However, study shows that when we further enlarge the translation unit to a whole document, supervised training of Transformer can fail.
Adaptive Nearest Neighbor Machine Translation
On four benchmark machine translation datasets, we demonstrate that the proposed method is able to effectively filter out the noises in retrieval results and significantly outperforms the vanilla kNN-MT model.
Towards Variable-Length Textual Adversarial Attacks
In this way, our approach is able to more comprehensively find adversarial examples around the decision boundary and effectively conduct adversarial attacks.
Domain Transfer based Data Augmentation for Neural Query Translation
Query translation (QT) serves as a critical factor in successful cross-lingual information retrieval (CLIR).
Exploiting Neural Query Translation into Cross Lingual Information Retrieval
As a crucial role in cross-language information retrieval (CLIR), query translation has three main challenges: 1) the adequacy of translation; 2) the lack of in-domain parallel training data; and 3) the requisite of low latency.
Constraint Translation Candidates: A Bridge between Neural Query Translation and Cross-lingual Information Retrieval
Query translation (QT) is a key component in cross-lingual information retrieval system (CLIR).
Incorporating BERT into Parallel Sequence Decoding with Adapters
Our framework is based on a parallel sequence decoding algorithm named Mask-Predict considering the bi-directional and conditional independent nature of BERT, and can be adapted to traditional autoregressive decoding easily.
Self-Paced Learning for Neural Machine Translation
Recent studies have proven that the training of neural machine translation (NMT) can be facilitated by mimicking the learning process of humans.
Iterative Domain-Repaired Back-Translation
In this paper, we focus on the domain-specific translation with low resources, where in-domain parallel corpora are scarce or nonexistent.
Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translation
In this paper, we research extensively the pros and cons of the standard transformer in document-level translation, and find that the auto-regressive property can simultaneously bring both the advantage of the consistency and the disadvantage of error accumulation.
Bilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences
In this paper, we propose a new task of machine translation (MT), which is based on no parallel sentences but can refer to a ground-truth bilingual dictionary.
Visual Agreement Regularized Training for Multi-Modal Machine Translation
Further analysis demonstrates that the proposed regularized training can effectively improve the agreement of attention on the image, leading to better use of visual information.
Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation
However, existing transfer methods involving a common target language are far from success in the extreme scenario of zero-shot translation, due to the language space mismatch problem between transferor (the parent model) and transferee (the child model) on the source side.
Neural Zero-Inflated Quality Estimation Model For Automatic Speech Recognition System
The performances of automatic speech recognition (ASR) systems are usually evaluated by the metric word error rate (WER) when the manually transcribed data are provided, which are, however, expensively available in the real scenario.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Zero-Shot Cross-Lingual Abstractive Sentence Summarization through Teaching Generation and Attention
But there is no cross-lingual parallel corpus, whose source sentence language is different to the summary language, to directly train a cross-lingual ASSUM system.
Lattice Transformer for Speech Translation
Recent advances in sequence modeling have highlighted the strengths of the transformer architecture, especially in achieving state-of-the-art machine translation results.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Alibaba's Neural Machine Translation Systems for WMT18
We participated in 5 translation directions including English ↔ Russian, English ↔ Turkish in both directions and English → Chinese.
Alibaba Submission to the WMT18 Parallel Corpus Filtering Task
This paper describes the Alibaba Machine Translation Group submissions to the WMT 2018 Shared Task on Parallel Corpus Filtering.
Alibaba Submission for WMT18 Quality Estimation Task
The goal of WMT 2018 Shared Task on Translation Quality Estimation is to investigate automatic methods for estimating the quality of machine translation results without reference translations.
"Bilingual Expert" Can Find Translation Errors
Recent advances in statistical machine translation via the adoption of neural sequence-to-sequence models empower the end-to-end system to achieve state-of-the-art in many WMT benchmarks.
Cost Weighting for Neural Machine Translation Domain Adaptation
We compare cost weighting to two traditional domain adaptation techniques developed for statistical machine translation: data selection and sub-corpus weighting.
