Search Results for author:
Found 81 papers, 45 papers with code
Self-supervised Product Title Rewrite for Product Listing Ads
Product Listing Ads (PLAs) are primary online advertisements merchants pay to attract more customers.
RoBLEURT Submission for WMT2021 Metrics Task
After investigating the recent advances of trainable metrics, we conclude several aspects of vital importance to obtain a well-performed metric model by: 1) jointly leveraging the advantages of source-included model and reference-only model, 2) continuously pre-training the model with massive synthetic data pairs, and 3) fine-tuning the model with data denoising strategy.
Bridging the Gap between Training and Inference: Multi-Candidate Optimization for Diverse Neural Machine Translation
Diverse NMT aims at generating multiple diverse yet faithful translations given a source sentence.
GCPG: A General Framework for Controllable Paraphrase Generation
Under GCPG, we reconstruct commonly adopted lexical condition (i. e., Keywords) and syntactical conditions (i. e., Part-Of-Speech sequence, Constituent Tree, Masked Template and Sentential Exemplar) and study the combination of the two types.
Unsupervised Preference-Aware Language Identification
Recognizing the language of ambiguous texts has become a main challenge in language identification (LID).
CoRT: Code-integrated Reasoning within Thinking
Large Reasoning Models (LRMs) like o1 and DeepSeek-R1 have shown remarkable progress in natural language reasoning with long chain-of-thought (CoT), yet they remain inefficient or inaccurate when handling complex mathematical operations.
Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
In this work, we introduce the Qwen3 Embedding series, a significant advancement over its predecessor, the GTE-Qwen series, in text embedding and reranking capabilities, built upon the Qwen3 foundation models.
Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs
Directly training Large Language Models (LLMs) for Multi-Agent Systems (MAS) remains challenging due to intricate reward modeling, dynamic agent interactions, and demanding generalization requirements.
MTR-Bench: A Comprehensive Benchmark for Multi-Turn Reasoning Evaluation
To fill these gaps, we present MTR-Bench for LLMs' Multi-Turn Reasoning evaluation.
Parallel Scaling Law for Language Models
We apply $P$ diverse and learnable transformations to the input, execute forward passes of the model in parallel, and dynamically aggregate the $P$ outputs.
WorldPM: Scaling Human Preference Modeling
Motivated by scaling laws in language modeling that demonstrate how test loss scales as a power law with model and dataset sizes, we find that similar laws exist in preference modeling.
Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Gating mechanisms have been widely utilized, from early models like LSTMs and Highway Networks to recent state space models, linear attention, and also softmax attention.
START: Self-taught Reasoner with Tools
In this paper, we introduce START (Self-Taught Reasoner with Tools), a novel tool-integrated long CoT reasoning LLM that significantly enhances reasoning capabilities by leveraging external tools.
DataMan: Data Manager for Pre-training Large Language Models
In this paper, we train a Data Manager (DataMan) to learn quality ratings and domain recognition from pointwise rating, and use it to annotate a 447B token pre-training corpus with 14 quality ratings and domain type.
SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines
To address this gap, we present SuperGPQA, a comprehensive benchmark that evaluates graduate-level knowledge and reasoning capabilities across 285 disciplines.
HellaSwag-Pro: A Large-Scale Bilingual Benchmark for Evaluating the Robustness of LLMs in Commonsense Reasoning
To investigate this question, we present the first extensive robustness evaluation of LLMs in commonsense reasoning.
Qwen2.5-1M Technical Report
By leveraging our inference framework, the Qwen2. 5-1M models achieve a remarkable 3x to 7x prefill speedup in scenarios with 1 million tokens of context.
RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques
In this work, we introduce a new benchmark designed to assess the critique capabilities of LLMs.
Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models
Existing MoE training frameworks usually employ the parallel training strategy so that $f_i$ and the LBL are calculated within a $\textbf{micro-batch}$ and then averaged across parallel groups.
The Lessons of Developing Process Reward Models in Mathematical Reasoning
Furthermore, we identify potential biases in conventional Best-of-N (BoN) evaluation strategies for PRMs: (1) The unreliable policy models generate responses with correct answers but flawed processes, leading to a misalignment between the evaluation criteria of BoN and the PRM objectives of process verification.
Enabling Scalable Oversight via Self-Evolving Critic
Despite their remarkable performance, the development of Large Language Models (LLMs) faces a critical challenge in scalable oversight: providing effective feedback for tasks where human evaluation is difficult or where LLMs outperform humans.
CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratings
CodeElo benchmark is mainly based on the official CodeForces platform and tries to align with the platform as much as possible.
Qwen2.5 Technical Report
In addition, for hosted solutions, the proprietary models currently include two mixture-of-experts (MoE) variants: Qwen2. 5-Turbo and Qwen2. 5-Plus, both available from Alibaba Cloud Model Studio.
Ranked #7 on
on GPQA
ProcessBench: Identifying Process Errors in Mathematical Reasoning
We conduct extensive evaluation on ProcessBench, involving two types of models: process reward models (PRMs) and critic models, where for the latter we prompt general language models to critique each solution step by step.
Language Models can Self-Lengthen to Generate Long Texts
The framework consists of two roles: the Generator and the Extender.
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
We present the Qwen2-VL Series, an advanced upgrade of the previous Qwen-VL models that redefines the conventional predetermined-resolution approach in visual processing.
Ranked #3 on
Temporal Relation Extraction
on Vinoground
Natural Language Visual Grounding
Temporal Relation Extraction
+2
Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
This RM is then applied to the iterative evolution of data in supervised fine-tuning (SFT).
Ranked #2 on
Math Word Problem Solving
on MATH
(using extra training data)
Qwen2.5-Coder Technical Report
In this report, we introduce the Qwen2. 5-Coder series, a significant upgrade from its predecessor, CodeQwen1. 5.
Towards a Unified View of Preference Learning for Large Language Models: A Survey
Finally, based on our unified perspective, we explore the challenges and future research directions for aligning large language models with human preferences.
Qwen2 Technical Report
This report introduces the Qwen2 series, the latest addition to our large language models and large multimodal models.
Ranked #3 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models
Large Language Models (LLMs) have gained widespread adoption in various natural language processing tasks, including question answering and dialogue systems.
DotaMath: Decomposition of Thought with Code Assistance and Self-correction for Mathematical Reasoning
In this paper, we introduce a series of LLMs that employs the Decomposition of thought with code assistance and self-correction for mathematical reasoning, dubbed as DotaMath.
MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization
Prompt engineering, as an efficient and effective way to leverage Large Language Models (LLM), has drawn a lot of attention from the research community.
LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback
In recent progress, mathematical verifiers have achieved success in mathematical reasoning tasks by validating the correctness of solutions generated by policy models.
An Empirical Study of Parameter Efficient Fine-tuning on Vision-Language Pre-train Model
A natural expectation for PEFTs is that the performance of various PEFTs is positively related to the data size and fine-tunable parameter size.
Noisy Pair Corrector for Dense Retrieval
Most dense retrieval models contain an implicit assumption: the training query-document pairs are exactly matched.
OccuQuest: Mitigating Occupational Bias for Inclusive Large Language Models
Furthermore, we assemble three test sets for comprehensive evaluation, an occu-test set covering 25 occupational categories, an estate set focusing on real estate, and an occu-quora set containing real-world questions from Quora.
How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
We propose four intriguing research questions to explore the association between model performance and various factors including data amount, composition ratio, model size and SFT strategies.
Qwen Technical Report
Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans.
Ranked #3 on
Multi-Label Text Classification
on CC3M-TagMask
PolyLM: An Open Source Polyglot Large Language Model
Large language models (LLMs) demonstrate remarkable ability to comprehend, reason, and generate following nature language instructions.
Bridging the Domain Gaps in Context Representations for k-Nearest Neighbor Neural Machine Translation
$k$-Nearest neighbor machine translation ($k$NN-MT) has attracted increasing attention due to its ability to non-parametrically adapt to new translation domains.
Interactive Natural Language Processing
Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP, aimed at addressing limitations in existing frameworks while aligning with the ultimate goals of artificial intelligence.
Towards Fine-Grained Information: Identifying the Type and Location of Translation Errors
In this paper, we propose Fine-Grained Translation Error Detection (FG-TED) task, aiming at identifying both the position and the type of translation errors on given source-hypothesis sentence pairs.
Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?
The lack of competency awareness makes NMT untrustworthy.
Alibaba-Translate China's Submission for WMT 2022 Metrics Shared Task
In this report, we present our submission to the WMT 2022 Metrics Shared Task.
Alibaba-Translate China's Submission for WMT 2022 Quality Estimation Shared Task
In this paper, we present our submission to the sentence-level MQM benchmark at Quality Estimation Shared Task, named UniTE (Unified Translation Evaluation).
Draft, Command, and Edit: Controllable Text Editing in E-Commerce
To address this challenge, we explore a new draft-command-edit manner in description generation, leading to the proposed new task-controllable text editing in E-commerce.
Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis
In this paper, we propose the CORE (Counterfactual Analysis based Relation Extraction) debiasing method that guides the RE models to focus on the main effects of textual context without losing the entity information.
Dangling-Aware Entity Alignment with Mixed High-Order Proximities
We study dangling-aware entity alignment in knowledge graphs (KGs), which is an underexplored but important problem.
UniTE: Unified Translation Evaluation
Translation quality evaluation plays a crucial role in machine translation.
RoBLEURT Submission for the WMT2021 Metrics Task
After investigating the recent advances of trainable metrics, we conclude several aspects of vital importance to obtain a well-performed metric model by: 1) jointly leveraging the advantages of source-included model and reference-only model, 2) continuously pre-training the model with massive synthetic data pairs, and 3) fine-tuning the model with data denoising strategy.
Attention Mechanism with Energy-Friendly Operations
Attention mechanism has become the dominant module in natural language processing models.
Tailor: A Prompt-Based Approach to Attribute-Based Controlled Text Generation
We experimentally find that these prompts can be simply concatenated as a whole to multi-attribute CTG without any re-training, yet raises problems of fluency decrease and position sensitivity.
RMBR: A Regularized Minimum Bayes Risk Reranking Framework for Machine Translation
Recently, Minimum Bayes Risk (MBR) decoding has been proposed to improve the quality for NMT, which seeks for a consensus translation that is closest on average to other candidates from the n-best list.
Frequency-Aware Contrastive Learning for Neural Machine Translation
Inspired by the observation that low-frequency words form a more compact embedding space, we tackle this challenge from a representation learning perspective.
KGR^4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation
Generative commonsense reasoning requires machines to generate sentences describing an everyday scenario given several concepts, which has attracted much attention recently.
Leveraging Advantages of Interactive and Non-Interactive Models for Vector-Based Cross-Lingual Information Retrieval
Interactive and non-interactive model are the two de-facto standard frameworks in vector-based cross-lingual information retrieval (V-CLIR), which embed queries and documents in synchronous and asynchronous fashions, respectively.
Computational Efficiency
Cross-Lingual Information Retrieval
+5
POS-Constrained Parallel Decoding for Non-autoregressive Generation
However, in this work, we experimentally reveal that this assumption does not always hold for the text generation tasks like text summarization and story ending generation.
Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation
A well-known limitation in pretrain-finetune paradigm lies in its inflexibility caused by the one-size-fits-all vocabulary.
Towards User-Driven Neural Machine Translation
Furthermore, we contribute the first Chinese-English parallel corpus annotated with user behavior called UDT-Corpus.
Prediction, Selection, and Generation: Exploration of Knowledge-Driven Conversation System
The system includes modules such as dialogue topic prediction, knowledge matching and dialogue generation.
Mask Attention Networks: Rethinking and Strengthen Transformer
We therefore introduce a new layer named dynamic mask attention network (DMAN) with a learnable mask matrix which is able to model localness adaptively.
Ranked #11 on
Machine Translation
on WMT2014 English-German
BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
In this paper, we propose BANG, a new pretraining model to Bridge the gap between Autoregressive (AR) and Non-autoregressive (NAR) Generation.
GLGE: A New General Language Generation Evaluation Benchmark
Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP).
ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training
This paper presents a new sequence-to-sequence pre-training model called ProphetNet, which introduces a novel self-supervised objective named future n-gram prediction and the proposed n-stream self-attention mechanism.
Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
In this paper, we propose a novel data augmentation method, referred to as Controllable Rewriting based Question Data Augmentation (CRQDA), for machine reading comprehension (MRC), question generation, and question-answering natural language inference tasks.
AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation
Previous studies usually use the supervised models which deeply rely on parallel data.
Cultural Vocal Bursts Intensity Prediction
Language Modeling
+2
RikiNet: Reading Wikipedia Pages for Natural Question Answering
The representations are then fed into the predictor to obtain the span of the short answer, the paragraph of the long answer, and the answer type in a cascaded manner.
Let's be Humorous: Knowledge Enhanced Humor Generation
To our knowledge, this is the first attempt to generate punchlines with knowledge enhanced model.
Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
Furthermore, we propose a simple and effective method to mine the keyphrases of interest in the news article and build a first large-scale keyphrase-aware news headline corpus, which contains over 180K aligned triples of $<$news article, headline, keyphrase$>$.
Generating Chinese Poetry from Images via Concrete and Abstract Information
We proposed an infilling-based Chinese poetry generation model which can infill the Concrete keywords into each line of poems in an explicit way, and an abstract information embedding to integrate the Abstract information into generated poems.
ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
This paper presents a new sequence-to-sequence pre-training model called ProphetNet, which introduces a novel self-supervised objective named future n-gram prediction and the proposed n-stream self-attention mechanism.
Ranked #6 on
Question Generation
on SQuAD1.1
(using extra training data)
Deep Poetry: A Chinese Classical Poetry Generation System
For the user's convenience, we deploy the system at the WeChat applet platform, users can use the system on the mobile device whenever and wherever possible.
Deep Learning-Based Automatic Downbeat Tracking: A Brief Review
Thereinto, downbeat tracking has been a fundamental and continuous problem in Music Information Retrieval (MIR) area.
Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning
We propose a new framework that utilizes the gradients to revise the sentence in a continuous space during inference to achieve text style transfer.
TIGS: An Inference Algorithm for Text Infilling with Gradient Search
Text infilling is defined as a task for filling in the missing part of a sentence or paragraph, which is suitable for many real-world natural language generation scenarios.
mu-Forcing: Training Variational Recurrent Autoencoders for Text Generation
It has been previously observed that training Variational Recurrent Autoencoders (VRAE) for text generation suffers from serious uninformative latent variables problem.
Ancient-Modern Chinese Translation with a Large Training Dataset
Ancient Chinese brings the wisdom and spirit culture of the Chinese nation.
Cultural Vocal Bursts Intensity Prediction
Machine Translation
+2
BFGAN: Backward and Forward Generative Adversarial Networks for Lexically Constrained Sentence Generation
Incorporating prior knowledge like lexical constraints into the model's output to generate meaningful and coherent sentences has many applications in dialogue system, machine translation, image captioning, etc.
