Search Results for author:
Found 33 papers, 23 papers with code
Focus on the Target’s Vocabulary: Masked Label Smoothing for Machine Translation
Label smoothing and vocabulary sharing are two widely used techniques in neural machine translation models.
S^4-Tuning: A Simple Cross-lingual Sub-network Tuning Method
The emergence of multilingual pre-trained language models makes it possible to adapt to target languages with only few labeled examples. However, vanilla fine-tuning tends to achieve degenerated and unstable results, owing to the Language Interference among different languages, and Parameter Overload under the few-sample transfer learning scenarios. To address two problems elegantly, we propose S^4-Tuning, a Simple Cross-lingual Sub-network Tuning method.
Inference-Time Scaling for Generalist Reward Modeling
Recently, the incentivization of reasoning capabilities in LLMs from RL indicates that $\textit{proper learning methods could enable effective inference-time scalability}$.
CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction
Reasoning is a fundamental capability of Large Language Models.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1.
Ranked #2 on
Mathematical Reasoning
on AIME24
DeepSeek-V3 Technical Report
We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token.
Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
However, existing benchmarks like GSM8K or MATH are now being solved with high accuracy (e. g., OpenAI o1 achieves 94. 8\% on MATH dataset), indicating their inadequacy for truly challenging these models.
Towards a Unified View of Preference Learning for Large Language Models: A Survey
Finally, based on our unified perspective, we explore the challenges and future research directions for aligning large language models with human preferences.
Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models
In this work, we study the PEFT method for LLMs with the Mixture-of-Experts (MoE) architecture and the contents of this work are mainly threefold: (1) We investigate the dispersion degree of the activated experts in customized tasks, and found that the routing distribution for a specific task tends to be highly concentrated, while the distribution of activated experts varies significantly across different tasks.
LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback
In recent progress, mathematical verifiers have achieved success in mathematical reasoning tasks by validating the correctness of solutions generated by policy models.
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
Through this continued pre-training, DeepSeek-Coder-V2 substantially enhances the coding and mathematical reasoning capabilities of DeepSeek-V2, while maintaining comparable performance in general language tasks.
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation.
Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models
To fill this gap, we introduce Multimodal ArXiv, consisting of ArXivCap and ArXivQA, for enhancing LVLMs scientific comprehension.
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Mathematical reasoning poses a significant challenge for language models due to its complex and structured nature.
Ranked #28 on
Math Word Problem Solving
on MATH
(using extra training data)
Bi-Drop: Enhancing Fine-tuning Generalization via Synchronous sub-net Estimation and Optimization
Also, Bi-Drop needs only one mini-batch to estimate the sub-net so it achieves higher utility of training data.
A Double-Graph Based Framework for Frame Semantic Parsing
In this paper, we propose a Knowledge-guided Incremental semantic parser with Double-graph (KID).
A Two-Stream AMR-enhanced Model for Document-level Event Argument Extraction
In this paper, we focus on extracting event arguments from an entire document, which mainly faces two critical problems: a) the long-distance dependency between trigger and arguments over sentences; b) the distracting context towards an event in the document.
Abstract Meaning Representation
Document-level Event Extraction
+3
ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs
As Abstract Meaning Representation (AMR) implicitly involves compound semantic annotations, we hypothesize auxiliary tasks which are semantically or formally related can better enhance AMR parsing.
Ranked #7 on
AMR Parsing
on LDC2020T02
(using extra training data)
Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base
Pre-trained language models (PLMs) like BERT have made significant progress in various downstream NLP tasks.
Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency
Structured pruning has been extensively studied on monolingual pre-trained language models and is yet to be fully evaluated on their multilingual counterparts.
Making Pre-trained Language Models End-to-end Few-shot Learners with Contrastive Prompt Tuning
Pre-trained Language Models (PLMs) have achieved remarkable performance for various language understanding tasks in IR systems, which require the fine-tuning process based on labeled training data.
Focus on the Target's Vocabulary: Masked Label Smoothing for Machine Translation
Label smoothing and vocabulary sharing are two widely used techniques in neural machine translation models.
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression
Unified in contrastive learning, CAP enables the pruned model to learn from the pre-trained model for task-agnostic knowledge, and fine-tuned model for task-specific knowledge.
An Enhanced Span-based Decomposition Method for Few-Shot Sequence Labeling
Few-Shot Sequence Labeling (FSSL) is a canonical paradigm for the tagging models, e. g., named entity recognition and slot filling, to generalize on an emerging, resource-scarce domain.
Ranked #8 on
Few-shot NER
on Few-NERD (INTER)
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning
Recent pretrained language models extend from millions to billions of parameters.
Behind the Scenes: An Exploration of Trigger Biases Problem in Few-Shot Event Classification
However, we find they suffer from trigger biases that signify the statistical homogeneity between some trigger words and target event types, which we summarize as trigger overlapping and trigger separability.
Explicit Interaction Network for Aspect Sentiment Triplet Extraction
Table encoder extracts sentiment at token-pair level, so that the compositional feature between targets and opinions can be easily captured.
Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker
Existing methods are not effective due to two challenges of this task: a) the target event arguments are scattered across sentences; b) the correlation among events in a document is non-trivial to model.
Ranked #2 on
Document-level Event Extraction
on ChFinAnn
Volctrans Parallel Corpus Filtering System for WMT 2020
In this paper, we describe our submissions to the WMT20 shared task on parallel corpus filtering and alignment for low-resource conditions.
Double Graph Based Reasoning for Document-level Relation Extraction
Document-level relation extraction aims to extract relations among entities within a document.
Ranked #12 on
Relation Extraction
on DocRED
Xiaomingbot: A Multilingual Robot News Reporter
This paper proposes the building of Xiaomingbot, an intelligent, multilingual and multimodal software robot equipped with four integral capabilities: news generation, news translation, news reading and avatar animation.
Adaptive Gradient Methods Can Be Provably Faster than SGD after Finite Epochs
Adaptive gradient methods have attracted much attention of machine learning communities due to the high efficiency.
ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization
Due to its simplicity and outstanding ability to generalize, stochastic gradient descent (SGD) is still the most widely used optimization method despite its slow convergence.
