Search Results for author:
Found 37 papers, 23 papers with code
S^4-Tuning: A Simple Cross-lingual Sub-network Tuning Method
The emergence of multilingual pre-trained language models makes it possible to adapt to target languages with only few labeled examples. However, vanilla fine-tuning tends to achieve degenerated and unstable results, owing to the Language Interference among different languages, and Parameter Overload under the few-sample transfer learning scenarios. To address two problems elegantly, we propose S^4-Tuning, a Simple Cross-lingual Sub-network Tuning method.
Rethinking Denoised Auto-Encoding in Language Pre-Training
Pre-trained self-supervised models such as BERT have achieved striking success in learning sequence representations, especially for natural language processing.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1.
Ranked #1 on
Mathematical Reasoning
on AIME24
DeepSeek-V3 Technical Report
We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token.
DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search
We introduce DeepSeek-Prover-V1. 5, an open-source language model designed for theorem proving in Lean 4, which enhances DeepSeek-Prover-V1 by optimizing both training and inference processes.
Ranked #3 on
Automated Theorem Proving
on miniF2F-test
(using extra training data)
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
Through this continued pre-training, DeepSeek-Coder-V2 substantially enhances the coding and mathematical reasoning capabilities of DeepSeek-V2, while maintaining comparable performance in general language tasks.
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation.
DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
The rapid development of large language models has revolutionized code intelligence in software development.
Ranked #5 on
Code Generation
on APPS
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Subsequently, we scale up DeepSeekMoE to 16B parameters and show that it achieves comparable performance with LLaMA2 7B, with only about 40% of computations.
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
The rapid development of open-source large language models (LLMs) has been truly remarkable.
Parameter-Efficient Sparsity for Large Language Models Fine-Tuning
With the dramatically increased number of parameters in language models, sparsity methods have received ever-increasing research focus to compress and accelerate the models.
Towards Unified Prompt Tuning for Few-shot Text Classification
Prompt-based fine-tuning has boosted the performance of Pre-trained Language Models (PLMs) on few-shot text classification by employing task-specific prompts.
Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base
Pre-trained language models (PLMs) like BERT have made significant progress in various downstream NLP tasks.
Probing Structured Pruning on Multilingual Pre-trained Models: Settings, Algorithms, and Efficiency
Structured pruning has been extensively studied on monolingual pre-trained language models and is yet to be fully evaluated on their multilingual counterparts.
Making Pre-trained Language Models End-to-end Few-shot Learners with Contrastive Prompt Tuning
Pre-trained Language Models (PLMs) have achieved remarkable performance for various language understanding tasks in IR systems, which require the fine-tuning process based on labeled training data.
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression
Unified in contrastive learning, CAP enables the pruned model to learn from the pre-trained model for task-agnostic knowledge, and fine-tuned model for task-specific knowledge.
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning
Recent pretrained language models extend from millions to billions of parameters.
SemVLP: Vision-Language Pre-training by Aligning Semantics at Multiple Levels
Vision-language pre-training (VLP) on large-scale image-text pairs has recently witnessed rapid progress for learning cross-modal representations.
VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation
Existing work in multilingual pretraining has demonstrated the potential of cross-lingual transferability by training a unified Transformer encoder for multiple languages.
CAPT: Contrastive Pre-Training for Learning Denoised Sequence Representations
Pre-trained self-supervised models such as BERT have achieved striking success in learning sequence representations, especially for natural language processing.
VECO: Variable Encoder-decoder Pre-training for Cross-lingual Understanding and Generation
Recent studies about learning multilingual representations have achieved significant performance gains across a wide range of downstream cross-lingual tasks.
Inductively Representing Out-of-Knowledge-Graph Entities by Optimal Estimation Under Translational Assumptions
Conventional Knowledge Graph Completion (KGC) assumes that all test entities appear during training.
Pun-GAN: Generative Adversarial Network for Pun Generation
It consists of a generator to produce pun sentences, and a discriminator to distinguish between the generated pun sentences and the real sentences with specific word senses.
Learning to Control the Fine-grained Sentiment for Story Ending Generation
Therefore, we propose a generic and novel framework which consists of a sentiment analyzer and a sentimental generator, respectively addressing the two challenges.
Towards Fine-grained Text Sentiment Transfer
In this paper, we focus on the task of fine-grained text sentiment transfer (FGST).
MAAM: A Morphology-Aware Alignment Model for Unsupervised Bilingual Lexicon Induction
The task of unsupervised bilingual lexicon induction (UBLI) aims to induce word translations from monolingual corpora in two languages.
A Deep Reinforced Sequence-to-Set Model for Multi-Label Classification
In this way, we can reduce the dependence of the model on the label order, as well as capture high-order correlations between labels.
Enhancing Topic-to-Essay Generation with External Commonsense Knowledge
Experiments show that with external commonsense knowledge and adversarial training, the generated essays are more novel, diverse, and topic-consistent than existing methods in terms of both automatic and human evaluation.
Cross-Modal Commentator: Automatic Machine Commenting Based on Cross-Modal Information
Automatic commenting of online articles can provide additional opinions and facts to the reader, which improves user experience and engagement on social media platforms.
Towards Comprehensive Description Generation from Factual Attribute-value Tables
To relieve these problems, we first propose force attention (FA) method to encourage the generator to pay more attention to the uncovered attributes to avoid potential key attributes missing.
A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer
Unsupervised text style transfer aims to alter text styles while preserving the content, without aligned data for supervision.
A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer
Therefore, in this paper, we propose a dual reinforcement learning framework to directly transfer the style of the text via a one-step mapping model, without any separation of content and style.
Ranked #1 on
Unsupervised Text Style Transfer
on GYAFC
Knowledgeable Storyteller: A Commonsense-Driven Generative Model for Visual Storytelling
The visual storytelling (VST) task aims at generating a reasonable and coherent paragraph-level story with the image stream as input.
Ranked #21 on
Visual Storytelling
on VIST
Learning Unsupervised Word Mapping by Maximizing Mean Discrepancy
In order to avoid such sophisticated alternate optimization, we propose to learn unsupervised word mapping by directly maximizing the mean discrepancy between the distribution of transferred embedding and target embedding.
Leveraging Gloss Knowledge in Neural Word Sense Disambiguation by Hierarchical Co-Attention
The goal of Word Sense Disambiguation (WSD) is to identify the correct meaning of a word in the particular context.
Incorporating Glosses into Neural Word Sense Disambiguation
GAS models the semantic relationship between the context and the gloss in an improved memory network framework, which breaks the barriers of the previous supervised methods and knowledge-based methods.
Ranked #3 on
Word Sense Disambiguation
on SemEval 2015 Task 13
