Search Results for author:
Found 21 papers, 14 papers with code
Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model
This technical report highlights the key design decisions that enhance the performance of the medium-sized diffusion model.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1.
Ranked #1 on
Mathematical Reasoning
on AIME24
DeepSeek-V3 Technical Report
We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token.
FlexSP: Accelerating Large Language Model Training via Flexible Sequence Parallelism
Furthermore, we implement our method in a high-performance system that supports adaptive parallelization in distributed LLM training.
Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning
For DL training, we deployed the Fire-Flyer 2 with 10, 000 PCIe A100 GPUs, achieved performance approximating the DGX-A100 while reducing costs by half and energy consumption by 40%.
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
Through this continued pre-training, DeepSeek-Coder-V2 substantially enhances the coding and mathematical reasoning capabilities of DeepSeek-V2, while maintaining comparable performance in general language tasks.
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation.
UniFL: Improve Latent Diffusion Model via Unified Feedback Learning
Latent diffusion models (LDM) have revolutionized text-to-image generation, leading to the proliferation of various advanced models and diverse downstream applications.
ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models
Especially, after learning a deep understanding of pure resolution priors, ResAdapter trained on the general dataset, generates resolution-free images with personalized diffusion models while preserving their original style domain.
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Subsequently, we scale up DeepSeekMoE to 16B parameters and show that it achieves comparable performance with LLaMA2 7B, with only about 40% of computations.
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
The rapid development of open-source large language models (LLMs) has been truly remarkable.
SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity
Our framework consists of two parts: (a) A fine-grained kernel sparsity schema with a sparsity granularity between structured pruning and unstructured pruning.
Control-A-Video: Controllable Text-to-Video Diffusion Models with Motion Prior and Reward Feedback Learning
Recent advances in text-to-image (T2I) diffusion models have enabled impressive image generation capabilities guided by text prompts.
Common Diffusion Noise Schedules and Sample Steps are Flawed
We discover that common diffusion noise schedules do not enforce the last timestep to have zero signal-to-noise ratio (SNR), and some implementations of diffusion samplers do not start from the last timestep.
Multi-Objective Evolutionary for Object Detection Mobile Architectures Search
Our improved backbone network can reduce the computational effort while improving the accuracy of the object detection network.
Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios
Then, Next Hybrid Strategy (NHS) is designed to stack NCB and NTB in an efficient hybrid paradigm, which boosts performance in various downstream tasks.
Ranked #253 on
Image Classification
on ImageNet
(GFLOPs metric)
MoCoViT: Mobile Convolutional Vision Transformer
Recently, Transformer networks have achieved impressive results on a variety of vision tasks.
TRT-ViT: TensorRT-oriented Vision Transformer
We revisit the existing excellent Transformers from the perspective of practical application.
SepViT: Separable Vision Transformer
Vision Transformers have witnessed prevailing success in a series of vision tasks.
Fast and Accurate Quantized Camera Scene Detection on Smartphones, Mobile AI 2021 Challenge: Report
To address this problem, we introduce the first Mobile AI challenge, where the target is to develop quantized deep learning-based camera scene classification solutions that can demonstrate a real-time performance on smartphones and IoT platforms.
OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks
Many channel pruning works utilize structured sparsity regularization to zero out all the weights in some channels and automatically obtain structure-sparse network in training stage.
