Search Results for author:
Found 9 papers, 0 papers with code
Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs
Ultimately, our experimental findings demonstrate that a 300B MoE LLM can be effectively trained on lower-performance devices while achieving comparable performance to models of a similar scale, including dense and MoE models.
Toward Stable and Consistent Evaluation Results: A New Methodology for Base Model Evaluation
To address these issues, we propose Base model Oriented Systematic Evaluation (BOSE), a method specifically designed to optimize the evaluation of base models.
AntDT: A Self-Adaptive Distributed Training Framework for Leader and Straggler Nodes
Previous works only address part of the stragglers and could not adaptively solve various stragglers in practice.
Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors
However, we argue that a critical obstacle remains in deploying LLMs for practical use: the efficiency of LLMs when processing long textual user behaviors.
G-Meta: Distributed Meta Learning in GPU Clusters for Large-Scale Recommender Systems
However, the existing systems are not tailored for meta learning based DLRM models and have critical problems regarding efficiency in distributed training in the GPU cluster.
One Model for All: Large Language Models are Domain-Agnostic Recommendation Systems
We expect that by mixing the user's behaviors across different domains, we can exploit the common knowledge encoded in the pre-trained language model to alleviate the problems of data sparsity and cold start problems.
Rethinking Memory and Communication Cost for Efficient Large Language Model Training
In this paper, we rethink the impact of memory consumption and communication costs on the training speed of large language models, and propose a memory-communication balanced strategy set Partial Redundancy Optimizer (PaRO).
AntM$^{2}$C: A Large Scale Dataset For Multi-Scenario Multi-Modal CTR Prediction
3) AntM$^{2}$C provides 1 billion CTR data with 200 features, including 200 million users and 6 million items.
Data-Free Adversarial Perturbations for Practical Black-Box Attack
Adversarial examples often exhibit black-box attacking transferability, which allows that adversarial examples crafted for one model can fool another model.
