Search Results for author:
Found 16 papers, 5 papers with code
Harmless Transfer Learning for Item Embeddings
On the definition side, we reduce the bias in transfer loss by focusing on the items to which information from high-frequency items can be efficiently transferred.
Movie Gen: A Cast of Media Foundation Models
Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation.
The Llama 3 Herd of Models
This paper presents a new set of foundation models, called Llama 3.
Ranked #3 on
Question Answering
on PeerQA
SpinQuant: LLM quantization with learned rotations
With 4-bit quantization of weight, activation, and KV-cache, SpinQuant narrows the accuracy gap on zero-shot reasoning tasks with full precision to merely 2. 9 points on the LLaMA-2 7B model, surpassing LLM-QAT by 19. 1 points and SmoothQuant by 25. 0 points.
Microscaling Data Formats for Deep Learning
Narrow bit-width data formats are key to reducing the computational and storage costs of modern deep learning applications.
FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Models
Sequence-based deep learning recommendation models (DLRMs) are an emerging class of DLRMs showing great improvements over their prior sum-pooling based counterparts at capturing users' long term interests.
RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure
RecD addresses immense storage, preprocessing, and training overheads caused by feature duplication inherent in industry-scale DLRM training datasets.
Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems
One of the key challenges of learning an online recommendation model is the temporal domain shift, which causes the mismatch between the training and testing data distribution and hence domain generalization error.
AutoShard: Automated Embedding Table Sharding for Recommender Systems
This is a significant design challenge of distributed systems named embedding table sharding, i. e., how we should partition the embedding tables to balance the costs across devices, which is a non-trivial task because 1) it is hard to efficiently and precisely measure the cost, and 2) the partition problem is known to be NP-hard.
Positive Unlabeled Contrastive Learning
We extend this paradigm to the classical positive unlabeled (PU) setting, where the task is to learn a binary classifier given only a few labeled positive samples, and (often) a large amount of unlabeled samples (which could be positive or negative).
Frequency-aware SGD for Efficient Embedding Learning with Provable Benefits
We show that incorporating frequency information of tokens in the embedding learning problems leads to provably efficient algorithms, and demonstrate that common adaptive algorithms implicitly exploit the frequency information to a large extent.
Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale
We share in this paper our search strategies to adapt reference recommendation models to low-precision hardware, our optimization of low-precision compute kernels, and the design and development of tool chain so as to maintain our models' accuracy throughout their lifespan during which topic trends and users' interests inevitably evolve.
Alternate Model Growth and Pruning for Efficient Training of Recommendation Systems
Our method leverages structured sparsification to reduce computational cost without hurting the model capacity at the end of offline training so that a full-size model is available in the recurring training stage to learn new data in real-time.
Training Recommender Systems at Scale: Communication-Efficient Model and Data Parallelism
This is done by identifying and updating only the most relevant neurons of the neural network for each training sample in the data.
Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data
To the best of our knowledge, this is the first work to provide in-depth analysis and discussion of applying pruning to online recommendation systems with non-stationary data distribution.
On the Runtime-Efficacy Trade-off of Anomaly Detection Techniques for Real-Time Streaming Data
Further, the lack of characterization of performance -- both with respect to real-timeliness and accuracy -- on production data sets makes model selection very challenging.
