Search Results for author:
Found 24 papers, 12 papers with code
The Llama 3 Herd of Models
This paper presents a new set of foundation models, called Llama 3.
Ranked #3 on
Question Answering
on PeerQA
Wukong: Towards a Scaling Law for Large-Scale Recommendation
Scaling laws play an instrumental role in the sustainable improvement in model quality.
Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendation
We study a mismatch between the deep learning recommendation models' flat architecture, common distributed training paradigm and hierarchical data center topology.
Microscaling Data Formats for Deep Learning
Narrow bit-width data formats are key to reducing the computational and storage costs of modern deep learning applications.
With Shared Microexponents, A Little Shifting Goes a Long Way
This paper introduces Block Data Representations (BDR), a framework for exploring and evaluating a wide spectrum of narrow-precision formats for deep learning.
Learning to Collide: Recommendation System Model Compression with Learned Hash Functions
However, this approach introduces collisions between semantically dissimilar ids that degrade model quality.
Differentiable NAS Framework and Application to Ads CTR Prediction
Neural architecture search (NAS) methods aim to automatically find the optimal deep neural network (DNN) architecture as measured by a given objective function, typically some combination of task accuracy and inference efficiency.
Supporting Massive DLRM Inference Through Software Defined Memory
Deep Learning Recommendation Models (DLRM) are widespread, account for a considerable data center footprint, and grow by more than 1. 5x per year.
Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale
We share in this paper our search strategies to adapt reference recommendation models to low-precision hardware, our optimization of low-precision compute kernels, and the design and development of tool chain so as to maintain our models' accuracy throughout their lifespan during which topic trends and users' interests inevitably evolve.
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers.
Time-based Sequence Model for Personalization and Recommendation Systems
In this paper we develop a novel recommendation model that explicitly incorporates time information.
Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems
Large-scale training is important to ensure high performance and accuracy of machine-learning models.
Distributed, Parallel, and Cluster Computing 68T05, 68M10 H.3.3; I.2.6; C.2.1
Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems
Embedding representations power machine intelligence in many applications, including recommendation systems, but they are space intensive -- potentially occupying hundreds of gigabytes in large-scale settings.
Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
We propose a novel approach for reducing the embedding size in an end-to-end fashion by exploiting complementary partitions of the category set to produce a unique embedding vector for each category without explicit definition.
The Architectural Implications of Facebook's DNN-based Personalized Recommendation
The widespread application of deep learning has changed the landscape of computation in the data center.
Deep Learning Recommendation Model for Personalization and Recommendation Systems
With the advent of deep learning, neural network-based recommendation models have emerged as an important tool for tackling personalization and recommendation tasks.
Spatial-Winograd Pruning Enabling Sparse Winograd Convolution
To achieve a high Winograd-domain weight sparsity without changing network structures, we propose a new pruning method, spatial-Winograd pruning.
On the Dimensionality of Embeddings for Sparse Features and Data
In this note we discuss a common misconception, namely that embeddings are always used to reduce the dimensionality of the item space.
On Periodic Functions as Regularizers for Quantization of Neural Networks
We apply these functions component-wise and add the sum over the model parameters as a regularizer to the model loss during training.
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
The application of deep learning techniques resulted in remarkable improvement of machine learning models.
Bandana: Using Non-volatile Memory for Storing Deep Learning Models
Typical large-scale recommender systems use deep learning models that are stored on a large amount of DRAM.
Parallel Complexity of Forward and Backward Propagation
We show that the forward and backward propagation can be formulated as a solution of lower and upper triangular systems of equations.
AdaBatch: Adaptive Batch Sizes for Training Deep Neural Networks
Training deep neural networks with Stochastic Gradient Descent, or its variants, requires careful choice of both learning rate and batch size.
Feedforward and Recurrent Neural Networks Backward Propagation and Hessian in Matrix Form
In this paper we focus on the linear algebra theory behind feedforward (FNN) and recurrent (RNN) neural networks.
