Search Results for author: Adam Paszke

Found 9 papers, 6 papers with code

Automap: Towards Ergonomic Automated Parallelism for ML Models

no code implementations6 Dec 2021 Michael Schaarschmidt, Dominik Grewe, Dimitrios Vytiniotis, Adam Paszke, Georg Stefan Schmid, Tamara Norman, James Molloy, Jonathan Godwin, Norman Alexander Rink, Vinod Nair, Dan Belov

The rapid rise in demand for training large neural network architectures has brought into focus the need for partitioning strategies, for example by using data, model, or pipeline parallelism.

Memory-efficient array redistribution through portable collective communication

no code implementations2 Dec 2021 Norman A. Rink, Adam Paszke, Dimitrios Vytiniotis, Georg Stefan Schmid

In this paper we address the problem of redistributing multi-dimensional array data in SPMD computations, the most prevalent form of parallelism in deep learning.

Decomposing reverse-mode automatic differentiation

no code implementations20 May 2021 Roy Frostig, Matthew J. Johnson, Dougal Maclaurin, Adam Paszke, Alexey Radul

We decompose reverse-mode automatic differentiation into (forward-mode) linearization followed by transposition.

Tensors Fitting Perfectly

1 code implementation26 Feb 2021 Adam Paszke, Brennan Saeta

Multidimensional arrays (NDArrays) are a central abstraction in modern scientific computing environments.

PyTorch Distributed: Experiences on Accelerating Data Parallel Training

3 code implementations28 Jun 2020 Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, Soumith Chintala

This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module.

Automatic Differentiation in PyTorch

1 code implementation NIPS 2017 2017 Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, Adam Lerer

In this article, we describe an automatic differentiation module of PyTorch — a library designed to enable rapid research on machine learning models.

Clustering Dimensionality Reduction +1

An Analysis of Deep Neural Network Models for Practical Applications

4 code implementations24 May 2016 Alfredo Canziani, Adam Paszke, Eugenio Culurciello

Since the emergence of Deep Neural Networks (DNNs) as a prominent technique in the field of computer vision, the ImageNet classification challenge has played a major role in advancing the state-of-the-art.

Cannot find the paper you are looking for? You can Submit a new open access paper.