Search Results for author: Nir Shavit

Found 21 papers, 2 papers with code

Inducing and Exploiting Activation Sparsity for Fast Inference on Deep Neural Networks

no code implementations ICML 2020 Mark Kurtz, Justin Kopinsky, Rati Gelashvili, Alexander Matveev, John Carr, Michael Goin, William Leiserson, Sage Moore, Nir Shavit, Dan Alistarh

In this paper, we present an in-depth analysis of methods for maximizing the sparsity of the activations in a trained neural network, and show that, when coupled with an efficient sparse-input convolution algorithm, we can leverage this sparsity for significant performance gains.

Image Classification

Sparse Expansion and Neuronal Disentanglement

no code implementations24 May 2024 Shashata Sawmya, Linghao Kong, Ilia Markov, Dan Alistarh, Nir Shavit

We show how to improve the inference efficiency of an LLM by expanding it into a mixture of sparse experts, where each expert is a copy of the original weights, one-shot pruned for a specific cluster of input values.

Disentanglement

Cliff-Learning

no code implementations14 Feb 2023 Tony T. Wang, Igor Zablotchi, Nir Shavit, Jonathan S. Rosenfeld

We conduct an in-depth investigation of foundation-model cliff-learning and study toy models of the phenomenon.

Transfer Learning

Revisiting Latent-Space Interpolation via a Quantitative Evaluation Framework

1 code implementation13 Oct 2021 Lu Mi, Tianxing He, Core Francisco Park, Hao Wang, Yue Wang, Nir Shavit

In this work, we show how data labeled with semantically continuous attributes can be utilized to conduct a quantitative evaluation of latent-space interpolation algorithms, for variational autoencoders.

Connectome-constrained Latent Variable Model of Whole-Brain Neural Activity

no code implementations ICLR 2022 Lu Mi, Richard Xu, Sridhama Prakhya, Albert Lin, Nir Shavit, Aravinthan Samuel, Srinivas C Turaga

Brain-wide measurements of activity and anatomical connectivity of the $\textit{C. elegans}$ nervous system in principle allow for the development of detailed mechanistic computational models.

Learning Guided Electron Microscopy with Active Acquisition

1 code implementation7 Jan 2021 Lu Mi, Hao Wang, Yaron Meirovitch, Richard Schalek, Srinivas C. Turaga, Jeff W. Lichtman, Aravinthan D. T. Samuel, Nir Shavit

Single-beam scanning electron microscopes (SEM) are widely used to acquire massive data sets for biomedical study, material analysis, and fabrication inspection.

On the Predictability of Pruning Across Scales

no code implementations18 Jun 2020 Jonathan S. Rosenfeld, Jonathan Frankle, Michael Carbin, Nir Shavit

We show that the error of iteratively magnitude-pruned networks empirically follows a scaling law with interpretable coefficients that depend on the architecture and task.

L3 Fusion: Fast Transformed Convolutions on CPUs

no code implementations4 Dec 2019 Rati Gelashvili, Nir Shavit, Aleksandar Zlateski

Fast convolutions via transforms, either Winograd or FFT, had emerged as a preferred way of performing the computation of convolutional layers, as it greatly reduces the number of required operations.

Benchmarking

Training-Free Uncertainty Estimation for Dense Regression: Sensitivity as a Surrogate

no code implementations28 Sep 2019 Lu Mi, Hao Wang, Yonglong Tian, Hao He, Nir Shavit

Uncertainty estimation is an essential step in the evaluation of the robustness for deep learning models in computer vision, especially when applied in risk-sensitive areas.

regression

A Constructive Prediction of the Generalization Error Across Scales

no code implementations ICLR 2020 Jonathan S. Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, Nir Shavit

In this work, we present a functional form which approximates well the generalization error in practice.

Cross-Classification Clustering: An Efficient Multi-Object Tracking Technique for 3-D Instance Segmentation in Connectomics

no code implementations CVPR 2019 Yaron Meirovitch, Lu Mi, Hayk Saribekyan, Alexander Matveev, David Rolnick, Nir Shavit

Pixel-accurate tracking of objects is a key element in many computer vision applications, often solved by iterated individual object tracking or instance segmentation followed by object matching.

Clustering General Classification +4

Deep Learning is Robust to Massive Label Noise

no code implementations ICLR 2018 David Rolnick, Andreas Veit, Serge Belongie, Nir Shavit

Deep neural networks trained on large supervised datasets have led to impressive results in image classification and other tasks.

Image Classification

Generative Compression

no code implementations4 Mar 2017 Shibani Santurkar, David Budden, Nir Shavit

Traditional image and video compression algorithms rely on hand-crafted encoder/decoder pairs (codecs) that lack adaptability and are agnostic to the data being compressed.

Decoder Video Compression

Toward Streaming Synapse Detection with Compositional ConvNets

no code implementations23 Feb 2017 Shibani Santurkar, David Budden, Alexander Matveev, Heather Berlin, Hayk Saribekyan, Yaron Meirovitch, Nir Shavit

Connectomics is an emerging field in neuroscience that aims to reconstruct the 3-dimensional morphology of neurons from electron microscopy (EM) images.

Deep Tensor Convolution on Multicores

no code implementations ICML 2017 David Budden, Alexander Matveev, Shibani Santurkar, Shraman Ray Chaudhuri, Nir Shavit

Deep convolutional neural networks (ConvNets) of 3-dimensional kernels allow joint modeling of spatiotemporal features.

Cannot find the paper you are looking for? You can Submit a new open access paper.