Search Results for author:
Found 18 papers, 0 papers with code
PDP: Parameter-free Differentiable Pruning is All You Need
Also, PDP yields over 83. 1% accuracy on Multi-Genre Natural Language Inference with 90% sparsity for BERT, while the next best from the existing techniques shows 81. 5% accuracy.
R^2: Range Regularization for Model Compression and Quantization
Model parameter regularization is a widely used technique to improve generalization, but also can be used to shape the weight distributions for various purposes.
Ranked #1 on
Model Compression
on QNLI
HEiMDaL: Highly Efficient Method for Detection and Localization of wake-words
Streaming keyword spotting is a widely used solution for activating voice assistants.
Ranked #1 on
Keyword Spotting
on hey Siri
I see what you hear: a vision-inspired method to localize words
This paper explores the possibility of using visual object detection techniques for word localization in speech data.
Improving Voice Trigger Detection with Metric Learning
A detector is typically trained on speech data independent of speaker information and used for the voice trigger detection task.
DKM: Differentiable K-Means Clustering Layer for Neural Network Compression
For MobileNet-v1, which is a challenging DNN to compress, DKM delivers 63. 9% top-1 ImageNet1k accuracy with 0. 72 MB model size (22. 4x model compression factor).
Exploring Avenues Beyond Revised DSD Functionals: I. range separation, with xDSD as a special case
We have explored the use of range separation as a possible avenue for further improvement on our revDSD minimally empirical double hybrid functionals.
Chemical Physics
Large Scale Neural Architecture Search with Polyharmonic Splines
Neural Architecture Search (NAS) is a powerful tool to automatically design deep neural networks for many tasks, including image classification.
NASTransfer: Analyzing Architecture Transferability in Large Scale Neural Architecture Search
The typical way of conducting large scale NAS is to search for an architectural building block on a small dataset (either using a proxy set from the large dataset or a completely different small scale dataset) and then transfer the block to a larger dataset.
SNOW: Subscribing to Knowledge via Channel Pooling for Transfer & Lifelong Learning of Convolutional Neural Networks
SNOW is an efficient learning method to improve training/serving throughput as well as accuracy for transfer and lifelong learning of convolutional neural networks based on knowledge subscription.
SimEx: Express Prediction of Inter-dataset Similarity by a Fleet of Autoencoders
Our intuition is that, the more similarity exists between the unknown data samples and the part of known data that an autoencoder was trained with, the better chances there could be that this autoencoder makes use of its trained knowledge, reconstructing output samples closer to the originals.
Enabling real-time multi-messenger astrophysics discoveries with deep learning
Multi-messenger astrophysics is a fast-growing, interdisciplinary field that combines data, which vary in volume and speed of data processing, from many different instruments that probe the Universe using different cosmic messengers: electromagnetic waves, cosmic rays, gravitational waves and neutrinos.
MUTE: Data-Similarity Driven Multi-hot Target Encoding for Neural Network Design
Target encoding is an effective technique to deliver better performance for conventional machine learning methods, and recently, for deep neural networks as well.
Deep Learning for Multi-Messenger Astrophysics: A Gateway for Discovery in the Big Data Era
We discuss key aspects to realize this endeavor, namely (i) the design and exploitation of scalable and computationally efficient AI algorithms for Multi-Messenger Astrophysics; (ii) cyberinfrastructure requirements to numerically simulate astrophysical sources, and to process and interpret Multi-Messenger Astrophysics data; (iii) management of gravitational wave detections and triggers to enable electromagnetic and astro-particle follow-ups; (iv) a vision to harness future developments of machine and deep learning and cyberinfrastructure resources to cope with the scale of discovery in the Big Data Era; (v) and the need to build a community that brings domain experts together with data scientists on equal footing to maximize and accelerate discovery in the nascent field of Multi-Messenger Astrophysics.
Data-parallel distributed training of very large models beyond GPU capacity
GPUs have limited memory and it is difficult to train wide and/or deep models that cause the training process to go out of memory.
A Unified Approximation Framework for Compressing and Accelerating Deep Neural Networks
To address this issue, various approximation techniques have been investigated, which seek for a light weighted network with little performance degradation in exchange of smaller model size or faster inference.
PowerAI DDL
We train Resnet-101 on Imagenet 22K with 64 IBM Power8 S822LC servers (256 GPUs) in about 7 hours to an accuracy of 33. 8 % validation accuracy.
MEC: Memory-efficient Convolution for Deep Neural Network
However, all these indirect methods have high memory-overhead, which creates performance degradation and offers a poor trade-off between performance and memory consumption.
