Search Results for author:
Found 15 papers, 4 papers with code
SLaM: Student-Label Mixing for Distillation with Unlabeled Examples
Knowledge distillation with unlabeled examples is a powerful training paradigm for generating compact and lightweight student models in applications where the amount of labeled data is limited but one has access to a large pool of unlabeled data.
The Power of External Memory in Increasing Predictive Model Capacity
One way of introducing sparsity into deep networks is by attaching an external table of parameters that is sparsely looked up at different layers of the network.
Weighted Distillation with Unlabeled Examples
Our method is hyper-parameter free, data-agnostic, and simple to implement.
Robust Active Distillation
Distilling knowledge from a large teacher model to a lightweight one is a widely successful approach for generating compact, powerful models in the semi-supervised learning setting where a limited amount of labeled data is available.
A Theoretical View on Sparsely Activated Networks
After representing LSH-based sparse networks with our model, we prove that sparse networks can match the approximation power of dense networks on Lipschitz functions.
Bandit Sampling for Multiplex Networks
Most of the existing work focuses primarily on the monoplex setting where we have access to a network with only a single type of connection between entities.
Graph Belief Propagation Networks
With the wide-spread availability of complex relational data, semi-supervised node classification in graphs has become a central machine learning problem.
Low-Regret Active learning
We develop an online learning algorithm for identifying unlabeled data points that are most informative for training (i. e., active learning).
Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy
Neural network pruning is a popular technique used to reduce the inference costs of modern, potentially overparameterized, networks.
On Coresets for Support Vector Machines
A coreset is a small, representative subset of the original data points such that a models trained on the coreset are provably competitive with those trained on the original data set.
Provable Filter Pruning for Efficient Neural Networks
We present a provable, sampling-based approach for generating compact Convolutional Neural Networks (CNNs) by identifying and removing redundant filters from an over-parameterized network.
SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks
We introduce a pruning algorithm that provably sparsifies the parameters of a trained model in a way that approximately preserves the model's predictive accuracy.
Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Bounds
We present an efficient coresets-based neural network compression algorithm that sparsifies the parameters of a trained fully-connected neural network in a manner that provably approximates the network's output.
Small Coresets to Represent Large Training Data for Support Vector Machines
Support Vector Machines (SVMs) are one of the most popular algorithms for classification and regression analysis.
Training Support Vector Machines using Coresets
We present a novel coreset construction algorithm for solving classification tasks using Support Vector Machines (SVMs) in a computationally efficient manner.
