Search Results for author:
Found 12 papers, 9 papers with code
REDS: Resource-Efficient Deep Subnetworks for Dynamic Resource Constraints
REDS support conventional deep networks frequently deployed on the edge and provide computational benefits even for small and simple networks.
Geometric Data Augmentations to Mitigate Distribution Shifts in Pollen Classification from Microscopic Images
We leverage the domain knowledge that geometric features are highly important for accurate pollen identification and introduce two novel geometric image augmentation techniques to significantly narrow the accuracy gap between the model performance on the train and test datasets.
Subspace-Configurable Networks
The obtained subspace is low-dimensional and has a surprisingly simple structure even for complex, non-invertible transformations of the input, leading to an exceptionally high efficiency of subspace-configurable networks (SCNs) when limited storage and computing resources are at stake.
DataComp: In search of the next generation of multimodal datasets
Multimodal datasets are a critical component in recent breakthroughs such as Stable Diffusion and GPT-4, yet their design does not receive the same research attention as model architectures or training algorithms.
The Role of Pre-training Data in Transfer Learning
We investigate the impact of pre-training data distribution on the few-shot and full fine-tuning performance using 3 pre-training methods (supervised, contrastive language-image and image-image), 7 pre-training datasets, and 9 downstream datasets.
REPAIR: REnormalizing Permuted Activations for Interpolation Repair
In this paper we look into the conjecture of Entezari et al. (2021) which states that if the permutation invariance of neural networks is taken into account, then there is likely no loss barrier to the linear interpolation between SGD solutions.
Studying the impact of magnitude pruning on contrastive learning methods
We study the impact of different pruning techniques on the representation learned by deep neural networks trained with contrastive loss functions.
Understanding the effect of sparsity on neural networks robustness
This paper examines the impact of static sparsity on the robustness of a trained network to weight perturbations, data corruption, and adversarial examples.
Deep Neural Network Pruning for Nuclei Instance Segmentation in Hematoxylin & Eosin-Stained Histological Images
We investigate the impact of weight pruning on the performance of both branches separately and on the final nuclei instance segmentation result.
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks
In this paper, we conjecture that if the permutation invariance of neural networks is taken into account, SGD solutions will likely have no barrier in the linear interpolation between them.
Interpretable and Transferable Models to Understand the Impact of Lockdown Measures on Local Air Quality
In this work, we estimate pollution reduction over the lockdown period by using the measurements from ground air pollution monitoring stations, training a long-term prediction model and comparing its predictions to measured values over the lockdown month. We show that our models achieve state-of-the-art performance on the data from air pollution measurement stations in Switzerland and in China: evaluate up to -15. 8% / +34. 4% change in NO2 / PM10 in Zurich; -35. 3 % / -3. 5 % and -42. 4 % / -34. 7 % in NO2 / PM2. 5 in Beijing and Wuhan respectively.
Class-dependent Compression of Deep Neural Networks
Motivated by the success of the lottery ticket hypothesis, in this paper we propose an iterative deep model compression technique, which keeps the number of false negatives of the compressed model close to the one of the original model at the price of increasing the number of false positives if necessary.
