Search Results for author:
Found 17 papers, 7 papers with code
Oh! We Freeze: Improving Quantized Knowledge Distillation via Signal Propagation Analysis for Large Language Models
Large generative models such as large language models (LLMs) and diffusion models have revolutionized the fields of NLP and computer vision respectively.
ZiCo-BC: A Bias Corrected Zero-Shot NAS for Vision Tasks
To solve the problem, we propose a novel bias correction on ZiCo, called ZiCo-BC.
Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities
Recently, zero-shot (or training-free) Neural Architecture Search (NAS) approaches have been proposed to liberate NAS from the expensive training process.
TIPS: Topologically Important Path Sampling for Anytime Neural Networks
Anytime neural networks (AnytimeNNs) are a promising solution to adaptively adjust the model complexity at runtime under various hardware resource constraints.
ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients
Based on this theoretical analysis, we propose a new zero-shot proxy, ZiCo, the first proxy that works consistently better than #Params.
Restructurable Activation Networks
To address this question, we propose a new paradigm called Restructurable Activation Networks (RANs) that manipulate the amount of non-linearity in models to improve their hardware-awareness and efficiency.
Super-Efficient Super Resolution for Fast Adversarial Defense at the Edge
Autonomous systems are highly vulnerable to a variety of adversarial attacks on Deep Neural Networks (DNNs).
On the relationship between topology and gradient propagation in deep networks
(ii) Can certain topological characteristics of deep networks indicate a priori (i. e., without training) which models, with a different number of parameters/FLOPS/layers, achieve a similar accuracy?
New Directions in Distributed Deep Learning: Bringing the Network at Forefront of IoT Design
In this paper, we first highlight three major challenges to large-scale adoption of deep learning at the edge: (i) Hardware-constrained IoT devices, (ii) Data security and privacy in the IoT era, and (iii) Lack of network-aware deep learning algorithms for distributed inference across multiple IoT devices.
FedMAX: Mitigating Activation Divergence for Accurate and Communication-Efficient Federated Learning
In this paper, we identify a new phenomenon called activation-divergence which occurs in Federated Learning (FL) due to data heterogeneity (i. e., data being non-IID) across multiple users.
EdgeAI: A Vision for Deep Learning in IoT Era
The significant computational requirements of deep learning present a major bottleneck for its large-scale adoption on hardware-constrained IoT-devices.
How does topology influence gradient propagation and model performance of deep networks with DenseNet-type skip connections?
In this paper, we reveal that the topology of the concatenation-type skip connections is closely related to the gradient propagation which, in turn, enables a predictable behavior of DNNs' test performance.
Memory- and Communication-Aware Model Compression for Distributed Deep Learning Inference on IoT
Therefore, we propose Network of Neural Networks (NoNN), a new distributed IoT learning paradigm that compresses a large pretrained 'teacher' deep network into several disjoint and highly-compressed 'student' modules, without loss of accuracy.
Dream Distillation: A Data-Independent Model Compression Framework
Model compression is eminently suited for deploying deep learning on IoT-devices.
On Network Science and Mutual Information for Explaining Deep Neural Networks
In this paper, we present a new approach to interpret deep learning models.
A Dynamic Network and Representation LearningApproach for Quantifying Economic Growth fromSatellite Imagery
To this end, we propose a new network science- and representation learning-based approach that can quantify economic indicators and visualize the growth of various regions.
