Search Results for author: Ali Edalati

Found 4 papers, 0 papers with code

Robustness to distribution shifts of compressed networks for edge devices

no code implementations22 Jan 2024 Lulan Shen, Ali Edalati, Brett Meyer, Warren Gross, James J. Clark

It is important to investigate the robustness of compressed networks in two types of data distribution shifts: domain shifts and adversarial perturbations.

Knowledge Distillation Quantization

KronA: Parameter Efficient Tuning with Kronecker Adapter

no code implementations20 Dec 2022 Ali Edalati, Marzieh Tahaei, Ivan Kobyzev, Vahid Partovi Nia, James J. Clark, Mehdi Rezagholizadeh

We apply the proposed methods for fine-tuning T5 on the GLUE benchmark to show that incorporating the Kronecker-based modules can outperform state-of-the-art PET methods.

Language Modelling

Kronecker Decomposition for GPT Compression

no code implementations ACL 2022 Ali Edalati, Marzieh Tahaei, Ahmad Rashid, Vahid Partovi Nia, James J. Clark, Mehdi Rezagholizadeh

GPT is an auto-regressive Transformer-based pre-trained language model which has attracted a lot of attention in the natural language processing (NLP) domain due to its state-of-the-art performance in several downstream tasks.

Knowledge Distillation Language Modelling +1

Cannot find the paper you are looking for? You can Submit a new open access paper.