Token and Head Adaptive Transformers for Efficient Natural Language Processing

no code implementations • COLING 2022 • Chonghan Lee, Md Fahim Faysal Khan, Rita Brugarolas Brufau, Ke Ding, Vijaykrishnan Narayanan

While pre-trained language models like BERT have achieved impressive results on various natural language processing tasks, deploying them on resource-restricted devices is challenging due to their intensive computational cost and memory footprint.

Transformer-based Machine Learning for Fast SAT Solvers and Logic Synthesis

1 code implementation • 15 Jul 2021 • Feng Shi, Chonghan Lee, Mohammad Khairul Bashar, Nikhil Shukla, Song-Chun Zhu, Vijaykrishnan Narayanan

Our model has a scale-free structure which could process varying size of instances.

BIG-bench Machine Learning Computational Efficiency

STAR: Sparse Transformer-based Action Recognition

1 code implementation • 15 Jul 2021 • Feng Shi, Chonghan Lee, Liang Qiu, Yizhou Zhao, Tianyi Shen, Shivran Muralidhar, Tian Han, Song-Chun Zhu, Vijaykrishnan Narayanan

The cognitive system for human action and behavior has evolved into a deep learning regime, and especially the advent of Graph Convolution Networks has transformed the field in recent years.

Action Recognition Temporal Action Localization