Search Results for author: Han Lei

Found 3 papers, 1 papers with code

Finding the Pillars of Strength for Multi-Head Attention

2 code implementations22 May 2023 Jinjie Ni, Rui Mao, Zonglin Yang, Han Lei, Erik Cambria

Specifically, the heads of MHA were originally designed to attend to information from different representation subspaces, whereas prior studies found that some attention heads likely learn similar features and can be pruned without harming performance.

feature selection

Cannot find the paper you are looking for? You can Submit a new open access paper.