no code implementations • 20 Mar 2022 • Haichao Zhang, Kuangrong Hao, Witold Pedrycz, Lei Gao, Xuesong Tang, Bing Wei
The high-performance backbone network searched by VTCAS introduces the desirable features of convolutional neural networks into the Transformer architecture while maintaining the benefits of the multi-head attention mechanism.
no code implementations • 23 Mar 2021 • Haichao Zhang, Kuangrong Hao, Lei Gao, Xuesong Tang, Bing Wei
At the stage of block-level search, a relaxation method based on the gradient is proposed, using an enhanced gradient to design high-performance and low-complexity blocks.
no code implementations • 21 Dec 2020 • Haichao Zhang, Kuangrong Hao, Lei Gao, Bing Wei, Xuesong Tang
Deep neural networks (DNNs) have achieved remarkable success in computer vision; however, training DNNs for satisfactory performance remains challenging and suffers from sensitivity to empirical selections of an optimization algorithm for training.