1 code implementation • 1 Apr 2023 • Tomer Ronen, Omer Levy, Avram Golbert
In this work, we apply this approach to Vision Transformers by introducing a novel image tokenization scheme, replacing the standard uniform grid with a mixed-resolution sequence of tokens, where each token represents a patch of arbitrary size.
no code implementations • 19 Apr 2022 • Niv Nayman, Avram Golbert, Asaf Noy, Tan Ping, Lihi Zelnik-Manor
Encouraged by the recent transferability results of self-supervised models, we propose a method that combines self-supervised and supervised pretraining to generate models with both high diversity and high accuracy, and as a result high transferability.