L2-Net: Deep Learning of Discriminative Patch Descriptor in Euclidean Space

The research focus of designing local patch descriptors has gradually shifted from handcrafted ones (e.g., SIFT) to learned ones. In this paper, we propose to learn high per- formance descriptor in Euclidean space via the Convolu- tional Neural Network (CNN). Our method is distinctive in four aspects: (i) We propose a progressive sampling strat- egy which enables the network to access billions of train- ing samples in a few epochs. (ii) Derived from the ba- sic concept of local patch matching problem, we empha- size the relative distance between descriptors. (iii) Extra supervision is imposed on the intermediate feature maps. (iv) Compactness of the descriptor is taken into account. The proposed network is named as L2-Net since the out- put descriptor can be matched in Euclidean space by L2 distance. L2-Net achieves state-of-the-art performance on the Brown datasets [16], Oxford dataset [18] and the new- ly proposed Hpatches dataset [11]. The good generaliza- tion ability shown by experiments indicates that L2-Net can serve as a direct substitution of the existing handcrafted de- scriptors. The pre-trained L2-Net is publicly available.