Correlative Multi-Label Multi-Instance Image Annotation

In this paper, each image is viewed as a bag of local re-gions, as well as it is investigated globally. A novel methodis developed for achieving multi-label multi-instance im-age annotation, where image-level (bag-level) labels andregion-level (instance-level) labels are both obtained. Theassociations between semantic concepts and visual featuresare mined both at the image level and at the region level.Inter-label correlations are captured by a co-occurence ma-trix of concept pairs. The cross-level label coherence en-codes the consistency between the labels at the image leveland the labels at the region level. The associations be-tween visual features and semantic concepts, the corre-lations among the multiple labels, and the cross-level la-bel coherence are sufficiently leveraged to improve anno-tation performance. Structural max-margin technique isused to formulate the proposed model and multiple inter-related classifiers are learned jointly. To leverage the avail-able image-level labeled samples for the model training, theregion-level label identification on the training set is firstlyaccomplished by building the correspondences between themultiple bag-level labels and the image regions. JEC dis-tance based kernels are employed to measure the similari-ties both between images and between regions. Experimen-tal results on real image datasets MSRC and Corel demon-strate the effectiveness of our method