Rank4Class: Examining Multiclass Classification through the Lens of Learning to Rank
Multiclass classification (MCC) is a classical machine learning problem which aims to classify each instance into one of a predefined set of classes. Given an instance, a classification model computes a score for each class, all of which are then used to sort the classes. The performance of a classification model is usually measured by Top-K Accuracy (e.g., K=1 or 5). In this paper, we examine MCC through the lens of learning to rank (LTR) in the deep learning setting. By viewing MCC as to rank classes for an instance, we first argue that ranking metrics from the information retrieval literature, such as Normalized Discounted Cumulative Gain (NDCG), can be more informative than the existing Top-K metrics in evaluating the performance of classification models, especially for real-world user-facing applications. We further demonstrate that the most popular MCC architecture in deep learning can be mathematically formulated as a LTR pipeline equivalently, with a specific set of choices in terms of ranking model architecture and loss function. Based on these observations, we propose several techniques, stemmed from the rich LTR literature, to improve the MCC performance. Comprehensive empirical results on both text and image classification tasks, with diverse datasets and backbone models (e.g., BERT for text classification and ResNet for image classification) show the value of our proposed framework.
PDF Abstract
CIFAR-10
MIND
GoEmotions
