According to the WHO, World report on vision 2019, the number of visually impaired people worldwide is estimated to be 2.2 billion, of whom at least 1 billion have a vision impairment that could have been prevented or is yet to be addressed. The world faces considerable challenges in terms of eye care, including inequalities in the coverage and quality of prevention, treatment, and rehabilitation services. Early detection and diagnosis of ocular pathologies would enable forestall of visual impairment. One challenge that limits the adoption of a computer-aided diagnosis tool by the ophthalmologist is, the sight-threatening rare pathologies such as central retinal artery occlusion or anterior ischemic optic neuropathy and others are usually ignored. In the past two decades, many publicly available datasets of color fundus images have been collected with a primary focus on diabetic retinopathy, glaucoma, and age-related macular degeneration, and few other frequent pathologies. The challenge for which this dataset was introduced aimed to unite the medical image analysis community to develop methods for automatic ocular disease classification of frequent diseases along with the rare pathologies. The Retinal Fundus Multi-disease Image Dataset (RFMiD) consists of a total of 3200 fundus images captured using three different fundus cameras with 46 conditions annotated through adjudicated consensus of two senior retinal experts. To the best of the authors knowledge, the dataset, RFMiD represents the only publicly available dataset that constitutes such a wide variety of diseases that appear in routine clinical settings. This aforementioned challenge promoted the development of generalizable models for screening retina, unlike the previous efforts that focused on the detection of specific diseases.