Inter- and intra-observer agreement in dermatologists' diagnoses of hyperpigmented facial lesions and development of an algorithm for automated diagnosis

Background Hyperpigmentation has varied aetio-pathologies. Hence, accurate and reproducible diagnosis of the type of hyperpigmentation is essential for effective management. It is typically made clinically by dermatologists but the rate of inter- and intra-observer agreement/variability is unknown. Hyperpigmented facial lesions are extremely common but access to dermatological services is difficult or costly in most countries. Thus, it is desired to evaluate dermatologists' inter- and intra-observer agreement in the diagnosis and to develop an algorithm for automated diagnosis. Materials and Methods Hyperpigmented lesions on 392 facial images were diagnosed by three experienced dermatologists either jointly or independently, and this process was subsequently repeated for 52 randomly selected images. When there was non-concordance amongst the dermatologists for the diagnosis, a majority decision was taken as correct diagnosis. Inter-observer and intra-observer agreement were analysed for the diagnosis of the hyperpigmented lesions. Thereafter, a multiclass classification method was developed to perform the task in an automatic manner. The developed algorithm was compared and validated against the ground truth derived from the dermatologists. Results Both inter- and intra-observer agreements are in the moderate range. The algorithm agreed well with the derived ground truth, with a Kappa value of 0.492, which is similar to the Kappa values of inter- and intra- observer agreements. Conclusion The rates of inter- and intra-observer agreement in the diagnosis of hyperpigmented facial lesions amongst dermatologists were moderate in this study. Compared to visual assessment from the dermatologists, automated diagnosis using the developed algorithm achieved a high rate of concordance.