Characterization of renal masses with MRI-based radiomics: assessment of inter-package and inter-observer reproducibility in a prospective pilot study

Objectives To evaluate radiomics features' reproducibility using inter-package/inter-observer measurement analysis in renal masses (RMs) based on MRI and to employ machine learning (ML) models for RM characterization. Methods 32 Patients (23M/9F; age 61.8 +/- 10.6 years) with RMs (25 renal cell carcinomas (RCC)/7 benign masses; mean size, 3.43 +/- 1.73 cm) undergoing resection were prospectively recruited. All patients underwent 1.5 T MRI with T2-weighted (T2-WI), diffusion-weighted (DWI)/apparent diffusion coefficient (ADC), and pre-/post-contrast-enhanced T1-weighted imaging (T1-WI). RMs were manually segmented using volume of interest (VOI) on T2-WI, DWI/ADC, and T1-WI pre-/post-contrast imaging (1-min, 3-min post-injection) by two independent observers using two radiomics software packages for inter-package and inter-observer assessments of shape/histogram/texture features common to both packages (104 features; n = 26 patients). Intra-class correlation coefficients (ICCs) were calculated to assess inter-observer and inter-package reproducibility of radiomics measurements [good (ICC >= 0.8)/moderate (ICC = 0.5-0.8)/poor (ICC < 0.5)]. ML models were employed using reproducible features (between observers and packages, ICC > 0.8) to distinguish RCC from benign RM. Results Inter-package comparisons demonstrated that radiomics features from T1-WI-post-contrast had the highest proportion of good/moderate ICCs (54.8-58.6% for T1-WI-1 min), while most features extracted from T2-WI, T1-WI-pre-contrast, and ADC exhibited poor ICCs. Inter-observer comparisons found that radiomics measurements from T1-WI pre/post-contrast and T2-WI had the greatest proportion of features with good/moderate ICCs (95.3-99.1% T1-WI-post-contrast 1-min), while ADC measurements yielded mostly poor ICCs. ML models generated an AUC of 0.71 [95% confidence interval = 0.67-0.75] for diagnosis of RCC vs. benign RM. Conclusion Radiomics features extracted from T1-WI-post-contrast demonstrated greater inter-package and inter-observer reproducibility compared to ADC, with fair accuracy for distinguishing RCC from benign RM. Clinical relevance Knowledge of reproducibility of MRI radiomics features obtained on renal masses will aid in future study design and may enhance the diagnostic utility of radiomics models for renal mass characterization.