PREDICTION OF MICROVASCULAR INVASION OF HEPATOCELLAR CARCINOMA WITH CONTRAST-ENHANCED MR USING 3D CNN AND LSTM

Microvascular invasion (MVI) of hepatocellular carcinoma (HCC) is a crucial histopathologic prognostic factor associated with cancer recurrence after liver transplantation or hepatectomy. In this study, we propose a deep learning framework based on Convolutional Neural Network (CNN) and long short-term memory (LSTM) to extract discriminative information of HCCs from Contrast-enhanced MR images for MVI prediction. Specifically, 3D CNN is adopted to extract high-level deep features (spatial information) of LICC, and LSTM is used to extract sequential information (temporal information) of Contrast-enhanced MR for MVI prediction. Furthermore, we propose a deeply supervised loss function to improve the prediction performance by taking advantage of both the spatially and temporally discriminative features from Contrast-enhanced MR. Experimental results of 63 clinically confirmed HCCs with Contrast-enhanced MR demonstrate the superior performance of the proposed framework as follows: (1) 3D deep feature outperforms texture features in the radiomics approach for MVI prediction. (2) 3D CNN and LSTM used in the proposed deep learning framework yields better results than that of the 3D deep feature concatenation for MVI prediction. (3) Deeply supervised loss function that makes full use of the spatially and temporally discriminative features from Contrast-enhanced MR can further improve the performance of MVI prediction.