Deep learning applied to the imaging diagnosis of hepatocellular carcinoma

Abstract

Each year, hepatocellular carcinoma is diagnosed in more than half a million people worldwide. It is the fifth most common cancer in men and the seventh most common cancer in women. Its diagnosis is currently made using imaging techniques, such as computed tomography and magnetic resonance imaging. For most cirrhotic patients, these methods are enough for diagnosis, foregoing the necessity of a liver biopsy. In order to improve outcomes and bypass obstacles, many companies and clinical centers have been trying to develop deep learning systems that could be able to diagnose and classify liver nodules in the cirrhotic liver, in which the neural networks are one of the most efficient approaches to accurately diagnose liver nodules. Despite the advances in deep learning systems for the diagnosis of imaging techniques, there are many issues that need better development in order to make such technologies more useful in daily practice.

Key Words: Hepatocellular carcinoma; Cirrhosis; Machine learning; Artificial intelligence

Core Tip: Hepatocellular carcinoma is diagnosed using imaging techniques, such as computed tomography and magnetic resonance imaging. In order to improve outcomes and bypass obstacles, many companies and clinical centers have been trying to develop deep learning systems that could be able to diagnose and classify liver nodules in the cirrhotic liver. Neural networks have become one of the most efficient approaches to accurately diagnose liver nodules using deep learning systems. Therefore, with the improvement of these techniques in the long term, they could be applicable in daily practice, modifying outcomes.

INTRODUCTION

Each year, hepatocellular carcinoma (HCC) is diagnosed in more than half a million people worldwide, and it is the fifth most common cancer in men and the seventh most common cancer in women[1]. The greatest burden of this disease is in developing countries, such as Southeast Asia and Sub-Saharan Africa, where hepatitis B is endemic[2,3].

The incidence of HCC has been rising, unlike many other types of neoplasms[4]. This is expected to change, as the worldwide incidence of viral hepatitis B and C is expected to subdue in the next generation via vaccination and treatment, respectively. Nevertheless, the acute rise in the prevalence of nonalcoholic steatohepatitis in the last couple of decades might become a key risk factor for HCC and could become solely responsible for sustaining its incidence, both in the Western and Eastern population[5,6].

Therefore, understanding the diagnostic and therapeutic approaches to this disease is essential, especially if we keep in mind the quintessential basics of prevention and early detection to improve results[7,8].

DIAGNOSIS OF HCC

HCC diagnosis is currently made using imaging techniques, such as computed tomography and magnetic resonance imaging (MRI). For most cirrhotic patients, these methods are enough for diagnosis, foregoing the necessity of a liver biopsy[9-11]. Nevertheless, the precise diagnosis of a liver nodule via imaging techniques is a rather challenging task, requiring a highly trained and specialized multidisciplinary team of radiologists, hepatologists and oncologists.

In order to facilitate communication between professionals of such a team, a system for reporting imaging of liver nodules has been developed and adopted worldwide–the Liver Imaging Reporting And Data System (LI-RADS)[12]. The LI-RADS classification[13] can be found in Table 1. Although this was an attempt into standardization, a high discordance rate among radiologists has been described[14]. Inter-rater reliability has varied greatly in studies, with Cohen’s kappa coefficients ranging from 0.35 to 0.73[15-19]. This is expected, since this classification requires high-quality imaging and radiologists with vast experience[19,20]. Another very important argument is that where HCC incidence is higher (developing countries), highly specialized radiologists are scarcest despite a high volume of patients[21]. In order to improve outcomes and bypass these obstacles, many companies and clinical centers have been trying to develop deep learning systems (DLS) intended to accurately diagnose liver nodules in the cirrhotic liver[22].

Table 1 Liver imaging reporting and data system classification[13].

Category	Description
LR-1	Definitely benign
LR-2	Probably benign
LR-3	Intermediate probability of HCC
LR-4	High probability of HCC, not 100%
LR-5	Definitely HCC
LR-5V	Definite venous invasion regardless of other imaging features
LR treated	LR-5 lesion status post-locoregional treatment
LR-M	Non-HCC malignancies that may occur in cirrhosis: metastases, lymphoma, cholangiocarcinoma, PTLD

HCC: Hepatocellular carcinoma; PTLD: Post-transplant lymphoproliferative disorder.

DLS AND HCC

There are many DLS approaches available in the literature, where neural networks are gaining much attention currently as one of the best approaches to accurately diagnose liver nodules. Particularly, a DLS based on convolutional neural networks (CNN) could achieve such capacities after machine learning (ML) by using examples of images with and without the disease in question[8]. Unlike other DLS, CNN does not demand a clear definition of the lesion in order to interpret the images[23], which might lead to discovery of additional differential characteristics that are not currently known by radiologists[24]. Table 2 summarizes the main characteristics about the studies in diagnosis of liver tumors with images and clinical data using DLS.

Table 2 Main characteristics of the studies that evaluate deep learning for liver tumor diagnosis throughout images or clinical data.

Ref.	Country	Deep learning method	Accuracy	Sensitivity	Specificity	AUROC	DLS performance compared	Multicenter validation	Conclusion
Hamm et al[8], 2019	United States	Proof-of-concept validation CNN	92%	92%	98%	0.992	Better than radiologists	Not done	DLS was feasibility for classifying lesions with typical imaging features from six common hepatic lesion types
Yamashita et al[14], 2020	United States	CNN architectures: custom-made network and transfer learning-based network	60.4%	NA	NA	LR-1/2: 0.85. LR-3: 0.90. LR-4: 0.63. LR-5: 0.82	Transfer learning model was better	Performed	There is a feasibility of CNN for assigning LI-RADS categories from a relatively small dataset but highlights the challenges of model development and validation
Shi et al[23], 2020	China	Three CDNs	Model-A: 83.3%, B: 81.1%, C: 85.6%	NA	NA	Model-A: 0.925; B: 0.862; C: 0.920	Three model compared, A and C with better results	Not done	Three-phase CT protocol without precontrast showed similar diagnosis accuracy as four-phase protocol in differentiating HCC. It can reduce the radiation dose
Yasaka et al[25], 2018	Japan	CNN	84%	Category1: A: 71%; B: 33%; C: 94%; D: 90%; E: 100%	NA	0.92	Not applicable	Not done	Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT
Trivizakis et al[28], 2019	Greece	3D and 2D CNN	83%	93%	67%	0.80	Superior compared with 2D CNN model	Not done	3D CNN architecture can bring significant benefit in DW-MRI liver discrimination and potentially in numerous other tissue classification problems based on tomographic data, especially in size-limited, disease specific clinical datasets
Wang et al[41], 2019	United States	Proof-of-concept “interpretable” CNN	88%	82.9%	NA	NA	Not applicable	Not done	This interpretable deep learning system demonstrates proof of principle for illuminating portions of a pre-trained deep neural network’s decision-making, by analyzing inner layers and automatically describing features contributing to predictions
Frid-Adar et al[45], 2018	Israel	GANs	Classic data: 78.6%. Synthetic data: 85.7%	Classic data: 78.6%. Synthetic data: 85.7%	Classic data: 88.4%. Synthetic data: 92.4%	NA	Synthetic data augmentation is better than classic data augmentation	Not done	This approach to synthetic data augmentation can generalize to other medical classification applications and thus support radiologists’ efforts to improve diagnosis
Wang et al[47], 2019	Japan	CNN with clinical data	NA	NA	NA	Clinical model: 0.723. Model: A: 0.788; B: 0.805; C: 0825.	Combined model C present with better results	Not done	The AUC of the combined model is about 0.825, which is much better than the models using clinical data only or CT image only
Sato et al[48], 2019	Japan	Fully connected neural network with 4 layers of neurons using only biomarkers, gradient boosting (non-linear model) and others	DLS: 83.54%. Gradient boosting: 87.34%	Gradient boosting: 93.27%	Gradient boosting: 75.93%	DLS: 0.884. Gradient boosting: 0.940	Deep learning was not the optimal classifier in the current study	Not done	The gradient boosting model reduced the misclassification rate by about half compared with a single tumor marker. The model can be applied to various kinds of data and thus could potentially become a translational mechanism between academic research and clinical practice
Naeem et al[49], 2020	Pakistan	MLP, SVM, RF, and J48 using ten-fold cross-validation	MLP: 99%	NA	NA	MLP: 0.983. SVM: 0.966. RF: 0.964. J48: 0.959	MLP model present with better results	Radiopaedia dataset	Our proposed system has the capability to verify the results on different MRI and CT scan databases, which could help radiologists to diagnose liver tumors

¹Five categories: A: Classic hepatocellular carcinomas; B: Malignant liver tumors other than classic and early hepatocellular carcinomas; C: Indeterminate masses or mass like lesions (including early hepatocellular carcinomas and dysplastic nodules) and rare benign liver masses other than hemangiomas and cysts; D: Hemangiomas; E: Cysts. AUC: Area under the curve; AUROC: Area under the receiver operating characteristic curve; CDNs: Convolutional dense networks CNN: Convolutional neural network; CT: Computed tomography; DLS: Deep learning system; DW-MRI: Diffusion weighted magnetic resonance imaging; GANs: Generative adversarial networks; HCC: Hepatocellular carcinoma; LI-RADS: Liver Imaging Reporting and Data System; LR: LI-RADS; MLP: Multiplayer perceptron; MRI: Magnetic resonance imaging; NA: Not available; RF: Random forest; SVM: Support vector machine.

There are several DLS applied in the recognition of image patterns[25,26], from which CNN-based approaches have achieved the highest performance[25]. While conventional deep learning algorithms require specific features to be extracted from images before the learning process, the application of CNNs requires rather a simpler feature representation based on the original image pixel intensities, also allowing to use all available image information in the learning process[27]. Moreover, CNNs can process extracted image features by several convolution filters, which allow analysis of the image at different granularities. Therefore, CNN is one of the most advanced techniques for artificial intelligence[25], which has been implemented with success for imaging and clinical interpretation in many medical fields. For example, CNN has been validated to identify liver tumors[28], the prognosis of esophageal variceal bleeding in cirrhotic patients[29], to predict the mortality of liver transplantation[30, 31], to predict the prognosis of HCC[32-37] Helicobacter pylori infection[38], colonic polyps[39], to help classify mammary cancer, head and neck cancer and gliomas[36] and to focal liver disease detection[40].

In the topic of liver tumors, many studies have shown that CNNs performed the same or better when compared to experienced radiologists. Hamm et al[8] developed and validated a CNN that classified six types of common hepatic lesions on multi-phasic MRI, achieving better sensitivity and specificity when compared to board-certified radiologists[8]. Nevertheless, this study was developed in only one center, using local and typical images, with no external validation. In a follow-up to this study, Wang et al[41] used a pre-trained CNN in a model-agonistic approach capable of distinguishing among several types of lesions and developed a post-hoc algorithm with the purpose of standardizing the lesion features used in the diagnosis. Such a tool could interact with other standardized scales, such as LI-RADS, validating auxiliary resources and improving clinical practicality[41]. This study found a sensitivity of 82.9% for adequate identification of imaging characteristics when analyzing lesions from a databank. It is expected that this type of DLS that can be transparent regarding its steps towards the diagnosis will have better clinical acceptance.

Yamashita et al[14] developed a DLS applied to diagnose liver carcinoma by using two CNNs: a pre-trained network with an input of triple-phase images (trained with transfer learning from other CNNs) and a custom-made network with an input of quadruple-phase images (trained from scratch from internal data)[14]. However, by using external data from other pre-trained CNNs, Zech et al[42] showed that the performance of the DLS worsened when compared to CNNs trained with internal data, showing that it is not still proved that CNNs trained on X-rays from one hospital or one group of hospitals will work equally well at different hospitals. This has also been demonstrated for the detection of pneumonia in chest X-rays, where CNN performed worse when exposed to external data with a wide range of diseases and radiological findings[42]. Besides, such CNNs could be used for the determination of LI-RADS category, which has been shown to be possible[14], even from a small data set. Nevertheless, external validation seems to be a major obstacle for the dissemination of ML tools. There are many devices that produce images, and there are many ways to store data from these exams.

When compared to other DLS, another advantage of the use of CNNs is that it can improve the diagnosis by using less images for ML, reducing the time of exam and the amount of exposure to radiation[23,43,44]. Moreover, by generating additional training samples through data augmentation, the liver lesion classification sensitivity and accuracy are enhanced whilst less images are required in the ML process[45]. Moreover, the sensitivity, specificity, and accuracy can be manually calculated with the confusion matrix. In Table 3, we compare the best ML algorithms for classification[46].

Table 3 Best machine learning algorithms for classification[36].

Algorithm	Pros	Cons
Naïve Bayes Classifier	Simple, easy and fast. Not sensitive to irrelevant features. Works great in practice. Needs less training data. For both multi-class and binary classification. Works with continuous and discrete data	Accepts every feature as independent. This is not always the truth
Decision Trees	Easy to understand. Easy to generate rules. There are almost no hyperparameters to be tuned. Complex decision tree models can be significantly simplified by its visualizations	Might suffer from overfitting. Does not easily work with nonnumerical data. Low prediction accuracy for a dataset in comparison with other algorithms. When there are many class labels, calculations can be complex
Support Vector Machines	Fast algorithm. Effective in high dimensional spaces. Great accuracy. Power and flexibility from kernels. Works very well with a clear margin of separation. Many applications	Does not perform well with large data sets. Not so simple to program. Does not perform so well when the data comes with more noise i.e. target classes are overlapping
Random Forest Classifier	The overfitting problem does not exist. Can be used for feature engineering i.e. for identifying the most important features among all available features in the training dataset. Runs very well on large databases. Extremely flexible and have very high accuracy. No need for preparation of the input data	Complexity. Requires a lot of computational resources. Time-consuming. Need to choose the number of trees
KNN Algorithm	Simple to understand and easy to implement. Zero to little training time. Works easily with multi-class data sets. Has good predictive power. Does well in practice	Computationally expensive testing phase. Can have skewed class distributions. The accuracy can be decreased when it comes to high-dimension data. Needs to define a value for the parameter k

KNN: K-nearest neighbors.

A DLS has been proposed for the prediction of HCC recurrence, using data from computed tomography combined with clinical information[47]. The triple layer model including imaging studies, clinical data and a filtering of this data has had the better performance, with an area under the receiver operating characteristic curve (AUROC) of 0.825. This is way more precise than the current tools are. Furthermore, Sato et al[48] proposed a ML model for predicting HCC using data obtained during clinical practice[48]. The AUROC of the optimal hyperparameter, gradient boosting model, involving multiple laboratories and tumor markets was 0.940. However, when compared with single tumor markers the AUROC to the prediction of HCC for alpha-fetoprotein, des-gamma-carboxy prothrombin and alpha-fetoprotein-L3 were 0.766, 0.644 and 0.683, respectively. Accordingly, a combination of multiple data can provide a reliable diagnostic tool.

A preliminary study has attempted to diagnose liver masses using a CNN without the aid of a radiologist, achieving a high accuracy to differentiate HCC from benign liver masses, achieving an AUROC of 0.92[25]. In another study, a CNN was designed to differentiate HCC from metastatic liver masses on MRI, but this time the DLS used a 3-D representation, with higher accuracy (83.0% of the 3-D model vs 65.2% of the 2-D model)[28]. Nevertheless, the authors stressed that more studies with larger databanks are needed to verify the accuracy of this method. Besides that, Naeem et al[49] performed a hybrid-feature analysis between computed tomography scans and MRI for differentiation of liver tumors using DLS. The accuracy of multilayer perceptron model for hepatoblastoma, cyst, hemangioma, hepatocellular adenoma, HCC and metastasis were 99.67%, 99.33%, 98.33%, 99.67%, 97.33% and 99.67% respectively[49]. This method can be helpful to reduce human error.

Therefore, despite the advances in DLS for the diagnosis of imaging techniques, there are many points that need better development in order to become useful and common tools in daily practice. These techniques currently require comparison with trained radiologists and the application for many databanks with atypical images to achieve better results and the use of less radiation for HCC diagnosis.

We previously presented several DLS applied to liver nodule diagnosis; however, they are not able to segment the nodule from the liver in the analyzed images. Moreover, automatic nodule segmentation in an image is a challenging task since this kind of lesion may show a high variability in shape, appearance and localization and is dependent on the equipment, contrast, lesion type, lesion stage and so on[50].

There are some liver nodule segmentation methods available in the literature, and in one of them[50] a fully convolutional network architecture was adopted to determine an approximation for where the nodule was located on the image. This CNN works on four resolution levels, learning local and global image features. The final nodule segmentation is obtained by using post-processing techniques and a random forest classifier, achieving a quality comparable to a human expert.

However, this method uses hand-crafted features that need the supervision of an expert. There are also automatic approaches that can segment the nodule[51], where a CNN is used for ML. To refine the segmentation results, this method applies conditional random fields to eliminate the false segmentation points in the segmentation results, improving accuracy. However, liver nodule segmentation in general still needs improvements to achieve a better accuracy and practical applicability. Furthermore, it is necessary for more research effort in DLS to at the same time detect the tumor in the liver and segment it on the image.

CONCLUSION

In conclusion, the goal of statistical methods is to achieve conclusions about a population from data that are collected from a representative sample of that population, such as linear and logistic regression. Therefore, the objective is to comprehend the associations among variables. However, as reported by Sidey-Gibbons and Sidey-Gibbons[36], the primary concern about DLS is an accurate prediction. Moreover, explaining the relationship between predictors and outcomes when the relationship is non-linear is difficult. However, in several DLS as improving navigation, translating documents or recognizing objects in videos, understanding the relationship between features and outcomes is less important[46]. In summary, enhancement of DLS features will allow more accurate diagnosis in the medical field. For future research, we recommend to test deep learning methods in other datasets (e.g., other hospitals), develop an easy usable interface and introduce the tool in daily medical practice.