Development of a machine learning-based model for predicting risk of early postoperative recurrence of hepatocellular carcinoma

Table 1 Demographic and clinical characteristics of patients

Variable		Overall (n = 371)	No HCC recurrence (n = 152)	HCC recurrence (n = 219)	P value	Training set (n = 296)	Test set (n = 75)	P value
Liver cirrhosis, n (%)	No	85 (22.911)	28 (18.421)	57 (26.027)	0.086	85 (22.911)	69 (23.311)	0.716
	Yes	286 (77.089)	124 (81.579)	162 (73.973)		227 (76.689)	59 (78.667)
Tumor size, n (%), cm	≤ 2	42 (11.321)	16 (10.526)	26 (11.872)	0.027	29 (9.797)	13 (17.333)	0.308
	2-5	168 (45.283)	78 (51.316)	90 (41.096)		137 (46.284)	31 (41.333)
	5-10	109 (29.380)	46 (30.263)	63 (28.767)		87 (29.392)	22 (29.333)
	≥ 10	52 (14.016)	12 (7.895)	40 (18.265)		43 (14.527)	9 (12.000)
Tumor number, n (%)	1	285 (76.819)	128 (84.211)	157 (71.689)	0.005	229 (77.365)	56 (74.667)	0.621
	≥ 2	86 (23.181)	24 (15.789)	62 (28.311)		67 (22.635)	19 (25.333)
Homogeneous tumor enhancement, n (%)	No	297 (80.054)	119 (78.289)	178 (81.279)	0.479	238 (80.405)	59 (78.667)	0.736
	Yes	74 (19.946)	33 (21.711)	41 (18.721)		58 (19.595)	16 (21.333)
Intratumoral necrosis, n (%)	No	142 (38.275)	69 (45.395)	73 (33.333)	0.019	113 (38.176)	29 (38.667)	0.938
	Yes	229 (61.725)	83 (54.605)	146 (66.667)		183 (61.824)	46 (61.333)
Tumor envelope intactness, n (%)	No	145 (39.084)	60 (39.474)	85 (38.813)	0.898	117 (39.527)	28 (37.333)	0.728
	Yes	226 (60.916)	92 (60.526)	134 (61.187)		179 (60.473)	47 (62.667)
Regular tumor margin, n (%)	No	176 (47.439)	73 (48.026)	103 (47.032)	0.850	139 (46.959)	37 (49.333)	0.713
	Yes	195 (52.561)	79 (51.974)	116 (52.968)		157 (53.041)	38 (50.667)
Intratumoral arteries, n (%)	No	174 (46.900)	89 (58.553)	85 (38.813)	< 0.001	135 (45.608)	39 (52.000)	0.322
	Yes	197 (53.100)	63 (41.447)	134 (61.187)		161 (54.392)	36 (48.000)
Adjacency to major vessels, n (%)	No	216 (58.221)	93 (61.184)	123 (56.164)	0.335	171 (57.770)	45 (60.000)	0.727
	Yes	155 (41.779)	59 (38.816)	96 (43.836)		125 (42.230)	30 (40.000)
Ascitic fluid, n (%)	No	262 (70.620)	106 (69.737)	156 (71.233)	0.756	207 (69.932)	55 (73.333)	0.564
	Yes	109 (29.380)	46 (30.263)	63 (28.767)		89 (30.068)	20 (26.667)
Age, n (%), yr	< 60	227 (61.186)	85 (55.921)	142 (64.840)	0.083	187 (63.176)	40 (53.333)	0.118
	≥ 60	144 (38.814)	67 (44.079)	77 (35.160)		109 (36.824)	35 (46.667)
Gender, n (%)	Male	280 (75.472)	118 (77.632)	162 (73.973)	0.421	221 (74.662)	59 (78.667)	0.472
	Female	91 (24.528)	34 (22.368)	57 (26.027)		75 (25.338)	16 (21.333)
HBsAg, n (%)	Negative	101 (27.224)	41 (26.974)	60 (27.397)	0.928	76 (25.676)	25 (33.333)	0.183
	Positive	270 (72.776)	111 (73.026)	159 (72.603)		220 (74.324)	50 (66.667)
HBeAg, n (%)	Negative	289 (77.898)	123 (80.921)	166 (75.799)	0.242	229 (77.365)	60 (80.000)	0.623
	Positive	82 (22.102)	29 (19.079)	53 (24.201)		67 (22.635)	15 (20.000)
ALBI, n (%), grade	1	67 (18.059)	26 (17.105)	41 (18.721)	0.907	54 (18.243)	13 (17.333)	0.710
	2	288 (77.628)	119 (78.289)	169 (77.169)		228 (77.027)	60 (80.000)
	3	16 (4.313)	7 (4.605)	9 (4.110)		14 (4.730)	2 (2.667)
MLR, n (%)	< 0.285	171 (46.092)	75 (49.342)	96 (43.836)	0.295	140 (47.297)	31 (41.333)	0.355
	≥ 0.285	200 (53.908)	77 (50.658)	123 (56.164)		156 (52.703)	44 (58.667)
GLR, n (%)	< 32.16	190 (51.213)	92 (60.526)	98 (44.749)	0.003	152 (51.351)	38 (50.667)	0.916
	≥ 32.16	181 (48.787)	60 (39.474)	121 (55.251)		144 (48.649)	37 (49.333)
PLR, n (%)	< 138.18	300 (80.863)	132 (86.842)	168 (76.712)	0.015	241 (81.419)	59 (78.667)	0.588
	≥ 138.18	71 (19.137)	20 (13.158)	51 (23.288)		55 (18.581)	16 (21.333)
ALP, n (%)	< 80.29	280 (75.472)	121 (79.605)	159 (72.603)	0.123	222 (75.000)	58 (77.333)	0.675
	≥ 80.29	91 (24.528)	31 (20.395)	60 (27.397)		74 (25.000)	17 (22.667)
LMR, n (%)	< 5.33	82 (22.102)	28 (18.421)	54 (24.658)	0.155	67 (22.635)	15 (20.000)	0.623
	≥ 5.33	289 (77.898)	124 (81.579)	165 (75.342)		229 (77.365)	60 (80.000)
FAR, n (%)	< 0.07	220 (59.299)	96 (63.158)	124 (56.621)	0.208	174 (58.784)	46 (61.333)	0.688
	≥ 0.07	151 (40.701)	56 (36.842)	95 (43.379)		122 (41.216)	29 (38.667)
SII, n (%)	< 312.71	198 (53.369)	93 (61.184)	105 (47.945)	0.012	157 (53.041)	41 (54.667)	0.801
	≥ 312.71	173 (46.631)	59 (38.816)	114 (52.055)		139 (46.959)	34 (45.333)
AFP, n (%), ng/mL	< 20	171 (46.092)	81 (53.289)	90 (41.096)	0.012	138 (46.622)	33 (44.000)	0.857
	20-400	86 (23.181)	37 (24.342)	49 (22.374)		69 (23.311)	17 (22.667)
	≥ 400	114 (30.728)	34 (22.368)	80 (36.530)		89 (30.068)	25 (33.333)
PLT, median (IQR), 109/L		168.000 (113.000, 209.000)	154.000 (112.000, 200.000)	171.000 (118.000, 216.000)	0.110	165.000 (111.000, 208.000)	178.000 (133.000, 220.000)	-1.451
Blood glucose, median (IQR), mmol/L		4.720 (4.340, 5.310)	4.840 (4.390, 5.660)	4.660 (4.300, 5.170)	0.009	4.710 (4.350, 5.310)	4.770 (4.300, 5.310)	-0.066

HCC: Hepatocellular carcinoma; ALBI: Albumin-bilirubin; MLR: Monocyte to lymphocyte ratio; GLR: γ-glutamyl transferase to lymphocyte ratio; PLR: Platelet count to lymphocyte count ratio; ALR: Alkaline phosphatase to lymphocyte count ratio; LMR: Lymphocyte count to monocyte count; FAR: Fibrinogen to albumin ratio; SII: Systemic immune-inflammation index; PLT: Blood platelets; AFP: Alpha-fetoprotein; IQR: Interquartile range.

Table 2 Performance metrics of six models in the training dataset

Model	AUC (95%CI)	Accuracy (95%CI)	Sensitivity (95%CI)	Specificity (95%CI)	F1 score (95%CI)
XGBoost	0.993 (0.984-1.000)	0.974 (0.969-0.979)	0.980 (0.971-0.988)	0.976 (0.966-0.986)	0.981 (0.977-0.985)
GNB	0.692 (0.624-0.761)	0.671 (0.660-0.683)	0.754 (0.689-0.820)	0.566 (0.486-0.646)	0.728 (0.705-0.750)
MLP	0.574 (0.501-0.648)	0.563 (0.516-0.611)	0.517 (0.274-0.761)	0.637 (0.410-0.863)	0.549 (0.411-0.686)
SVM	0.308 (0.240-0.376)	0.453 (0.391-0.514)	0.204 (-0.182-0.591)	0.800 (0.413-1.187)	NaN
Logistic	0.702 (0.634-0.770)	0.677 (0.657-0.696)	0.747 (0.654-0.841)	0.589 (0.499-0.679)	0.727 (0.689-0.765)
AdaBoost	0.814 (0.761-0.868)	0.725 (0.701-0.748)	0.661 (0.585-0.737)	0.825 (0.773-0.876)	0.737 (0.700-0.773)

95%CI: 95% confidence interval; AUC: Area under the curve; GNB: Complement NB; MLP: Multilayer perceptron; SVM: Support vector machine; NaN: Not a number.

Table 3 Performance metrics of six models in the validation dataset

Model	AUC (95%CI)	Accuracy (95%CI)	Sensitivity (95%CI)	Specificity (95%CI)	F1 score (95%CI)
XGBoost	0.734 (0.601-0.867)	0.683 (0.658-0.708)	0.662 (0.596-0.729)	0.782 (0.717-0.847)	0.694 (0.636-0.752)
GNB	0.657 (0.512-0.802)	0.618 (0.583-0.653)	0.647 (0.443-0.851)	0.662 (0.464-0.861)	0.641 (0.535-0.748)
MLP	0.548 (0.396-0.699)	0.514 (0.452-0.575)	0.575 (0.398-0.752)	0.627 (0.491-0.764)	0.577 (0.471-0.682)
SVM	0.363 (0.217-0.509)	0.446 (0.381-0.511)	0.452 (0.035-0.870)	0.617 (0.188-1.046)	NaN
Logistic	0.661 (0.517-0.806)	0.632 (0.602-0.661)	0.797 (0.709-0.886)	0.525 (0.414-0.637)	0.728 (0.706-0.751)
AdaBoost	0.649 (0.504-0.793)	0.618 (0.569-0.667)	0.591 (0.330-0.852)	0.780 (0.610-0.950)	0.617 (0.403-0.831)

95%CI: 95% confidence interval; AUC: Area under the curve; GNB: Complement NB; MLP: Multilayer perceptron; SVM: Support vector machine; NaN: Not a number.