Wang H, Wang X, Du YS, Wang Y, Bai ZJ, Wu D, Tang WL, Zeng HL, Tao J, He J. Non-contrast computed tomography radiomics model to predict benign and malignant thyroid nodules with lobe segmentation: A dual-center study. World J Radiol 2025; 17(6): 106682 [DOI: 10.4329/wjr.v17.i6.106682]
Corresponding Author of This Article
Jian He, MD, PhD, Associate Professor, Chief Physician, Department of Nuclear Medicine, Nanjing Drum Tower Hospital, The Affiliated Hospital of Medicine School, Nanjing University, No. 321 Zhongshan Road, Nanjing 210008, Jiangsu Province, China. hjxueren@126.com
Research Domain of This Article
Radiology, Nuclear Medicine & Medical Imaging
Article-Type of This Article
Retrospective Study
Open-Access Policy of This Article
This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Hao Wang, Yu-Sheng Du, You Wang, Zhuo-Jie Bai, Department of Radiology, The Fourth Affiliated Hospital of Nanjing Medical University, Nanjing 210031, Jiangsu Province, China
Xuan Wang, Di Wu, Wu-Liang Tang, Department of Radiology, Zhongda Hospital Southeast University (Jiangbei), Nanjing 210048, Jiangsu Province, China
Han-Ling Zeng, Jing Tao, Department of General Surgery, The Fourth Affiliated Hospital of Nanjing Medical University, Nanjing 210031, Jiangsu Province, China
Jian He, Department of Nuclear Medicine, Nanjing Drum Tower Hospital, The Affiliated Hospital of Medicine School, Nanjing University, Nanjing 210008, Jiangsu Province, China
Author contributions: Wang H wrote the original manuscript, contributed software, and resources to the manuscript; Wang X reviewed and edited the manuscript; Wang H and Wang X wrote the manuscript, they contributed equally to this article, they are the co-first authors of this manuscript; Wang H, Du YS, and Tang WL organized the data; Wang Y and Wu D performed the validation; Wang H, Bai ZJ, and Zeng HL performed the methodology; Tao J supervised; He J performed project management; Tao J and He J contributed equally to this article, they are the co-corresponding authors of this manuscript; and all authors thoroughly reviewed and endorsed the final manuscript.
Supported by the Science and Technology Development Fund of Nanjing Medical University, No. NMUB20230037; and the Youth Scientific Research Nurturing Fund of Jiangbei Campus of Zhongda Hospital Affiliated with Southeast University, No. JB2024Q01.
Institutional review board statement: This study was approved by the Medical Ethics Committee of The Fourth Affiliated Hospital of Nanjing Medical University, approval No. 20240628-K077.
Informed consent statement: Due to the retrospective nature of the study, requirements for informed consent were waived.
Conflict-of-interest statement: All the authors report no relevant conflicts of interest for this article.
Data sharing statement: The data utilized and/or analyzed during the present study are available from the corresponding author upon reasonable request.
Open Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: https://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Jian He, MD, PhD, Associate Professor, Chief Physician, Department of Nuclear Medicine, Nanjing Drum Tower Hospital, The Affiliated Hospital of Medicine School, Nanjing University, No. 321 Zhongshan Road, Nanjing 210008, Jiangsu Province, China. hjxueren@126.com
Received: March 12, 2025 Revised: April 17, 2025 Accepted: May 21, 2025 Published online: June 28, 2025 Processing time: 106 Days and 20.8 Hours
Abstract
BACKGROUND
Accurate preoperative differentiation of benign and malignant thyroid nodules is critical for optimal patient management. However, conventional imaging modalities present inherent diagnostic limitations.
AIM
To develop a non-contrast computed tomography-based machine learning model integrating radiomics and clinical features for preoperative thyroid nodule classification.
METHODS
This multicenter retrospective study enrolled 272 patients with thyroid nodules (376 thyroid lobes) from center A (May 2021-April 2024), using histopathological findings as the reference standard. The dataset was stratified into a training cohort (264 lobes) and an internal validation cohort (112 lobes). Additional prospective temporal (97 lobes, May-August 2024, center A) and external multicenter (81 lobes, center B) test cohorts were incorporated to enhance generalizability. Thyroid lobes were segmented along the isthmus midline, with segmentation reliability confirmed by an intraclass correlation coefficient (≥ 0.80). Radiomics feature extraction was performed using Pearson correlation analysis followed by least absolute shrinkage and selection operator regression with 10-fold cross-validation. Seven machine learning algorithms were systematically evaluated, with model performance quantified through the area under the receiver operating characteristic curve (AUC), Brier score, decision curve analysis, and DeLong test for comparison with radiologists interpretations. Model interpretability was elucidated using SHapley Additive exPlanations (SHAP).
RESULTS
The extreme gradient boosting model demonstrated robust diagnostic performance across all datasets, achieving AUCs of 0.899 [95% confidence interval (CI): 0.845-0.932] in the training cohort, 0.803 (95%CI: 0.715-0.890) in internal validation, 0.855 (95%CI: 0.775-0.935) in temporal testing, and 0.802 (95%CI: 0.664-0.939) in external testing. These results were significantly superior to radiologists assessments (AUCs: 0.596, 0.529, 0.558, and 0.538, respectively; P < 0.001 by DeLong test). SHAP analysis identified radiomic score, age, tumor size stratification, calcification status, and cystic components as key predictive features. The model exhibited excellent calibration (Brier scores: 0.125-0.144) and provided significant clinical net benefit at decision thresholds exceeding 20%, as evidenced by decision curve analysis.
CONCLUSION
The non-contrast computed tomography-based radiomics-clinical fusion model enables robust preoperative thyroid nodule classification, with SHAP-driven interpretability enhancing its clinical applicability for personalized decision-making.
Core Tip: This study introduces a novel non-contrast computed tomography-based machine learning model integrating radiomics and clinical features with lobe segmentation for preoperative differentiation of benign and malignant thyroid nodules. Leveraging dual-center data and thyroid lobe segmentation, the extreme gradient boosting model demonstrated superior diagnostic accuracy and stability across diverse cohorts, outperforming traditional radiologist assessments. Key predictors, including radiomic score, age, and tumor size group, calcify and cystic, were showed through SHAP analysis, enhancing model interpretability. The approach offers a robust, non-invasive tool for personalized preoperative decision-making, with the potential to improve clinical management of thyroid nodules.
Citation: Wang H, Wang X, Du YS, Wang Y, Bai ZJ, Wu D, Tang WL, Zeng HL, Tao J, He J. Non-contrast computed tomography radiomics model to predict benign and malignant thyroid nodules with lobe segmentation: A dual-center study. World J Radiol 2025; 17(6): 106682
Thyroid nodules, the most prevalent endocrine tumors, have been ranked as the third most common malignancy in thyroid cancer according to the 2022 China Cancer Burden Report[1]. Among these, papillary thyroid carcinoma (PTC) represents the predominant malignant subtype, accounting for 85%-90% of all thyroid malignancies[2]. Although PTC generally exhibits a favorable prognosis, regional lymph node metastasis occurs in 30%-80% of patients, complicating diagnosis and increasing the risk of adverse outcomes[3,4]. Clinical management strategies vary significantly depending on malignancy status: Benign nodules are typically monitored conservatively, whereas malignant cases necessitate radical surgery, often supplemented by radioactive iodine therapy and targeted treatments[5]. This highlights the critical need for early and accurate differentiation between benign and malignant nodules to optimize therapeutic strategies and improve patient prognosis[6].
The National Comprehensive Cancer Network guidelines emphasize the essential role of imaging in thyroid nodule evaluation[7]. Ultrasound (US) is the primary diagnostic modality due to its noninvasive nature, absence of ionizing radiation, and ease of operation[8]. However, its diagnostic accuracy remains suboptimal (60%-70%) due to spatial resolution limitations and operator-dependent variability, particularly in detecting microcalcifications and occult lesions[9]. While US-guided fine-needle aspiration is the diagnostic gold standard, its invasive nature poses risks of bleeding and infection. Additionally, the American Thyroid Association discourages fine-needle aspiration for nodules < 1 cm unless local infiltration or lymph node metastasis is suspected[10]. Contrast-enhanced computed tomography (CECT) offers superior spatial resolution and 3D reconstruction for preoperative assessment of tumor invasiveness, but its clinical utility is constrained by radiation exposure, iodine contrast risks, and limited quantitative analytical capacity[11]. These limitations underscore the need for a more accessible, low-risk diagnostic approach with improved efficacy.
Recent advancements in artificial intelligence have revolutionized medical imaging analysis, with machine learning algorithms demonstrating significant potential in complex image classification by leveraging large-scale data training, thereby enhancing diagnostic accuracy and consistency[12]. The field of radiomics, first introduced in 2012, enables noninvasive tumor characterization through high-throughput extraction of imaging features, transforming visual data into quantifiable metrics[13-15]. Current radiomics research on thyroid nodules has predominantly focused on US and CECT, while non-contrast computed tomography (NCCT) remains underutilized[16,17]. NCCT presents several advantages for thyroid nodule assessment. Unlike CECT, it eliminates the need for iodine-based contrast agents while delivering lower radiation exposure. Additionally, NCCT provides sufficient morphological detail of thyroid nodules, particularly in characterizing calcification patterns - a critical feature for differentiating benign from malignant lesions. Moreover, many studies rely on single-nodule segmentation, which may introduce bias due to manual lesion localization and margin delineation in multinodular cases[18]. This study aims to develop an NCCT-based radiomics machine learning model utilizing thyroid lobe segmentation and to evaluate its clinical utility in thyroid nodule diagnosis, providing a low-risk, efficient diagnostic tool for early detection.
MATERIALS AND METHODS
Patient’s selection
This retrospective study was conducted in accordance with the Declaration of Helsinki and received formal approval from the Institutional Review Board of Nanjing Medical University's Fourth Affiliated Hospital, approval No. 20240628-K077). The ethics committee granted an exemption from informed consent requirements, as per institutional policies for retrospective data analysis. The study population comprised patients who underwent thyroid resection between May 2021 and April 2024. Eligible patients met the following inclusion criteria: (1) Histopathological confirmation of thyroid nodules, with malignant cases classified as papillary carcinoma and benign cases as adenoma/nodular goiter; (2) Preoperative neck computed tomography (CT) performed within 15 days prior to surgery; and (3) Thyroid nodules with a maximum diameter of ≥ 2 mm. Exclusion criteria included: (1) Diffuse thyroid pathology (Hashimoto’s or granulomatous thyroiditis; n = 8); (2) Nodules < 2 mm (n = 21); (3) Non-diagnostic image quality (n = 19); (4) Nodules located in the isthmus or pyramidal lobe (n = 4); (5) History of malignancy or prior radiotherapy (n = 12); and (6) Age < 18 years (n = 1). After excluding 65 ineligible patients, 376 thyroid lobes from 272 patients (104 bilateral cases) were included and stratified into training and validation cohorts (7:3 ratio) using the thyroid isthmus midline as the anatomical boundary. Additionally, the study incorporated a temporal test cohort (May-August 2024; 97 lobes from 75 patients) and an external test cohort (Zhongda Hospital; January-June 2024; 81 lobes from 75 patients) for independent evaluation. The study adhered to the CLEAR checklist[19], with patient selection workflows illustrated in Figures 1 and 2.
Figure 2 Workflow of model development.
CT: Computed tomography; PTC: Papillary thyroid carcinoma; TA: Thyroid adenoma; NG: Nodular goiter.
Imaging acquisition
Patients enrolled in the training, validation, and temporal test cohorts received both NCCT and CECT examinations through the IQon Spectral CT (Philips Medical Systems, Netherlands). The imaging protocol encompassed anatomical regions extending from the skull base through the superior mediastinal compartment. Subjects were positioned in a standardized supine orientation with cervical extension and shoulder depression to reduction of cervical sclerotic bundle artifacts while maintaining normal steadily breathing throughout image acquisition. Acquisition parameters included a 120 kVp tube voltage, DoseRight auto-adjusting tube current (mean 145 mAs, Index 23), 64 × 0.625 mm collimation, 250 mm field of view, pitch 0.969, matrix 512 × 512, iDose4 iterative reconstruction, and 1 mm isotropic resolution (350/60 window settings). CECT involved dual-phase imaging at 25 seconds (arterial) and 60 seconds (venous) post-iohexol injection (320 mgI/mL, 3 mL/s), with Digital Imaging and Communications in Medicine data archived in the Picture Achieving and Communication System. The external test cohort underwent imaging on a Revolution CT scanner (GE Healthcare, United states) using distinct acquisition parameters: 120 kVp tube voltage, automated tube current modulation, 256 mm × 0.625 mm collimation, 250 mm field of view, pitch 0.992, matrix 512 × 512, Adaptive Statistical Iterative Reconstruction-VEO (50%), and 0.625 mm slice thickness (350/50 window settings). Notably, contrast-phase data were excluded from all analyses to maintain NCCT specificity.
Pathological results, imaging features, and clinical variable collection
Histopathological assessment was performed on formalin-fixed, paraffin-embedded surgical specimens obtained via total or hemithyroidectomy. Systematic sampling was conducted, with representative sections harvested from each nodule - particularly targeting regions exhibiting suspicious calcifications or cytological atypia. All specimens underwent blinded evaluation by two fellowship-trained thyroid pathologists (with 15 years and 10 years of subspecialty experience, respectively). Malignant lesions were histologically confirmed as PTC, while benign pathologies were classified as follicular adenomas or nodular goiters. radiologist A (junior, 15 years’ experience) and radiologist B (senior, 25 years’ experience) independently evaluated CT imaging features through Picture Achieving and Communication System workstations, including tumor size group [small (≤ 5 mm), intermediate (5-10 mm), large (> 10 mm)], multiple nodules, calcify, and cystic. Interobserver discrepancies (18 cases, 3.25%) primarily involved sub-centimeter nodule measurements (6 cases) and microcalcification detection (12 cases) and were adjudicated by a third radiologist (25 years of experience). Calcify and cystic exhibited perfect inter-rater agreement due to standardized diagnostic criteria. Demographic and biochemical parameters - including age, sex, body mass index, free triiodothyronine, free thyroxine, thyroid-stimulating hormone, thyroglobulin antibody, and thyroid peroxidase antibody - were extracted from the hospital information system.
For human-machine comparative analysis, both radiologists performed blinded NCCT evaluations on the training and validation cohorts following a one-month washout period and standardized diagnostic training. The assessment protocol restricted access to clinical data, allowing only laterality information while concealing patient identifiers, cohort allocation, and prior interpretations. Malignancy classification was based on the following criteria: (1) Irregular nodule morphology; (2) Extracapsular invasion or infiltration into adjacent tissues; (3) Lower nodule density relative to surrounding thyroid parenchyma; (4) Gravelly microcalcifications within the lesion; and (5) Presence of metastatic cervical lymph nodes[20].
Lobe thyroid tissue segmentation
Segmentation was conducted using 3D Slicer 5.7.0 (https://www.slicer.org) with isotropic voxel resampling (0.5 mm × 0.5 mm × 1 mm) to enhance spatial resolution for small thyroid structures. A radiographer with 14 years of CT imaging expertise performed segmentation of the thyroid lobe region of interest on axial CT slices, guided by pathological diagnosis results. The segmentation adhered to anatomical boundaries defined by the midline of the thyroid isthmus. Radiologist A subsequently conducted a meticulous layer-by-layer review to verify delineation accuracy.
Radiomics feature extraction and consistency validation
The radiomics extraction process was conducted utilizing the PyRadiomics toolkit (https://github.com/Radiomics/pyradiomics), which resulted in the generation of 1130 features. These features encompassed 14 shape descriptors, 18 first-order histogram-based indices, and 75 textural characteristics derived through grey-level matrix transformations (including GLCM, GLDM, GLRLM, GLSZM, and NGTDM analyses). In addition, 279 Log-filtered features and 744 features obtained through wavelet decomposition were identified. To assess segmentation reproducibility, 40 randomly selected thyroid lobes underwent duplicate manual segmentation by radiologist A, followed by an independent segmentation by radiographer A one week later. The intraclass correlation coefficient (ICC) quantified feature consistency across intra- and inter-observer assessments, with features demonstrating ICC ≥ 0.8 retained to ensure robustness against segmentation variability.
Feature engineering and establishment of radiomics score
All radiomic features were standardized using Z-score normalization [Z = (X - μ)/σ, where X represents the feature value, μ the mean, and σ the standard deviation] to mitigate scale disparities. To counteract class imbalance in the training cohort, where benign samples were underrepresented, synthetic minority oversampling was applied at a 1:1.3 ratio to equalize benign and malignant cases. Feature selection involved Pearson correlation analysis, retaining a single representative feature per cluster exhibiting r > 0.7 to minimize multicollinearity. Least absolute shrinkage and selection operator (LASSO) regression with 10-fold cross-validation determined the optimal regularization parameter (λ min) and identified non-zero coefficient features, culminating in the construction of the radiomic score (RadScore) through linear combination: RadScore = Σ (feature_weight × feature_value) + intercept.
Machine learning model construction and evaluation
Seven machine learning models were constructed based on differential analysis of clinical features in the training cohort, integrating statistically significant variables alongside RadScore. These models included decision trees, random forests (RF), logistic regression, support vector machines (SVM), extreme gradient boosting (XGB), K-nearest neighbors, and light gradient boosting machines. Model performance was evaluated by comparing the area under the receiver operating characteristic curve (AUC), with the optimal algorithm determined by the highest AUC values and further visualized through confusion matrices. Calibration curves assessed predictive accuracy, while decision curve analysis quantified clinical net benefit across probability thresholds. Feature importance was interpreted using SHapley Additive exPlanations (SHAP) values to determine variable contributions to model predictions.
Statistical analysis
All analyses were conducted in R software (v4.2.1), utilizing the glmnet package for LASSO regression and the rms package for nomogram construction and calibration curve generation. The normality of data distribution was assessed via the Kolmogorov-Smirnov test, with non-normally distributed data expressed as median (interquartile range) and compared using the Mann-Whitney U test. Categorical variables were presented as frequencies (percentages) and analyzed via the χ2 test. The performance metrics of the model included AUC, accuracy, sensitivity, specificity, positive and negative predictive values (positive predictive value/negative predictive value), precision, recall, F1 score, and Brier score. AUC comparisons across models were conducted using DeLong's test, with statistical significance defined as P < 0.05.
RESULTS
Baseline characteristics
The retrospective cohort comprised 272 patients [60 males (22.1%), 212 females (77.9%); mean age, 48.2 ± 13.7 years] with 376 thyroid lobes, including 104 bilateral cases. Histopathological evaluation identified 270 PTC and 106 benign lobes. Patients were randomly allocated into a training cohort (271 lobes) and a validation cohort (105 lobes), with no significant baseline differences between groups (all P > 0.05). The temporal test cohort included 75 patients (97 lobes, 22 bilateral), while the external test cohort consisted of 75 patients (81 lobes, 6 bilateral) from an independent institution (Table 1). Univariate analysis identified four malignancy-associated variables: Patient age (P = 0.016), nodule size category (P < 0.001), calcify patterns (P = 0.003), and cystic (P = 0.012). Comparative statistics between benign and malignant characteristics are detailed in Table 2.
Table 1 Multi-cohort comparison of thyroid lobes characteristics, n (%).
Table 2 Comparison between the benign and malignant thyroid lobes in the training cohort, n (%).
Characteristics
Benign (n = 80)
Malignant (n = 184)
P value
Gender
0.234
Female
68 (85)
143 (78)
Male
12 (15)
41 (22)
Age (years), median (P25-P75)
54.50 (49.00-64.25)
49.00 (35.00-57.25)
< 0.001
BMI, median (P25-P75)
24.16 (21.93-25.98)
24.80 (22.19-26.35)
0.296
Multiple nodules
0.716
No
43 (54)
105 (57)
Yes
37 (46)
79 (43)
Tumor size group
< 0.001
Small (≤ 5 mm)
4 (5)
39 (21)
Medium (5-10 mm)
21 (26)
78 (42)
Large (> 10 mm)
55 (69)
67 (36)
Calcify
0.028
No
66 (82)
126 (68)
Yes
14 (18)
58 (32)
Cystic
< 0.001
No
57 (71)
172 (93)
Yes
23 (29)
12 (7)
FT3 (pmol/L), median (P25-P75)
4.76 (4.43-5.01)
4.76 (4.39-5.14)
0.421
FT4 (pmol/L), median (P25-P75)
16.20 (14.88-16.92)
16.20 (14.95-18.05)
0.142
TSH (mU/L), median (P25-P75)
1.73 (1.40-2.03)
1.73 (1.37-2.63)
0.227
TGAb (IU/mL), median (P25-P75)
16.70 (15.62-18.05)
16.70 (15.4-25.85)
0.164
TPOAb (IU/mL), median (P25-P75)
13.00 (12.80-13.82)
13.00 (12.35-13.00)
0.671
Feature screening and radscore calculation
Radiomic feature extraction using PyRadiomics yielded 1130 features per thyroid lobe, subjected to a multi-stage refinement process. Inter- and intra-observer reliability analysis (ICC ≥ 0.8) retained 612 stable features, followed by Pearson correlation filtering (r ≥ 0.7), which removed 51 redundant parameters. LASSO regression with ten-fold cross-validation (optimal λ = 0.01236625) identified 27 non-zero discriminative predictors. These features were linearly combined using LASSO-derived coefficients to generate the RadScore, which demonstrated significant diagnostic stratification between benign and malignant lesions, as detailed in Table 3 and illustrated through feature selection dynamics in Figures 3 and 4.
Figure 3 Least absolute shrinkage and selection operator feature selection and comparison.
A: Least absolute shrinkage and selection operator feature selection plot, where the horizontal axis represents the logarithm of the regularization parameter (λ), and the vertical axis denotes the corresponding feature weights. As λ increases, feature weights progressively decrease until exclusion at zero; B: 10-fold cross-validation curve, where the vertical axis represents the cross-validated mean squared error. The λ-value at the lowest point (λ min) is identified as the optimal regularization parameter, balancing model complexity and prediction error; C: 27 radiomic features and their corresponding weight coefficients retained after least absolute shrinkage and selection operator regression; D: Correlation heatmap of the 27 selected radiomic features, with darker colors indicating stronger correlations.
Figure 4 Radiomic score distribution across cohorts.
A: Comparison of radiomic score (Radscore) between benign and malignant lesions in the training; B: Comparison of Radscore between benign and malignant lesions in the validation; C: Comparison of Radscore between benign and malignant lesions in the temporal test; D: Comparison of Radscore between benign and malignant lesions in the external test cohorts. aP < 0.05; bP < 0.001.
Table 3 Weights of least absolute shrinkage and selection operator selected features and training set Z-score parameters.
Five clinically significant predictors - RadScore, age, tumor size group, calcify pattern, and cystic - were integrated into seven machine learning algorithms. Comprehensive evaluation across multiple validation metrics identified the XGB model as the optimal classifier, demonstrating superior discriminative performance. The XGB model achieved an AUC of 0.889 [95% confidence interval (CI): 0.845-0.932] in the training cohort, 0.803 (95%CI: 0.715-0.890) in the validation cohort, 0.855 (95%CI: 0.775-0.935) in the temporal test, and 0.802 (95%CI: 0.664-0.939) in the external test. Robust accuracy (0.696-0.845) and F1 scores (0.790-0.852) were maintained across all datasets, model performance across cohorts is summarized in Table 4. The model exhibited high calibration fidelity (Brier scores: 0.121-0.144) and substantial clinical utility, as indicated by decision curve analysis with net benefit thresholds exceeding 20% probability (Figure 5). These results collectively validate the XGB model’s generalizability and reliability in thyroid nodule characterization.
Figure 5 Performance evaluation of seven machine learning models.
A: Receiver operating characteristic curves of different models across the training, validation, temporal test, and external test cohorts (from left to right). The area under the receiver operating characteristic curve quantifies classification performance, with higher area under the receiver operating characteristic curve values indicating superior discrimination; B: Calibration curves for different models in the training, validation, temporal test, and external test cohorts (from left to right). These curves assess the agreement between predicted probabilities and actual outcomes, with lower Brier scores indicating better calibration, as reflected by proximity to the 45° diagonal line; C: Decision curve analysis in the training, validation, temporal test, and external test cohorts (from left to right), evaluating the clinical net benefit of different models across probability thresholds. Overall, the extreme gradient boosting model exhibits consistent and superior performance across multiple cohorts. DT: Decision trees; KNM: K-nearest neighbors; LR: Logistic regression; RF: Random forests; LGBM: Light gradient boosting machines; SVM: Support vector machines; XGB: Extreme gradient boosting; AUC: Area under the receiver operating characteristic curve.
Table 4 Multi-cohort predictive performance of various models.
Temporal test cohort (benign = 26, malignant = 71)
LR
0.814 (0.717-0.912)
0.825
0.930
0.538
0.846
0.737
0.846
0.930
0.886
0.137
DT
0.757 (0.652-0.851)
0.742
0.780
0.533
0.901
0.308
0.901
0.780
0.837
0.178
RF
0.795 (0.696-0.894)
0.763
0.901
0.385
0.800
0.588
0.800
0.901
0.848
0.152
XGB
0.855 (0.775-0.935)
0.773
0.831
0.615
0.855
0.571
0.855
0.831
0.843
0.139
SVM
0.816 (0.719-0.913)
0.845
0.972
0.500
0.841
0.867
0.841
0.972
0.902
0.141
KNN
0.719 (0.607-0.832)
0.711
0.803
0.462
0.803
0.462
0.803
0.803
0.803
0.208
LGBM
0.800 (0.697-0.904)
0.835
0.873
0.731
0.899
0.679
0.899
0.873
0.886
0.193
External test cohort (benign = 15, malignant = 66)
LR
0.782 (0.607-0.907)
0.765
0.848
0.400
0.862
0.375
0.862
0.848
0.855
0.127
DT
0.715 (0.566-0.863)
0.778
0.887
0.421
0.833
0.533
0.833
0.887
0.859
0.166
RF
0.751 (0.589-0.913)
0.815
0.879
0.533
0.892
0.500
0.892
0.879
0.885
0.143
XGB
0.802 (0.644-0.939)
0.728
0.727
0.733
0.923
0.379
0.923
0.727
0.814
0.121
SVM
0.800 (0.674-0.926)
0.815
0.909
0.400
0.870
0.500
0.870
0.909
0.889
0.120
KNN
0.813 (0.697-0.928)
0.803
0.848
0.600
0.903
0.474
0.903
0.848
0.875
0.158
LGBM
0.728 (0.668-0.789)
0.753
0.848
0.333
0.848
0.333
0.848
0.848
0.848
0.163
Model performance and interpretability
In the validation cohort, the XGB model outperformed both human radiologists (P < 0.001) and other machine learning models, as confirmed by the DeLong test (decision trees: P < 0.05; RF: P < 0.01), as detailed in Table 5. SHAP analysis ranked RadScore as the most influential predictor, followed by age, tumor size group, calcify, and cystic. This interpretability framework was further corroborated through representative case illustrations (Figure 6), highlighting the distinct predictive contributions of each feature in one benign and one malignant case.
Figure 6 SHapley Additive exPlanations analysis and clinical application.
A: Honeycomb and bar plot ranking SHapley Additive exPlanations values by feature importance; B: Case illustration of a 67-year-old female with an 8 mm right thyroid lobe nodule, pathologically confirmed as nodular goiter; C: Case illustration of a 43-year-old female with a 4 mm right thyroid lobe nodule, pathologically confirmed as papillary thyroid carcinoma. SHAP: SHapley Additive exPlanations.
Table 5 DeLong test for different models and human radiologists in the validation cohort.
Models
DT
LR
RF
XGB
SVM
KNN
LGBM
Senior radiologist
Junior radiologist
DT
-
0.011
0.118
< 0.001
< 0.001
0.022
0.309
0.235
0.179
LR
0.011
-
0.137
0.615
0.833
0.537
0.093
< 0.001
< 0.001
RF
0.118
0.137
-
0.038
0.016
0.100
0.950
0.031
0.017
XGB
< 0.001
0.615
0.038
-
0.469
0.825
0.239
< 0.001
< 0.001
SVM
< 0.001
0.833
0.016
0.469
-
0.535
0.159
< 0.001
< 0.001
KNN
0.022
0.537
0.100
0.825
0.535
-
0.321
0.002
< 0.001
LGBM
0.309
0.093
0.950
0.239
0.159
0.321
-
0.017
0.008
Senior radiologist
0.235
< 0.001
0.031
< 0.001
< 0.001
0.002
0.017
-
0.810
Junior radiologist
0.179
< 0.001
0.017
< 0.001
< 0.001
< 0.001
0.008
0.810
-
DISCUSSION
This retrospective study developed a machine learning model integrating NCCT radiomics and clinical features for thyroid nodule malignancy prediction. The XGB algorithm exhibited superior stability and generalizability across all datasets. Leveraging NCCT-derived radiomic signatures, the model enables noninvasive differentiation of thyroid nodules, presenting a novel framework for precision diagnostics. SHAP-based interpretability further enhances its clinical applicability by providing transparent decision-support insights.
PTC, the most prevalent thyroid malignancy, is characterized by BRAFV600E mutations and rearranged during transfection/PTC rearrangements, both of which drive tumor progression via sustained mitogen activated protein kinase/extracellular signal regulated kinase pathway activation[21]. In contrast, benign nodules typically maintain follicular architecture, exhibit homogeneous colloid distribution, and lack infiltrative margins[22]. Conventional imaging modalities, including US, which detects microcalcifications with 91% specificity, and CT, which assesses capsular integrity with 85% sensitivity, remain limited in their capacity to quantify tumor microenvironment heterogeneity or molecular features[23]. Radiomics, a high-throughput analytical approach, addresses these limitations by facilitating machine learning-driven differentiation of malignant phenotypes[24]. Feature analysis revealed multidimensional distinctions: Shape irregularity correlated with invasive growth patterns[25], first-order statistics reflected grayscale dispersion associated with cellular density heterogeneity[26], and texture features captured microstructural disorganization characteristic of malignancy[27]. Additionally, Log and wavelet transformations enhanced sensitivity to microcalcifications and angiogenic patterns[28]. Beyond diagnostic applications, these imaging biomarkers provide mechanistic insights into thyroid carcinogenesis and may inform targeted therapeutic strategies[29].
Our XGB model demonstrated robust and consistent diagnostic performance across all evaluation cohorts, achieving AUC values of 0.899 (95%CI: 0.845-0.932) in training, 0.803 (95%CI: 0.715-0.890) in validation, 0.855 (95%CI: 0.775-0.935) in temporal testing, and 0.802 (95%CI: 0.664-0.939) in external testing, all significantly superior to radiologists assessments (AUC range: 0.529-0.596; P < 0.001 by DeLong test). These results corroborate previous CT-based radiomics studies including Kong et al[30] (AUC = 0.84 using arterial-phase CT) and Lin et al[31] (AUC = 0.92 with multiphase-enhanced CT), while introducing the novel application of NCCT-derived RadScore that leverages high spatial resolution to capture comprehensive 3D morphologic and textural features without requiring hemodynamic data[32]. Compared to CECT, our NCCT-based approach provides distinct clinical advantages including significantly reduced radiation exposure and elimination of iodinated contrast-associated risks such as allergic reactions and nephrotoxicity[33,34]. The implementation of semilobar segmentation further enhances model robustness by facilitating complete volumetric assessment of thyroid lobes while minimizing potential biases from multinodular localization and segmentation variability[35]. We observed that certain machine learning algorithms (e.g., RF and K-nearest neighbors) were prone to overfitting, as evidenced by near-perfect training performance (AUC = 1.000) but poor generalization, which we mitigated through a rigorous multi-tiered validation strategy incorporating independent validation, prospective temporal testing, and multicenter external validation cohorts to ensure the selected XGB model maintains strong performance across diverse clinical settings - a critical prerequisite for successful clinical translation.
Several key clinical predictors emerged from the analysis. Malignant nodules were associated with younger patients (median age: 49.0 vs 54.5 years, P < 0.001), consistent with established thyroid cancer epidemiology[36]. The predominance of malignancy in nodules ≤ 10 mm (63% vs 31% > 10 mm in the benign group) may reflect the high prevalence of BRAFV600E mutations in microcarcinomas[37]. Calcify were significantly more frequent in malignant lesions (32% vs 18%, P = 0.028), aligning with the presence of psammoma bodies in PTC[38], while cystic was more common in benign nodules (29% vs 7%, P < 0.001), a pattern indicative of follicular adenoma degeneration[39]. These findings corroborate established sonographic malignancy markers, emphasizing the value of multimodal characterization.
Several important limitations should be acknowledged in this study. Despite employing standardized pre-processing procedures including image resampling, inherent biases persist in the external test set due to variations in CT scanner manufacturers and acquisition protocols across institutions. While synthetic minority oversampling was implemented to address class imbalance in the training cohort, its effectiveness in improving benign nodule prediction may be constrained by the original limited sample size of benign cases, necessitating future validation with expanded benign cohorts. Although we comprehensively evaluated multiple machine learning algorithms, the maximum achieved AUC below 0.9 indicates potential for improvement through larger, more diverse datasets. The manual segmentation approach, while ensuring consistency through rigorous inter-reader ICC verification, remains subject to inherent observer variability and represents a time-intensive process that may hinder clinical scalability. Future research directions will prioritize the development of automated segmentation pipelines leveraging U-Net architectures, coupled with the integration of advanced deep learning models (ResNet, DenseNet, Vision Transformer) within multicenter validation studies to enhance both predictive accuracy and clinical applicability while maintaining rigorous methodological standards.
CONCLUSION
In conclusion, this study developed an NCCT-based XGB model leveraging thyroid lobe segmentation for noninvasive differentiation of benign and malignant thyroid nodules. When coupled with SHAP interpretability, this model holds promise for clinical decision support in preoperative evaluation.
Footnotes
Provenance and peer review: Unsolicited article; Externally peer reviewed.
Peer-review model: Single blind
Specialty type: Radiology, nuclear medicine and medical imaging
Country of origin: China
Peer-review report’s classification
Scientific Quality: Grade A, Grade A, Grade B, Grade B, Grade C
Novelty: Grade A, Grade B, Grade B, Grade B, Grade C
Creativity or Innovation: Grade B, Grade B, Grade B, Grade B, Grade C
Scientific Significance: Grade A, Grade B, Grade B, Grade B, Grade C
P-Reviewer: Liu Y; Tang W; Zeng JQ S-Editor: Bai Y L-Editor: A P-Editor: Wang WB
Li X, Jian J, Zhang A, Xiang JM, Huang J, Chen Y. The role of immune cells and immune related genes in the tumor microenvironment of papillary thyroid cancer and their significance for immunotherapy.Sci Rep. 2024;14:18125.
[RCA] [PubMed] [DOI] [Full Text][Cited by in RCA: 5][Reference Citation Analysis (0)]
Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, Pacini F, Randolph GW, Sawka AM, Schlumberger M, Schuff KG, Sherman SI, Sosa JA, Steward DL, Tuttle RM, Wartofsky L. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer.Thyroid. 2016;26:1-133.
[RCA] [PubMed] [DOI] [Full Text][Cited by in Crossref: 10769][Cited by in RCA: 9516][Article Influence: 1057.3][Reference Citation Analysis (1)]
Kocak B, Baessler B, Bakas S, Cuocolo R, Fedorov A, Maier-Hein L, Mercaldo N, Müller H, Orlhac F, Pinto Dos Santos D, Stanzione A, Ugga L, Zwanenburg A. CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII.Insights Imaging. 2023;14:75.
[RCA] [PubMed] [DOI] [Full Text][Cited by in Crossref: 130][Cited by in RCA: 212][Article Influence: 106.0][Reference Citation Analysis (0)]
Ha EJ, Chung SR, Na DG, Ahn HS, Chung J, Lee JY, Park JS, Yoo RE, Baek JH, Baek SM, Cho SW, Choi YJ, Hahn SY, Jung SL, Kim JH, Kim SK, Kim SJ, Lee CY, Lee HK, Lee JH, Lee YH, Lim HK, Shin JH, Sim JS, Sung JY, Yoon JH, Choi M. 2021 Korean Thyroid Imaging Reporting and Data System and Imaging-Based Management of Thyroid Nodules: Korean Society of Thyroid Radiology Consensus Statement and Recommendations.Korean J Radiol. 2021;22:2094-2123.
[RCA] [PubMed] [DOI] [Full Text] [Full Text (PDF)][Cited by in Crossref: 14][Cited by in RCA: 154][Article Influence: 38.5][Reference Citation Analysis (0)]
Wentland AL, Yamashita R, Kino A, Pandit P, Shen L, Brooke Jeffrey R, Rubin D, Kamaya A. Differentiation of benign from malignant solid renal lesions using CT-based radiomics and machine learning: comparison with radiologist interpretation.Abdom Radiol (NY). 2023;48:642-648.
[RCA] [PubMed] [DOI] [Full Text][Cited by in RCA: 13][Reference Citation Analysis (0)]
Hamatani K, Eguchi H, Ito R, Mukai M, Takahashi K, Taga M, Imai K, Cologne J, Soda M, Arihiro K, Fujihara M, Abe K, Hayashi T, Nakashima M, Sekine I, Yasui W, Hayashi Y, Nakachi K. RET/PTC rearrangements preferentially occurred in papillary thyroid cancer among atomic bomb survivors exposed to high radiation dose.Cancer Res. 2008;68:7176-7182.
[RCA] [PubMed] [DOI] [Full Text][Cited by in Crossref: 112][Cited by in RCA: 117][Article Influence: 6.9][Reference Citation Analysis (0)]
