Predicting distant metastasis in nasopharyngeal carcinoma using gradient boosting tree model based on detailed magnetic resonance imaging reports

Abstract

BACKGROUND

Development of distant metastasis (DM) is a major concern during treatment of nasopharyngeal carcinoma (NPC). However, studies have demonstrated improved distant control and survival in patients with advanced NPC with the addition of chemotherapy to concomitant chemoradiotherapy. Therefore, precise prediction of metastasis in patients with NPC is crucial.

AIM

To develop a predictive model for metastasis in NPC using detailed magnetic resonance imaging (MRI) reports.

METHODS

This retrospective study included 792 patients with non-distant metastatic NPC. A total of 469 imaging variables were obtained from detailed MRI reports. Data were stratified and randomly split into training (50%) and testing sets. Gradient boosting tree (GBT) models were built and used to select variables for predicting DM. A full model comprising all variables and a reduced model with the top-five variables were built. Model performance was assessed by area under the curve (AUC).

RESULTS

Among the 792 patients, 94 developed DM during follow-up. The number of metastatic cervical nodes (30.9%), tumor invasion in the posterior half of the nasal cavity (9.7%), two sides of the pharyngeal recess (6.2%), tubal torus (3.3%), and single side of the parapharyngeal space (2.7%) were the top-five contributors for predicting DM, based on their relative importance in GBT models. The testing AUC of the full model was 0.75 (95% confidence interval [CI]: 0.69-0.82). The testing AUC of the reduced model was 0.75 (95%CI: 0.68-0.82). For the whole dataset, the full (AUC = 0.76, 95%CI: 0.72-0.82) and reduced models (AUC = 0.76, 95%CI: 0.71-0.81) outperformed the tumor node-staging system (AUC = 0.67, 95%CI: 0.61-0.73).

CONCLUSION

The GBT model outperformed the tumor node-staging system in predicting metastasis in NPC. The number of metastatic cervical nodes was identified as the principal contributing variable.

Key Words: Nasopharyngeal carcinoma, Distant metastasis, Machine learning, Detailed magnetic resonance imaging report, Gradient boosting tree model

Core Tip: A total of 469 imaging variables obtained from detailed magnetic resonance imaging (MRI) reports of 792 patients with nasopharyngeal carcinoma (NPC) with non-distant metastasis were evaluated in this retrospective study. Data were stratified and randomly split into training (50%) and testing (50%) sets. Gradient boosting tree (GBT) models were built based on the training set and used to select imaging variables to predict distant metastasis. The number of metastatic cervical nodes was the top contributor for predicting distant metastasis based on the relative importance in GBT models. The GBT model outperformed the tumor node-staging system in predicting metastasis in NPC.

INTRODUCTION

Nasopharyngeal carcinoma (NPC) is a head and neck malignancy that commonly affects the people of Southeast China[1]. Distant metastasis (DM) is a major cause of treatment failure in patients with NPC[2-4]. Recently, studies have demonstrated that the addition of chemotherapy to concomitant chemoradiotherapy can improve distant control and survival in patients with advanced NPC[5-8]. Hence, the precise prediction of metastasis in patients with NPC is of immense clinical value. Patients with lower chances of metastasis can opt for a personalized treatment plan aimed at reducing side-effects, pursuing better health, and lowering medical expenses. Conversely, patients who are more likely to develop metastasis can opt for an individualized regimen with greater focus on chemotherapy.

The American Joint Committee on Cancer tumor node (TN) staging system is the most widely used method to assess the risk of DM in patients with NPC. Magnetic resonance imaging (MRI)-based tumor and node characteristics are used to define the T and N classifications of NPC. However, the factors adopted by the TN-staging system encompass only a part of the MRI data, thus, some useful image data may be neglected. This may be the reason for the moderate predictive ability of TN-staging for DM in patients with NPC[9]. Detailed MRI reports can offer more substantial information about tumor invasion and lymph node metastasis than the TN-staging system and may be able to predict disease prognosis more accurately. However, they may also provide confounding information which may lead to low efficacy and pose difficulties if used in predictive models clinically. Consequently, statistical tools that can analyze large amounts of complex data with good predictive performance are required to address this issue.

The most extensively used method for clinical prediction is the generalized linear model[10]. However, when multiple highly correlated or non-normally distributed factors are included in these models, they fail to meet the assumptions of linearity and additivity, leading to unfavorable results. In contrast, machine learning methods incorporate larger numbers of predictors and can address the collinearity problem[11,12]. Thus, predictive models generated by machine learning methods might reduce the number of confounding variables in detailed MRI reports, increasing the efficacy of metastasis prediction. Among the several machine learning methods, the gradient boosting tree (GBT) model is particularly attractive[11]. It does not require factors to be normally distributed, can handle multiple variables simultaneously in a single model, and deals with complex collinearity and outliers flexibly. Moreover, this model can identify the chief contributors based on the relative importance of variables[13,14].

Therefore, this study aimed to build and validate a well-performed predictive model based on detailed MRI reports using the GBT model to examine the most contributing variables for NPC metastasis.

MATERIALS AND METHODS

Datasets

Between January 2010 and January 2013, a total of 3814 patients with NPC were hospitalized at the Sun Yat-sen University Cancer Center (Guangzhou, China), 792 of whom were included in this study. All patients had pathologically confirmed NPC and received intensity-modulated radiation therapy treatment. Patients with incomplete clinical data (n = 2973) or MRI data (n = 5), and those with DM at diagnosis (n = 24) or presence of other tumors (n = 20) were excluded from the study.

This study was approved by the Ethics Committee of the Sun Yat-sen University Cancer Center (No. SL-G2022-004-03).

MRI imaging protocol

All patients underwent pretreatment MRI with a 1.5-T or 3.0-T system. The scan ranged from the suprasellar cistern to the superior border of the thoracic cage. The detailed MRI sequences are shown in Supplementary material 1.

Imaging variables

All MRI images were independently assessed by three radiologists with 18 (Liu LZ), 12 (Cui CY), and 10 (Tian L) years of work experience. Any disagreements were resolved by consensus within 3 d. A detailed MRI report including large amounts of information regarding tumor invasion and lymph node metastasis for each patient was recorded, as shown in Supplementary material 2. A total of 469 imaging variables extracted from the detailed MRI reports were used to describe the size of the lesion and its relationship with the surrounding tissues, including tumors and lymph nodes.

Statistical analysis

We utilized several techniques for data preprocessing. The continuous variables were standardized and categorical variables were one-hot encoded. In total, 469 imaging variables were one-hot encoded and normally rescaled. We used a complete dataset without any missing records. Additionally, we adopted a well-established algorithm, the Synthetic Minority Over-sampling Technique (SMOTE), to address the data imbalance issue, as it can over-sample the minority and under-sample the majority. For variable selection, we used the method proposed by Deng et al[15]. Specifically, we stratified the dataset according to the metastasis outcome variable and randomly split the dataset into a training set (50%) and a testing set (50%). The training set was balanced with SMOTE and used for variable selection. Additionally, we generated two types of random variables, binary and continuous, which were included in the variable selection model. We then selected the variables with importance scores greater than those of random variables (as the threshold of random effect). The whole variable selection process was repeated 20 times, using different random seeds, to ensure the robustness of the analysis. In this case, the selected variables had greater average impacts than the average random effects (considered as statistically significant). Thereafter, the GBT model (full model) was rebuilt with the training set using all the selected variables and a reduced GBT model was constructed using the top-five selected variables. A five-fold cross validation together with a grid search was adopted to address the over-fitting problem and tune the models’ parameters, including the number of trees, maximum depth of trees, and learning rate. Model performance was assessed using area under the curve (AUC). The 95% confidence interval (CI) of AUC was assessed using the bootstrap method. The R software (version 3.0.2; R Foundation for Statistical Computing, Vienna, Austria) was used for analyses.

RESULTS

The demographic and clinicopathological characteristics of the 792 patients are listed in Table 1. The median patient age was 45 years (interquartile range, 38-53 years) and 576 (72.7%) patients were male, and 216 (27.3%) were female. Most of the patients had World Health Organization type III histological type (94.2%) of NPC. The tumor stage distribution for the entire series was as follows: 9.2% stage I, 22.1% stage II, 38.3% stage III, and 30.4% stage IV. The majority of the patients (86.5%) received neoadjuvant, concomitant, or adjuvant platinum-based chemotherapy.

Table 1 Demographic and clinicopathological characteristics of patients, n (%).

Variable	No. of patients
Median age (years)	45 (38-53)
Sex
Male	576 (72.7)
Female	216 (27.3)
WHO histologic type
I	5 (0.6)
II	41 (5.2)
III	746 (94.2)
T classification
T1	204 (25.8)
T2	97 (12.2)
T3	296 (37.4)
T4	195 (24.6)
N classification
N0	182 (23.0)
N1	438 (55.3)
N2	113 (14.3)
N3	59 (7.4)
AJCC stage
I	73 (9.2)
II	175 (22.1)
III	303 (38.3)
IV	241 (30.4)
Chemotherapy
No	107 (13.5)
Yes	685 (86.5)

WHO: World Health Organization; AJCC: American Joint Committee on Cancer.

After the end of intensity-modulated radiation therapy, follow-up was conducted every 3 mo in the first 2 years and every 6 mo for the next 5 years, or until death. The median follow-up was 62.1 mo (range, 1.4 to 83.4 mo). Among the 792 patients, 78 developed local and/or regional recurrence, 94 developed DM, and 87 died during the follow-up period. The 5-year overall survival and DM-free survival rates were 88.3% and 87.7%, respectively. The median overall survival and DM-free survival were not reached during follow-up.

Figure 1 shows the relative importance scores of the top-ten selected variables for predicting DM from the full predictive model. The number of metastatic cervical nodes contributed the maximum in predicting the occurrence of DM, accounting for 30.9% of the relative importance scores on the basis of GBT models. The next top-nine contributors were as follows: Tumor invasion in the posterior half of the nasal cavity (9.7%), two sides of the pharyngeal recess (6.2%), tubal torus (3.3%), single side of the parapharyngeal space (2.7%), carotid sheath (2.5%), chondroseptum (1.3%), prestyloid space (1.0%), posterior pharynx (0.7%), and cavernous sinus (0.5%).

Figure 1 Relative importance of top-ten variables based on the predictive full model. The full model included 23 variables. Only the top-ten variables with greater relative importance score are showed.

Figure 2A illustrates the performance of the full model with 23 selected variables and the reduced model with the top-five selected variables. The training AUC for the full model was 0.79 (95%CI: 0.72-0.86), which was similar to that of the reduced model (AUC = 0.78, 95%CI: 0.71-0.84). In addition, the testing AUC of the full model (AUC = 0.75, 95%CI: 0.69-0.82) was slightly lower than the training AUC but also very similar to that of the reduced model (AUC = 0.75, 95%CI: 0.68-0.82).

Figure 2 Performance comparison of the gradient boosting tree models and the tumor node-staging system. A: Performance assessment of the full model and reduced model. The full model included 23 selected variables. The reduced model only included the top-five variables. Area under the curve (AUC) is showed with 95% confidence interval (CI); B: Comparison of metastasis predictive performance among different predictive strategies in the whole dataset. Predictive strategies included using T-staging scores, N-stating scores, TN-staging scores, reduced model, and full model. The reduced model only included the top-five variables, while the full model included 23 selected variables. AUC is showed with 95%CI. AUC: Area under the curve.

For the whole dataset, a comparison of the DM predictive performance among the different predictive strategies is presented in Figure 2B. TN-staging scores (AUC = 0.67, 95%CI: 0.61-0.73) provided better prediction than using only T-staging scores (AUC = 0.63, 95%CI: 0.57-0.69) or N-staging scores (AUC = 0.64, 95%CI: 0.58-0.70). However, both the full model (AUC = 0.76, 95%CI: 0.72-0.82) and the reduced model (AUC = 0.76, 95%CI: 0.71-0.81) outperformed the TN-staging scores. Furthermore, the performance of the reduced model was nearly identical to that of the full model. Based on the receiver operating characteristic curve and Youden index, the sensitivity for the full model was 0.744 and specificity was 0.631. The sensitivity for the reduced model was 0.739 and specificity was 0.636.

DISCUSSION

In this study, we successfully built a predictive model, based on detailed MRI reports using GBT models, which outperformed the traditional TN-staging system in predicting DM in patients with NPC. The number of metastatic cervical nodes was identified as the most contributing factor in DM prediction.

MRI is routinely used for the detection, diagnosis, and assessment of the disease process in NPC. TN-staging of patients with NPC before treatment, which is the primary basis for clinical treatment planning, is usually based on MRI examination. However, regardless of the T or N stage, only a few MRI factors are selected for use in the staging system. According to previous studies, several other MRI factors may also be important for the prognosis of patients with NPC[16-19]. For example, extranodal extension[17] and the number of positive lymph nodes on MRI[18] were found to be independent factors for overall survival, DM-free survival, progression-free survival, and local recurrence-free survival in patients with NPC. This implies that some useful MRI data may be neglected by the TN-staging system. Conversely, in our study, detailed MRI reports with a total of 469 imaging variables offered extensive information about tumor invasion and lymph node metastasis. This is one of the reasons for the more efficient predictive performance of our model compared with that of the TN-staging system. In addition, the predictive performance of our model was greater than that of a previously reported model which used MR image features extracted by a three-dimensional convolutional neural network, both using GBT models (concordance indexes were 0.640 to 0.711 in different validation cohorts)[20].

The application of GBT models is another reason for the good performance of our model. Compared to the traditional statistical models, GBT models can efficiently deal with complex and non-linear correlations while maintaining a relatively high prediction accuracy[14]. The predictive performance of our model was comparable to that of another machine learning model using MRI-based tumor burden features and all clinical factors (concordance indexes were 0.766 and 0.760 in internal validation and external validation sets, respectively)[21]. Despite there being differences in demographics, study populations, and exposures between our study and the aforementioned study, it is evident that machine learning methods reduce the noise of detailed MRI reports and greatly increase the efficacy of prediction. Our model prefers sensitivity over specificity because of the severity of DM among patients with NPC. In another word, our model has a better ability to find a patient with potential DM, which conveys very important information for future treatment plans. The overfitting issue in machine learning models is well-known. To address this issue, we applied multiple approaches including splitting the datasets into training dataset (50%) and testing dataset (50%) and using grid search with 5-fold cross-validation for finding the best hyperparameters. From our results, we can conclude that our model did not have overfitting issues. Our reduced model with top-five variables performed as efficiently as the full model, which included all selected variables, enhancing the credibility of our results. Overall, our study provides a reference for predicting DM in patients with NPC.

In this study, we found that the number of metastatic cervical nodes played a substantial role in predicting DM in patients with NPC, with a much higher relative importance score than other variables in the GBT models. This finding is consistent with that of our previous study[18], wherein using a Cox model, we identified positive lymph node number as an independent risk factor for DM-free survival in patients with NPC, which was superior to other nodal factors. Thus, the present study confirmed the prognostic importance of metastatic nodal number in patients with NPC. The importance of metastatic nodal number may be explained as follows: In patients with NPC, LN metastases often follow an orderly manner from upper to lower level LNs[22]. This implies that the number of metastatic LNs is adequate to simultaneously reflect the location and laterality of positive LNs, and can better reflect the metastatic lymph node burden in patients. Currently, counting the number of metastatic lymph nodes on MRI can be a laborious task when a patient has many lymph nodes. However, the application of artificial intelligence in medical imaging will make counting of lymph nodes relatively easy. Moreover, we hope that the importance of metastatic nodal number and its potential clinical value, such as usage in selecting patients who may benefit from the additional chemotherapy, attract the attention of clinicians for further study.

Nonetheless, our study had some drawbacks. First, this was a retrospective study. Second, we did not control for some unknown confounding factors; however, we included 469 imaging variables, and some of these variables may be related to uncontrolled factors, which might have addressed this problem to some extent. Moreover, although our data were collected from only one hospital, we randomly split the dataset into training and testing sets. With the testing set as our external validation, our results can be applied to other populations to some extent. Third, we did not compare other machine-learning methods in this study. Instead, we used cross-validation and grid search to tune the parameters of the models, and the performance of our models was good. Further studies comparing other methods to this type of model are warranted in the future. Finally, we did not include any clinical data (e.g., age, sex, and EBV-DNA) in the model because we focused on the imaging variables obtained from the detailed MRI reports. Models combining imaging data and clinical data should be further studied in future, especially in large sample, multi-center, and prospective studies.

CONCLUSION

We have successfully built a predictive model for predicting DM in patients with NPC, which outperformed the traditional TN-staging system. The number of metastatic cervical nodes was found to be the most contributing factor.