Machine learning algorithms able to predict the prognosis of gastric cancer patients treated with immune checkpoint inhibitors

Abstract

BACKGROUND

Although immune checkpoint inhibitors (ICIs) have demonstrated significant survival benefits in some patients diagnosed with gastric cancer (GC), existing prognostic markers are not universally applicable to all patients with advanced GC.

AIM

To investigate biomarkers that predict prognosis in GC patients treated with ICIs and develop accurate predictive models.

METHODS

Data from 273 patients diagnosed with GC and distant metastasis, who un-derwent ≥ 1 cycle(s) of ICIs therapy were included in this study. Patients were randomly divided into training and test sets at a ratio of 7:3. Training set data were used to develop the machine learning models, and the test set was used to validate their predictive ability. Shapley additive explanations were used to provide insights into the best model.

RESULTS

Among the 273 patients with GC treated with ICIs in this study, 112 died within 1 year, and 129 progressed within the same timeframe. Five features related to overall survival and 4 related to progression-free survival were identified and used to construct eXtreme Gradient Boosting (XGBoost), logistic regression, and decision tree. After comprehensive evaluation, XGBoost demonstrated good accuracy in predicting overall survival and progression-free survival.

CONCLUSION

The XGBoost model aided in identifying patients with GC who were more likely to benefit from ICIs therapy. Patient nutritional status may, to some extent, reflect prognosis.

Key Words: Gastric cancer; Machine learning; Immune checkpoint inhibitors; Web-based calculator; Progression-free survival; Overall survival

Core Tip: This study identified predictive markers and developed machine learning models to assess the prognosis of patients with gastric cancer and treated with immune checkpoint inhibitors. Key findings highlighted the significance of peripheral blood markers such as platelet count/(lymphocyte count × serum prealbumin), prognostic nutrition index, and body mass index in predicting overall survival and progression-free survival. eXtreme Gradient Boosting was the most effective model for prediction and outperformed traditional methods. These insights underscore the potential of machine-learning algorithms in personalized medicine and emphasize the role of nutritional status in treatment outcomes of patients with gastric cancer.

INTRODUCTION

Gastric cancer (GC) is a major challenge for public health care systems worldwide[1]. Because most patients with GC are diagnosed at an older age and at a later stage, prognosis is not always favorable[2,3]. In China, the five-year survival rate of patients with advanced GC is approximately 40%, whereas that of patients with distant metastases is only approximately 10%[4]. Previous studies have demonstrated that immune checkpoint inhibitors (ICIs) are promising agents that can significantly inhibit tumor progression and prolong the survival of patients with GC and improve the current situation[5,6]. Accordingly, an increasing number of studies are focusing on outcomes, such as overall survival (OS) and progression-free survival (PFS), to determine whether patients with cancer can benefit from immunotherapy[7,8]. For patients with GC undergoing ICIs treatment, OS is typically defined as the period from the date the patient first received ICIs to the date the patient died from any cause, whereas PFS is usually defined as the period from the date the patient first received ICIs to the date of cancer progression or patient death[9].

In recent years, ICIs have been widely used to treat all types of cancers and have achieved good clinical results. ICIs can significantly reduce or completely eliminate late-stage tumors, and many patients with cancer experience prolonged survival[10-12]. Microsatellite instability status, programmed death-ligand 1 (PD-L1) expression, and tumor mutation burden are associated with immunotherapy benefits[13,14]. However, these markers do not seem to fully predict the response to immunotherapy. Some researchers have found that these markers can only be detected in a small number of patients with cancer[15,16]. Other studies have shown that patients treated with ICIs experience similar survival benefits regardless of their PD-L1 expression level[17]. Therefore, it is necessary to identify additional biomarkers and use them to build predictive models for determining which patients are more likely to benefit from immunotherapy.

Increasing evidence has demonstrated that the tumor immune microenvironment is involved in the occurrence and development of cancer and is closely associated with cancer prognosis[18,19]. In addition, as a malignant tumor of the digestive system, the nutritional status of patients with GC also significantly affected their survival time[20]. Although biomarkers based on tumor tissue can, to some extent, help identify patients who benefit more from ICIs therapy, tumor biopsies are invasive, and repeated invasive examinations are not conducive to patient health nor therapeutic effectiveness[21]. As such, more accessible peripheral blood markers appear to be suitable candidates for evaluating treatment benefits. Many inflammatory and nutritional markers have been proposed to predict the prognosis of patients with GC treated with ICIs[22]. We hope to verify the validity of these markers and mine some novel and practical markers as much as possible.

With scientific development, machine learning (ML) algorithms and nomogram have been widely used in the field of medicine to predict the occurrence and development of various diseases[23,24]. ML is a collection of statistical tools and methods suitable for dealing with datasets containing a large number of features and capturing complex, interactive, or nonlinear effects[25]. eXtreme Gradient Boosting (XGBoost) is a new type of ML algorithm with an optimized implementation of gradient boosting. It has the distinctive features of efficiently and flexibly processing missing data and assembling weak prediction models to build more accurate prediction models[26]. The logistic regression (LR) model is usually used to study the influence of characteristic variables on binary classification variables[27]. The decision tree (DT) model is simple to understand and explain[28]. In addition, the model can process both classified and numerical data[29]. Therefore, the ML algorithm may a potential tool for predicting the long-term prognosis of patients with cancer.

Accordingly, the present study aimed to screen potential markers for patients diagnosed with GC and treated with ICIs, use them to build interpretable ML models, and compare the effects of each marker on the OS and PFS. In addition, we built a nomogram to verify whether the prediction performance of the ML algorithms was better than that of traditional prediction models.

MATERIALS AND METHODS

Patients

The present study included 273 patients with GC, who were treated at the Harbin Medical University Cancer Hospital (Harbin, China), and underwent immunotherapy between September 2018 and August 2022. Patient diagnosis was based on tissue samples obtained during gastroscopy, and postoperative pathological tissue was examined by a pathologist to confirm the diagnosis.

The inclusion criteria were as follows: (1) GC diagnosed by pathological biopsy; (2) Patients who received more than one cycle of ICIs therapy; and (3) Due to ethical considerations and the rarity of GC in this population, this study only included patients > 18 years of age[30]. The exclusion criteria were as follows: (1) The patients received neoadjuvant therapy or conversion therapy before surgery; (2) The patient survived, but the follow-up period was less than 12 months; and (3) Patients with distant metastasis were not detected when receiving immunotherapy. Excluding these patients was intended to ensure the reliability of the results because those with GC who undergo neoadjuvant chemotherapy before surgery and those without distant metastasis typically have longer survival times than those with distant metastatic GC, which could affect the reliability and accuracy of the results[31,32]. Additionally, it was not possible predict whether patients with GC with a shorter follow-up period experienced death or disease progression, and excluding this group would make the results more credible[33]. This study was approved by the Ethics Committee of Harbin Medical University Cancer Hospital (Ethics Approval No. 2019-185) and all participants provided written informed consent. The study design and implementation strictly adhered to the Declaration of Helsinki and relevant ethical guidelines.

Data extraction

Thirty-four clinicopathological features from 273 patients with GC were collected, including: Sex; age; operation; smoking; drinking; body mass index (BMI); number of metastatic sites; 8 inflammatory or nutritional indices; and 19 other hematological indicators. Hematological samples were collected within 7 days of undergoing immunotherapy. The inflammation or nutrition index was calculated using the following parameters: Lymphocyte count/monocyte count (LMR); serum direct bilirubin level/indirect bilirubin level (DIR); serum albumin + 5 × lymphocyte count (PNI); serum albumin/(serum total protein-serum albumin) (AGR); aspartate aminotransferase level/platelet count (APRI); red cell distribution width/platelet count (RPR); and platelet count/(lymphocyte count × serum prealbumin) (PLPR); and alkaline phosphatase level/serum prealbumin (APR) (Figure 1).

Figure 1 Flow chart of the study. SHAP: Shapley additive explanation.

Study outcome and follow-up

The primary and secondary endpoints were PFS and OS, respectively. OS was defined as the period from the date the patient first received ICIs to the date of death of any cause. PFS was defined as the period from the date of the first administration of ICIs to the date of cancer progression or patient death[9]. In this study, the definition of tumor progression was based on the growth of tumor volume, appearance of new metastases, or enlargement of existing metastases. Follow up was performed by telephone every 3 months until the patient died or until 12 months of follow-up. The last follow-up date on September 1, 2023. During the course of ICIs treatment, all patients underwent tumor marker or radiological [ultrasound, computed tomography (CT), and gastroscopy], with review every 3-6 months. Polyethylene terephthalate/CT examinations were performed as required. In addition, in order to ensure the accuracy and objectivity of the evaluation, we used the internationally recognized tumor evaluation standard RECIST 1.1 to evaluate the CT staging. All imaging data were initially analyzed by a qualified radiologist, and the CT images of each patient were reviewed by another experienced radiologist to ensure the accuracy and consistency of the evaluation results.

Data preprocessing and patient randomization

Because some patients did not check all hematological indicators within 7 days before treatment, the KNNImputer method was used to fill-in missing information for variables with missing values < 30%. KNNImputer was chosen because it is an effective method for handling missing data in datasets in which variables may have complex relationships with one another. Unlike mean or median imputation, which does not consider correlations between variables, KNNImputer leverages the k-nearest neighbor algorithm to estimate missing values based on the similarity between data points, thus preserving the underlying structure and patterns of the data. For datasets with a 10% missing rate, the method achieved an accuracy of approximately 88% when imputing missing values[34]. The missing data are listed in Supplementary Table 1. A total of 30 peripheral blood indicators were included in this study, of which 18 were missing < 10%, and 2 were missing < 20% and < 30%. Although KNNImputer may introduce some degree of uncertainty, the impact was expected to be minimal given that the proportion of missing data was relatively small (< 30%)[34]. Subsequently, based on whether an outcome occurred, the 273 patients were randomly divided into training and test sets at a ratio of 7:3. The training set was used to build the models and the test set was used to evaluate the predictive performance of the models.

Construction and interpretation of ML models

First, univariate Cox proportional hazard analysis was performed on clinical data from 273 patients using OS and PFS as endpoints. For feature selection, only features with P < 0.05 in the univariate analysis were considered to be statistically significant and, thus, selected for further modeling. These selected features were then used to build the XGBoost, LR, and DT. After the prediction models were established, the Shapley additive explanation (SHAP) was used to explain the contribution of each feature in the models to better understand the results and working principles of the models.

Statistical analysis

Python version 3.9 (https://docs.python.org/3.9/) and R version 4.2.3 (R Core Team, 2023, R: Language and envi-ronment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL: https://www.R-project.org/) was used to perform statistical analysis. Classification variables are expressed as frequency and percentage, while continuous variables are expressed as mean and standard deviation or median and quartiles. The χ² test was used to compare classified variables, and the independent sample t-test was used to evaluate the distribution of continuous variables. In addition, 7 metrics were used to compare the predictive performances of the different models. These metrics included area under the receiver operating characteristic curve (AUC), sensitivity, specificity, F1 score, positive predictive value (PPV), and negative predictive value (NPV). AUC is a commonly used and effective measure for evaluating the performance of binary classifiers[35]. Sensitivity refers to the proportion of actual positive samples that the model correctly identifies, commonly known as the “true positive rate”; conversely, specificity is commonly referred to as the “true negative rate”. PPV indicates the proportion of actual positive cases among the samples predicted as positive, while NPV indicates the proportion of actual negative cases among the samples predicted as negative[36]. The F1 score is the harmonic mean of the precision and recall[37]. These 7 metrics are complementary and enable a more comprehensive and accurate identification of the best-performing models. Differences with P < 0.05 were considered to be statistically significant.

RESULTS

Patient characteristics

Clinicopathological features of the patients included in this study are summarized in Supplementary Table 2. In accordance with the screening criteria, 273 patients with GC and treated with ICIs were enrolled. Among these, the mean age was 60 years, 185 (67.8%) were male, 101 had undergone surgery before ICIs treatment (37.0%), 75 were smokers (27.5%), 51 had drinking habits (18.7%), and 70 had ≥ 3 distant metastases (25.6%). According to χ² test results, patients with GC, who had undergone surgical treatment, exhibited significantly fewer metastatic sites than those who had not undergone surgery, with a median number of metastatic sites of 1 and 2, respectively (Supplementary Table 3).

Feature selection and survival analysis

Univariate Cox proportional hazard analysis was performed to identify the most relevant characteristics of 273 patients with GC treated with ICIs. Results revealed that factors related to OS included PLPR, BMI, PNI, red blood cell (RBC) count, and number of metastatic sites (Table 1). Characteristics associated with PFS included PNI, age, RBC count, and BMI (Table 1).

Table 1 Univariate cox proportional hazard analysis for overall survival.

Characteristics	OS		PFS
	HR (95%CI)	P value	HR (95%CI)	P value
Sex	1.07 (0.72-1.59)	0.72	1.18 (0.82-1.7)	0.367
Age	0.99 (0.97-1)	0.144	0.98 (0.97-1)	0.024
Operation	0.81 (0.55-1.2)	0.299	0.85 (0.59-1.22)	0.376
Smoking	0.85 (0.56-1.29)	0.444	0.85 (0.58-1.26)	0.425
Alcohol drinking	0.76 (0.46-1.26)	0.285	0.88 (0.56-1.38)	0.575
BMI	0.92 (0.86-0.98)	0.006	0.93 (0.88-0.99)	0.018
Number of metastatic site	1.68 (1.13-2.49)	0.01	1.41 (0.97-2.06)	0.07
LMR	0.91 (0.81-1.01)	0.085	0.93 (0.84-1.03)	0.157
DIR	0.78 (0.23-2.66)	0.696	0.9 (0.29-2.73)	0.846
PNI	0.94 (0.91-0.97)	< 0.001	0.97 (0.94-0.99)	0.019
AGR	0.76 (0.37-1.53)	0.438	0.93 (0.48-1.77)	0.815
APRI	0.91 (0.23-3.59)	0.889	0.78 (0.21-2.87)	0.708
RPR	3.19 (0.34-29.61)	0.308	1.82 (0.19-17.8)	0.607
PLPR	1.18 (1.05-1.33)	0.005	1.13 (1-1.27)	0.055
APR	1.04 (0.96-1.13)	0.359	1.01 (0.93-1.11)	0.757
Eosinophil	1.11 (0.38-3.22)	0.843	1.08 (0.4-2.9)	0.876
Basophilic granulocyte	0.04 (0-202.01)	0.453	0.02 (0-73.72)	0.356
RBC	0.64 (0.48-0.85)	0.002	0.68 (0.52-0.88)	0.004
Mean corpuscular volume	1.02 (0.99-1.04)	0.167	1.02 (1-1.04)	0.098
Mean corpuscular hemoglobin	1.04 (0.98-1.1)	0.192	1.05 (1-1.1)	0.072
Platelet crit	2.49 (0.38-16.37)	0.343	2.46 (0.42-14.41)	0.317
γ-glutamyl transpeptidase	1 (1-1)	0.415	1 (1-1)	0.866
Urea	0.92 (0.82-1.03)	0.136	0.92 (0.83-1.02)	0.132
Creatinine	1 (0.99-1.01)	0.836	1 (0.99-1.01)	0.702
Carbondioxide combining power	1.01 (0.94-1.07)	0.854	1.01 (0.95-1.07)	0.759
Glucose	0.95 (0.85-1.07)	0.41	0.94 (0.84-1.05)	0.263
Kalium ion	0.71 (0.45-1.13)	0.148	0.75 (0.49-1.15)	0.194
Natrium ion	0.98 (0.92-1.04)	0.536	1 (0.94-1.07)	0.917
Chloride ion	0.95 (0.9-1.01)	0.082	0.96 (0.91-1.02)	0.179
Calcium ion	0.25 (0.06-1.05)	0.059	0.5 (0.14-1.78)	0.286
Phosphonium ion	0.9 (0.32-2.52)	0.845	1.04 (0.41-2.68)	0.93
Magnesian ion	0.32 (0.06-1.74)	0.186	0.74 (0.15-3.77)	0.718
Carcinoembryonic antigen	1 (1-1)	0.907	1 (1-1)	0.515
Carbohydrate antigen 199	1 (1-1)	0.193	1 (1-1)	0.441

OS: Overall survival; PFS: Progression-free survival; HR: Hazard ratio; CI: Confidence interval; BMI: Body mass index; LMR: Lymphocyte count/monocyte count; DIR: Direct bilirubin level/indirect bilirubin level; PNI: Serum albumin + 5 × lymphocyte count; AGR: Serum albumin/(serum total protein-serum albumin); APRI: Aspartate aminotransferase level/platelet count; RPR: Red cell distribution width/platelet count; PLPR: Platelet count/(lymphocyte count × serum prealbumin); APR: Alkaline phosphatase level/serum prealbumin; RBC: Red blood cell.

After these variables were selected, the surv_cutpoint function was used to determine the optimal cut-off values for continuous variables. First, all possible cut-off points within the range of the continuous variables were applied to the data, and survival analysis results were computed for each cut-off point. The optimal cut-off value was then selected based on criteria, such as maximizing the survival difference between groups or minimizing within-group variance, thereby dividing the continuous variable into high and low groups. The association between these factors and OS or PFS was analyzed. Results of analysis revealed that patients with higher PLPR levels or number of metastatic sites had shorter survival (Supplementary Figure 1A and B), while those with higher BMI, PNI, and RBC counts tended to experience longer survival (Supplementary Figure 1C-E). Additionally, patients with higher PNI, RBC count, BMI, or older age tended to experience longer PFS (Supplementary Figure 2).

Random grouping of the training and test sets

Based on the survival and death status, patients were randomly assigned to the training and test sets at a ratio of 7:3 (Supplementary Table 4). There were 191 and 82 participants in the training and test sets, respectively. Using the same method, patients were randomly assigned to the training and test sets at a ratio of 7:3 according to whether the patient was progressing, and all variables were evenly distributed (Supplementary Table 5).

Prediction performance of ML models

Data from the training set was used to build XGBoost, DT, and LR for OS and PFS, and verified the prediction ability of the models in the test set. Based on the results of feature selection, PLPR, BMI, PNI, RBC count, and number of metastatic sites were used to construct prediction models with OS as the outcome, while PNI, age, RBC count, and BMI were used to construct prediction models with PFS as the endpoint. Detailed performance data for the 3 models for the training and test sets are reported in Table 2. The results demonstrated that when the 3 ML models were used to predict patient OS in the training set, XGBoost outperformed the other 2 models in 3 indices: Precision (0.723), PPV (0.745), and NPV (0.701), and DT excelled in 4 indices: AUC (0.758), accuracy (0.723), recall ratio (0.708), and F1 score (0.710). In the test set, XGBoost outperformed the other models in terms of 6 indicators: AUC (0.695), accuracy (0.634), precision (0.621), recall ratio (0.619), F1 score (0.620), and PPV (0.563). Moreover, compared with the other models, DT exhibited the highest NPV (0.700). Therefore, considering the performances of the 3 models in both the training and test sets, XGBoost was selected to predict OS of patients with GC treated with ICIs. When the ML models were used to predict patient PFS in the training set, XGBoost scored higher than LR and DT in 6 indices: AUC (0.835), accuracy (0.759), precision (0.759), recall ratio (0.757), F1 score (0.758), and NPV (0.762). The DT exhibited the highest PPV (0.895). In the test set, when predicting PFS, XGBoost outperformed LR and DT in 5 aspects: Accuracy (0.565), precision (0.585), recall (0.586), F1 score (0.585), and NPV (0.610). The DT surpassed the other 2 models in 2 metrics: AUC (0.696) and PPV (0.625). Combining the performances of all predictive models, XGBoost was selected to predict PFS. The risk score for each patient was calculated based on XGBoost results. The best grouping cut-off was then screened using the surv_cutpoint function in the Survminer package, and the patients were divided into high- and low-risk groups. The Kaplan-Meier survival curve results revealed that patients in the high-risk group had significantly shorter OS and PFS in both the training and test sets (P < 0.05) (Figure 2).

Figure 2 Kaplan-Meier survival curve analysis of risk score based on XGBoost. A: When XGBoost was used to predict the patient’s overall survival, the Kaplan-Meier curve of the risk score in the training set; B: When XGBoost was used to predict the patient’s overall survival, the Kaplan-Meier curve of the risk score in the test set; C: When XGBoost was used to predict the patient’s progression-free survival, the Kaplan-Meier curve of the risk score in the training set; D: When XGBoost was used to predict the patient’s progression-free survival, the Kaplan-Meier curve of the risk score in the test set.

Table 2 Different machine learning algorithms were used to predict the patient’s overall survival and progression-free survival performance.

Indicator	OS						PFS
	XGB1	LR1	DT1	XGB2	LR2	DT2	XGB1	LR1	DT1	XGB2	LR2	DT2
AUC	0.732	0.652	0.758	0.695	0.628	0.608	0.835	0.581	0.667	0.677	0.511	0.696
Accuracy	0.712	0.613	0.723	0.634	0.573	0.610	0.759	0.529	0.607	0.565	0.524	0.549
Precision	0.723	0.692	0.713	0.621	0.523	0.612	0.759	0.264	0.735	0.585	0.262	0.583
Recall ratio	0.671	0.540	0.708	0.619	0.511	0.615	0.757	0.500	0.585	0.586	0.500	0.529
F1 score	0.673	0.485	0.710	0.620	0.464	0.608	0.758	0.346	0.519	0.585	0.344	0.448
PPV	0.745	0.611	0.524	0.563	0.455	0.524	0.756	0	0.895	0.561	0	0.625
NPV	0.701	0.613	0.700	0.680	0.592	0.700	0.762	0.529	0.576	0.610	0.524	0.541

¹Training set.

²Test set.

OS: Overall survival; PFS: Progression-free survival; XGB: eXtreme Gradient Boosting; LR: Logistic regression; DT: Decision tree; AUC: Area under the receiver operating characteristic curve; PPV: Positive predictive value; NPV: Negative predictive value.

Visualization of feature importance

The best model was selected as follows. SHAP was used to evaluate the effect of these screened variables on OS in patients with GC treated with ICIs. The mean (|SHAP value|) was used to sort the importance of the features. Results revealed that PLPR had the greatest influence on outcomes, followed by BMI (Figure 3A); the detailed impact on OS is illustrated in Figure 3B. The SHAP value (X-axis) refers to the influence of the value or status of different variables on the OS in the model, whereas the feature value (y-axis) refers to the change in a specific variable. Among the characteristics associated with PFS, PNI and age were the factors most closely associated with PFS (Figure 3C). PFS among the patients exhibited a trend toward prolongation with an increase in PNI, age, RBC count, and BMI (Figure 3D).

Figure 3 The Shapley additive explanation value analysis diagram of the XGBoost model is used to explain the importance and direction of the features that affect the overall survival and progression-free survival. A: In the XGBoost model with overall survival as the endpoint, the average contribution of each feature to overall survival is shown; B: In the XGBoost model with overall survival as the endpoint, with the change of the size of the value of each feature, the distribution of their detailed Shapley additive explanation values; C: In the XGBoost model with progression-free survival as the endpoint, the average contribution of each feature to progression-free survival is shown; D: In the XGBoost model with progression-free survival as the endpoint, with the change of the size of the value of each feature, the distribution of their detailed Shapley additive explanation values. PLPR: Platelet count/(lymphocyte count × serum prealbumin); BMI: Body mass index; PNI: Serum albumin + 5 × lymphocyte count; RBC: Red blood cell; SHAP: Shapley additive explanation.

Based on the SHAP value, the risk for each patient developing OS or PFS was calculated. The researchers explored 2 classic patients to explain the XGBoost at the end of OS. In the analysis, the red arrow indicates an increase in risk and the blue arrow indicates a decrease in risk. The SHAP value was calculated by combining the influences of the variables corresponding to the prediction score. Deceased patients with GC exhibited a lower BMI, and the widest red bar demonstrates that BMI had the greatest influence on the results (Supplementary Figure 3A). Patients who survived exhibited a lower PLPR, and the widest blue bar indicates that a lower PLPR had the greatest inhibitory effect on the outcome (Supplementary Figure 3B). Similarly, 1 patient with progression and 1 without progression were selected for analysis. According to these results, a lower PNI in patients with advanced GC played the most significant role in promoting outcomes (Supplementary Figure 4A). A higher RBC count in patients with non-progressive GC had the most significant inhibitory effect on the outcome (Supplementary Figure 4B).

Web-based calculator

To enhance the clinical utility of XGBoost, online calculators using the XGBoost algorithm were developed to predict the OS and PFS of patients with stage IV GC treated with ICIs. The websites used for these calculations included, https://wmhxgb.streamlit.app/ (for OS) and https://xgbpfs.streamlit.app/ (for PFS).

Subgroup analysis

Based on patient sex and whether they had undergone surgery, the 273 patients were divided into 4 subgroups: Males; females; operation; and non-operation. XGBoost demonstrated a higher accuracy when used to predict the OS of patients and females who had undergone surgery (Figure 4A). The results of SHAP revealed that PLPR was the most important factor affecting the length of survival of patients in any subgroup (Supplementary Figure 5). When XGBoost was used to predict PFS, it demonstrated greater accuracy in both females and the non-operative group (Figure 4B). In addition, in each subgroup, the characteristics that most correlated with PFS were PNI and age (Supplementary Figure 6). The risk score for each patient was calculated based on XGBoost results. In the training and test sets, the patients were divided into male, female, operation, and non-operation groups. The distribution of the patients’ predicted scores in each subgroup were analyzed. Results revealed that patients who died had higher predictive scores in all subgroups of the training and test sets. This difference was significant in most of the subgroups (Figure 5). Similarly, patients with tumor progression had higher predictive scores (Figure 6).

Figure 4 Performance in the four subgroups when XGBoost is used to predict the patient’s overall survival and progression-free survival. A: The performance of XGBoost in predicting overall survival for female patients, male patients, non-surgically treated patients, and surgically treated patients; B: The performance of XGBoost in predicting progression-free survival for female patients, male patients, non-surgically treated patients, and surgically treated patients. NPV: Negative predictive value; PPV: Positive predictive value.

Figure 5 When XGBoost is used to predict overall survival, the distribution of risk scores in each subgroup of the training set, and the test set.

Figure 6 When XGBoost is used to predict progression-free survival, the distribution of risk scores in each subgroup of the training set, and the test set.

Comparison of prediction performance between Nomogram and XGBoost

Using selected features, a nomogram was constructed to predict patient OS and PFS using the training and test set data, respectively. Results revealed that, in the training set, when the nomogram was used to predict OS, the AUC was 0.653, and when the nomogram was used to predict PFS, the AUC was 0.633. In the test set, when the nomogram was used to predict OS and PFS, the AUC were 0.661 and 0.622, respectively. These results demonstrate that, compared with the prediction performance of the nomogram, XGBoost achieved better results in predicting both OS and PFS (Supplementary Table 6).

DISCUSSION

For patients with metastatic GC, ICIs therapy may confer significant survival benefits in some[38]. However, ICIs may not be suitable for all patients with cancer[15,16]. We aimed to identify effective and readily available markers to help physicians determine which patients are more likely to benefit from ICIs therapy. In this study, we first identified the characteristics associated with the patient’s OS and PFS by univariate Cox proportional hazard analysis. Factors associated with OS include PLPR, BMI, PNI, RBC, and the number of metastatic sites. Characteristics associated with PFS include PNI, age, RBC, and BMI. Then, using the features we selected, XGBoost, LR and DT were constructed using the selected features. Subsequently, we used 7 metrics, including AUC, to evaluate the predictive performance of the models. These metrics are complementary and help identify the model with the best overall performance, thus improving the stratification of prognosis for patients with GC undergoing ICIs treatment and guiding treatment decision-making[35,36]. Through a comprehensive evaluation, we found that XGBoost demonstrated the best performance in predicting OS and PFS. As such, we used the SHAP method to explain XGBoost.

Previous studies have explored the relationship between the inflammatory response or nutritional status of patients and the prognosis of patients with GC treated with ICIs. For example, Ding et al[22] conducted prospective clinical trials and found that the systemic immune inflammation index and PNI could predict the chemosensitivity and prognosis of patients with GC treated with sintilimab. Chen et al[39] found that a nutritional index based on the total lymphocyte count, total cholesterol levels, and serum albumin was associated with OS and PFS in patients with advanced GC treated with ICIs. These results are similar to those of the present study. According to the SHAP results, the feature with the strongest correlation with OS was PLPR. Previous studies have shown that PLPR is an independent prognostic risk factor for OS in patients with resectable GC[40]. However, to date, no studies have analyzed whether PLPR can predict the prognosis of patients with cancer treated with ICIs. We also found that the PNI had the strongest correlation with PFS. Many studies have shown that patients with GC treated with ICIs often have a poor prognosis if they have a lower PNI before treatment[41,42]. The PLPR is composed of platelet count, lymphocyte count, and serum prealbumin levels. The PNI combines 2 parameters-lymphocyte count and serum albumin-which can, to some extent, reflect the nutritional status of patients. In previous studies, the relationship between peripheral blood markers and the prognosis of patients with cancer has garnered considerable attention. Tumor cells can induce platelet aggregation and thrombin formation, leading to the accumulation of fibrin around tumor cells and the formation of a dense fibrin layer, which helps tumor cells evade the killing effects of natural killer cells[43]. As fundamental components of various inflammatory indices, lymphocytes are significant for patients with cancer because they can inhibit the migration of circulating tumor cells by secreting interferon-gamma[44]. Albumin and prealbumin, both synthesized in the liver, are blood markers that reflect patient nutritional status. Patients with low albumin or prealbumin levels often have shorter survival times[20,45]. In addition, some studies have reported that patients with cancer and lower albumin levels have a higher risk for venous thromboembolism[46]. Furthermore, BMI and RBC count can, to some extent, predict the prognosis of patients. These results suggest that nutritional status is crucial for the prognosis of patients with GC treated with ICIs. Particularly in patients with GC and distant metastasis, even with the latest tumor-node-metastasis staging system and tumor markers, it is difficult to distinguish which patients are more likely to benefit from treatment. Therefore, in addition to the expression levels of proteins, such as PD-L1 or tumor mutation burden, some hematological markers that are readily available before treatment and reflect the nutritional status of patients also appear to be good candidates for distinguishing patients who are more likely to benefit from immunotherapy.

GC is a highly heterogeneous disease, and a single hematological index cannot accurately predict the prognosis of patients treated with ICIs[47,48]. Therefore, integrating multiple indicators is a better option for accurately distinguishing which patients are more likely to benefit from treatment. In this study, we constructed 3 separate ML models for OS and PFS. After assessing their overall performance on the training and testing sets, we opted to use XGBoost to predict the OS and PFS of patients with GC undergoing ICIs treatment. In addition, nomogram is a classical predictive model that has been widely used to predict the prognosis of patients with various diseases. However, according to our study, XGBoost is more accurate than nomogram in predicting the prognosis of GC patients treated with ICIs. These results are consistent with those of previous studies[49,50]. Based on results of the subgroup analysis, XGBoost appeared to be particularly good at predicting the prognosis of female patients, regardless of OS or PFS. This may be due to the small number of females with GC. Few previous studies have specifically analyzed factors related to the prognosis of females with GC and constructed prediction models. However, recent studies have shown an increase in the proportion of females with GC patients has increased in recent years[51]. Therefore, it is necessary to devote more attention to the prognosis of females diagnosed with GC.

It must be admitted that this study still has some inherent limitations. First, this is a retrospective study in which subjects come from the same center, which inevitably increases the risk of selection bias. Secondly, our sample size is very small, and our research results need to be verified in a larger data set. Third, our follow-up time to patients is relatively short, which may limit the maturation of survival data. Finally, due to the lack of peripheral blood data of some patients, we used KNNImputer’s method to fill in this part of the data, which may also lead to some deviations in the results of this study. Possibly due to the aforementioned constraints, we observed relatively lower accuracy when XGBoost was employed to predict OS in male patients and those who had not undergone surgical treatment. Previous studies have also found that, compared with ML algorithms such as support vector machines, the overfitting phenomenon in XGBoost is more pronounced, which may be explained by the relatively small sample size included in the study[52]. In addition, in different environments or datasets, variations in data features and distributions may lead to changes in the predictive performance of XGBoost. Accordingly, in future work, we hope to include more data from the center and, to the best of our ability, seek data from different regions as external verification to improve the accuracy and reliability of the prediction model as much as possible. In addition, considering that nutritional status may have a significant impact on the prognosis of patients with GC treated with ICIs, it is important to identify more markers reflecting the nutritional status of patients to screen for those who are more likely to benefit from treatment. We also hope to design clinical trials to artificially intervene in the nutritional status of patients and observe whether it can significantly improve treatment efficiency and prognosis. Finally, factors that may be related to the prognosis of females diagnosed with GC treated with ICIs were identified, and prediction models were built.

CONCLUSION

XGBoost demonstrated the highest accuracy in predicting OS and PFS in patients with GC and distant metastases undergoing ICIs treatment, demonstrating its potential in identifying which patients with GC are more likely to benefit from ICIs treatment. However, the interpretability of XGBoost makes the model’s output easier for clinicians to understand and apply, clearly highlighting the key variables influencing patient prognosis and their importance. In the future, incorporating more types of data (such as genomic, proteomic, and immune profiling data) into XGBoost models could further enhance predictive accuracy, assisting clinicians in making more precise treatment decisions and, thereby, improving the treatment outcomes and survival rates of patients with GC. Additionally, the nutritional status of patients with GC treated with ICIs can, to some extent, reflect their prognosis; therefore, it is necessary to identify more relevant markers to help clinicians design appropriate treatment plans.

ACKNOWLEDGEMENTS

We thank Professor Ying-Wei Xue and Professor Nan Wang for their support and supervision of this study.