Published online Aug 14, 2020. doi: 10.3748/wjg.v26.i30.4453
Peer-review started: April 22, 2020
First decision: April 29, 2020
Revised: July 13, 2020
Accepted: July 30, 2020
Article in press: July 30, 2020
Published online: August 14, 2020
Processing time: 114 Days and 7.4 Hours
Despite advancements in operative technique and improvements in postoperative managements, postoperative pancreatic fistula (POPF) is a life-threatening complication following pancreatoduodenectomy (PD). There are some reports to predict POPF preoperatively or intraoperatively, but the accuracy of those is questionable. Artificial intelligence (AI) technology is being actively used in the medical field, but few studies have reported applying it to outcomes after PD.
To develop a risk prediction platform for POPF using an AI model.
Medical records were reviewed from 1769 patients at Samsung Medical Center who underwent PD from 2007 to 2016. A total of 38 variables were inserted into AI-driven algorithms. The algorithms tested to make the risk prediction platform were random forest (RF) and a neural network (NN) with or without recursive feature elimination (RFE). The median imputation method was used for missing values. The area under the curve (AUC) was calculated to examine the discriminative power of algorithm for POPF prediction.
The number of POPFs was 221 (12.5%) according to the International Study Group of Pancreatic Fistula definition 2016. After median imputation, AUCs using 38 variables were 0.68 ± 0.02 with RF and 0.71 ± 0.02 with NN. The maximal AUC using NN with RFE was 0.74. Sixteen risk factors for POPF were identified by AI algorithm: Pancreatic duct diameter, body mass index, preoperative serum albumin, lipase level, amount of intraoperative fluid infusion, age, platelet count, extrapancreatic location of tumor, combined venous resection, co-existing pancreatitis, neoadjuvant radiotherapy, American Society of Anesthesiologists’ score, sex, soft texture of the pancreas, underlying heart disease, and preoperative endoscopic biliary decompression. We developed a web-based POPF prediction platform, and this application is freely available at http://popfrisk.smchbp.org.
This study is the first to predict POPF with multiple risk factors using AI. This platform is reliable (AUC 0.74), so it could be used to select patients who need especially intense therapy and to preoperatively establish an effective treatment strategy.
Core tip: Postoperative pancreatic fistula (POPF) is a life-threatening complication following pancreatoduodenectomy. This is a retrospective study to develop a risk prediction platform for POPF using an Artificial intelligence (AI) model. Compared with established POPF risk prediction methods, this machine learning algorithms better predict the POPF risk correctly (AUC 0.74). This AI-driven platform can identify patients who need especially intense therapy and aid in the establishment of an effective treatment strategy.
- Citation: Han IW, Cho K, Ryu Y, Shin SH, Heo JS, Choi DW, Chung MJ, Kwon OC, Cho BH. Risk prediction platform for pancreatic fistula after pancreatoduodenectomy using artificial intelligence. World J Gastroenterol 2020; 26(30): 4453-4464
- URL: https://www.wjgnet.com/1007-9327/full/v26/i30/4453.htm
- DOI: https://dx.doi.org/10.3748/wjg.v26.i30.4453
Despite advancements in surgical technique and operative management, postoperative pancreatic fistula (POPF) is still widely considered to be the greatest contributor to major morbidity and mortality after pancreatoduodenectomy (PD), with an incidence of 10%-30%[1-5]. Furthermore, it frequently delays the timely delivery of adjuvant therapies, and reduces overall patient survival[6]. The indications of PD have been widening, and the procedure is offered to an increasing number of elderly patients with multiple comorbidities[7,8], prompting the need to accurately define which patients are fit for PD and could tolerate a potentially life-threatening POPF.
Recently, the management of POPF has undergone a paradigm shift from a standardized and uniform approach that could not reflect an individual's characteristics to a proactive mitigation strategy. The new strategy uses various predictive systems to enable early prediction and prevention and optimize individual treatment decisions[6,9,10]. Previous predictive systems[6,9,10] might reflect POPF incidence and had the merit of simplicity, but their predictive accuracy is somewhat questionable[11,12]. Therefore, to more accurately predict POPF, further research is needed.
Machine-learning (ML) is an artificial intelligence (AI) technology that has been adopted in many areas of modern society, including medical science. In ML, computational models composed of multiple processing layers learn various data representations with multiple levels of abstraction[13]. ML is currently being used in not only surgery[5,14,15] but also other areas, such as, pharmacogenomics, image classification, and medical decision support systems[16-20]. Therefore, for this study we aimed to develop a new risk prediction platform for POPF after PD using ML algorithms. If so, we expected that a patient’s predicted POPF risk could direct their clinical management and prevent or mitigate untoward outcomes.
Under institutional review board approval (No. SMC 2017-01-017), we retrospectively collected clinicopathological variables for 1846 patients who underwent PD to treat various periampullary tumors at Samsung Medical Center (Seoul, Republic of Korea) between January 2007 and December 2016. Among them, we excluded 77 (4.2%) patients who had metastasis from sites other than the primary tumor origin, had direct invasion from the primary tumor into adjacent organs, underwent surgery for a recurrence, or lacked medical information about POPF. We analyzed the remaining 1769 patients (1079 men and 690 women).
Data on preoperative, intraoperative, and postoperative outcomes were collected and maintained on a web-based database (MDB, Seoul, Korea). We originally analyzed 38 preoperative and intraoperative variables that could be associated with POPF. Preoperative laboratory data, such as serum C-reactive protein, amylase, lipase, and carbohydrate antigen (CA) 19-9 level just before the operation was entered to algorithms. Co-existing pancreatitis was defined as classic feature of pancreatitis on preoperative Computed Tomography (CT) scan or intraoperative findings. Underlying heart diseases included hypertension on medication, coronary or valvular heart disease, or various arrhythmic diseases. The location of tumors and pancreatic duct (p-duct) diameter were determined or measured by preoperative CT scan. Total intraoperative fluid infusion consisted of total amount of intravenous crystalloid, colloid, volume expanders, or blood transfusion. The pancreatic texture was determined as soft or hard by the surgeon during the operation.
Among the continuous variables, there was some level of missing data. Median imputation[19], which is a common approach for dealing with missing values in ML algorithms, was used. None of the categorical variables had missing values. We used the one-hot encoding technique to encode the categorical variables when only one of the categories was assigned. When a categorical variable had three exclusive choices, then we transformed the categorical variable into three individual binary variables. When the choices within a categorical variable are not exclusive (e.g., “A”, “B”, “both A and B”, and “none”), binary encoding was used by splitting each choice into separate columns and converting to binary codes[21,22]. As a result, 44 encoded variables were input in the ML models (Table 1).
Variables | Values | Variables | Values |
Age (yr) | 67.7 ± 10.1 | Preoperative ERBD/ENBD (n, %) | 470 (26.6) |
Sex (male/female) | 1079: 690 | Preoperative PTBD (n, %) | 254 (14.4) |
Body mass index (kg/m2) | 22.5 ± 3.8 | Preoperative ERPD (n, %) | 19 (1.1) |
Heart disease including | 735 (41.5) | Neoadjuvant therapy (RT/chemotherapy/CCRT/No) | 2: 6: 16: 1745 |
Diabetes mellitus (n, %) | 472 (26.7) | Operative time (min) | 443.2 ± 90.1 |
Pulmonary disease (n, %) | 153 (8.6) | †Intraoperative fluid infusion (mL) | 3129 ± 3495 |
Liver disease (n, %) | 99 (5.6) | Intraoperative transfusion (n, %) | 171 (9.7) |
Cerebrovascular disease (n, %) | 73 (4.1) | Estimated blood loss (ml) | 962.4 ± 665.1 |
Chronic kidney disease (n, %) | 17 (1.0) | Soft pancreas (n, %) | 750 (43.2) |
ASA score (1-4) | 372:1267: 128: 2 | Pancreatic duct diameter (mm) | 4.2 ± 2.8 |
White blood cell count (x10³/μL) | 6.6 ± 2.3 | Type of surgery (PPPD/PRPD/PD) | 1254: 244: 271 |
Hemoglobin (g/dL) | 10.9 ± 1.5 | Combined organ resection (n, %) | 67 (3.8) |
Platelet count (x10³/μL) | 280.2 ± 52.7 | Combined vascular resection(n, %) | 188 (10.6) |
Albumin (d/dL) | 3.5 ± 0.4 | C- reactive protein (mg/dL) | 1.9 ± 10.9 |
Total bilirubin level (mg/dL) | 3.5 ± 4.3 | CA 19-9 (U/mL) | 1786.5 ± 7141.5 |
Combined portal vein resection (n, %) | 175 (9.9) | P-duct stent (Internal/external/none) | 1051: 134: 584 |
Amylase (U/L) | 94.5 ± 5586.9 | Co-existing pancreatitis (n, %) | 370 (20.9) |
Lipase (U/L) | 160.5 ± 277.0 | ‡Location of tumor (Pancreas/others) | 856: 913 |
Anastomotic methods (1) (Duct-to-mucosa/Dunkin) | 1756: 13 | Anastomotic methods (2) (P-J/P-G/Others) | 1761: 8: 0 |
In cases of cholangitis or jaundice, preoperative endoscopic or percutaneous biliary drainage was performed. After the introduction of a definition of borderline resectable pancreatic cancer, 24 (1.4%) patients received neoadjuvant treatment using various regimens. All surgical procedures were performed by experienced 5 pancreatic surgeons at Samsung Medical Center who underwent more than 50 PDs annually. To create pancreatic anastomosis, 1761 (99.5%) patients underwent pancreati-cojejunostomy (PJ) and 8 (0.5%) patients underwent pancreaticogastrostomy. Pancreatoenteric anastomosis with stents was conducted in 1185 patients (70.0%) (Table 1). At the end of each surgical procedure, two or three drains were placed adjacent to the PJ anastomosis and on the right side of the superior mesenteric arterial resection margin. Serum and drain fluid amylase levels were routinely measured on postoperative days 1-3 and 5, 6, or 7, if the drains were maintained. In this study, we used ‘the definition of POPF in the 2016 update of the International Study Group (ISGPS) definition and grading of POPF’[23]. As a result, the grade A fistula has been removed from the POPF classification in this study. Drains adjacent to the PJ anastomosis were removed if no evidence of a leak was found in an abdominal CT scan on postoperative days 5-7. Patients who experienced POPF received proper management, including conservative, interventional, or surgical treatment, depending on each patient’s clinical condition.
Two ML algorithms, random forest (RF) and neural network (NN), were used to predict POPF. RF method is a kind of ensemble learning algorithm that builds multiple decision trees expecting better performance by taking mode or mean of individual trees[24]. An NN is a ML algorithm that emulates the synaptic structure of the brain[13]. It contains singular or multiple hidden layers, between the input and output layers[13]. Recursive feature elimination (RFE), which is a feature selection method that removes the weakest features until the maximum area under the curve (AUC) is reached[25,26], was used to identify the subset of features used in the final NN model. We tuned hyperparameters of NN (such as number of hidden layers, number of nodes, learning rates, batch size, dropout rate, and so on) to maximize the performance by grid search algorithm on each RFE step[27].
These AI-driven POPF prediction algorithms were developed using MATLAB Release 2017 band Python software with Tensoflow library.
The characteristics of the study population were described for each dataset, including the mean and standard deviation for each variable. For the development of the ML algorithms, the total dataset was split into a training set and a test set. The training set was used to derive the POPF prediction algorithms, and the test set was used to evaluate the derived algorithms. In order to evaluate our ML approaches, we used a stratified 5-fold cross-validation test. This randomly divides all the data into 5 partitions (folds) keeping each one with similar positive and negative data distribution. Then, we train a model with four of the partitions and test the model with the remaining fold. By changing the folds for training and testing, this process is performed 5 times. Also, the whole cross-validation was repeated 10 times by random split of the dataset, evaluating the performance of the models at the end. These processes ensure the generalized performance of a model by preventing overfitting to the samples. Because the outputs from the ML-driven POPF prediction algorithms are probabilistic estimates of risk, the performance for the test data was evaluated using AUC. The clinical meaning of the AI-driven risk factors for POPF was identified by sliding window approach[28]. All statistical and mechanical analyses assessing algorithm performance were done by Cho K and Cho BH from Medical AI Research Center, Samsung Medical Center using GraphPad Prism version 5.00 for Windows (GraphPad Software, SD, CA) and MATLAB Release 2017b (MathWorks, Inc., Natick, MA) software.
Table 1 provides the clinicopathologic details of the 1769 patients. Among them, grade B or C POPF occurred in 221 (12.5%) patients according to the ISGPF 2016 definition, and 130 (7.3%) patients had an American Society of Anesthesiologists’ (ASA) score ≥ 3. The mean value of body mass index (BMI) was 22.5 kg/m2, and the mean albumin level was 4.0 g/dl. The mean operating time was 443.2 min. The mean total fluid input and estimated blood loss during the operation were 3129.5 and 962.4 mL, respectively. A soft pancreas was observed in 750 (42.4%) patients. The mean diameter of the pancreatic duct was 4.2 mm. The most common tumor location was the pancreas, which occurred in 568 (32.1%) patients. Presumed pancreatitis was observed in 370 (20.9%) patients (Table 1). 30-d postoperative mortality was observed in 23 (1.3%) patients.
Table 2 summarizes the results from each algorithm using three different configurations of the dataset. Firstly, the data with complete values for the 38 original variables were input into the two ML algorithms. The average AUCs over the 5-fold cross validation with 10 repetition were 0.67 with the RF and 0.74 with the NN, respectively. When complete data for 34 original variables (without serum C-reactive protein, amylase, lipase, and CA 19-9 level) were input into the algorithms, the 5-fold average AUCs were 0.67 with the RF and 0.72 with the NN. For the configuration of missing values treatment, we input data from 1769 patients into the ML algorithms. Those 5-fold average AUCs increased to 0.68 with the RF and 0.71 with the NN. All those AUCs are summarized in Table 2.
Dataset and algorithm | Number of original variables | Number of encoded variables | Number of samples | Area under curve |
Random forest with complete cases | 38 | 44 | 889 | 0.670.02 |
Neural network with complete cases | 0.740.02 | |||
Random forest with complete variables | 34 | 40 | 1769 | 0.670.01 |
Neural network with complete variables | 0.720.02 | |||
Random forest with missing data treatment | 38 | 44 | 1769 | 0.680.02 |
Neural network with missing data treatment | 0.710.02 |
Using all 1769 data samples after missing data treatment, we could further improve the AUC from 0.71 to 0.74 using NN with RFE method (Table 2). Sixteen risk factors for POPF were identified using NN with RFE method: Pancreatic duct diameter, BMI, preoperative serum albumin, lipase level, amount of intraoperative fluid infusion, age, platelet count, extrapancreatic location of tumor, combined venous resection, co-existing pancreatitis, neoadjuvant radiotherapy, ASA score, sex, soft texture of the pancreas, underlying heart disease, and preoperative endoscopic biliary decompression. (Figure 1). The post hoc analysis revealed a nonlinear relationship by showing the response of NN model to each input variable at every RFE step. Ten discrete points cover the observed range of variation for each corresponding variable. We found several patterns of NN output response, in which the predicted POPF risk seemed to have a positive, negative, or biphasic relationship with each variable. The contribution profiles of the top 16 variables are shown in Supplement Figure 1. Based on these multiple and complex relationships among the risk factors for POPF after PD, we made a network connections illustration to improve understanding (Figure 2).
NN algorithm using RFE that had the best performance across the metrics of discrimination, calibration, and overall performance was integrated into an interactive interface. We designed our clinical decision tool to collect values entered by a clinician, feed those values into the pre-trained algorithm, retrieve the result, and output that result to the clinician in real time. This POPF prediction platform is available as an open-access, web-based application programmed to be accessible and adaptable for use on desktops, tablets, and smartphones. It is freely available at https://popfrisk.smchbp.org/.
POPF is a serious inherent risk of a pancreatic resection. The best option for managing POPF is undoubtedly prevention using a preoperative and intraoperative POPF risk assessment that guides response measures postoperatively[6,9,10]. Theoretically, ML could offer an opportunity to improve the accuracy of risk assessment by exploiting the complex interactions among risk factors that affect POPF. To the best of our knowledge, this study presents the first ML algorithm for predicting POPF using multiple pre- and intraoperative variables derived from a large, single-institutional dataset. The maximum AUC of this model was considerable: 0.74 with NN with RFE method (Figure 1). Ultimately, a patient’s predicted POPF risk could direct their clinical management and prevent or mitigate untoward outcomes.
AI in the form of ML discovers intricate structures in large datasets by using a backpropagation algorithm to indicate how a machine should change the internal parameters it uses to compute the representation in each layer based on the representation in the previous layer[13]. ML can identify latent variables that are unlikely to be observed but might be inferred from other variables[18,19,24]. For example, NN with RFE algorithm found many pre- and perioperative predictors for POPF that we used in our final modeling (Figure 1). Previously developed risk assessment models[6,9,10] implicitly assume that the risk factors are related to POPF in a linear fashion. Those models could thus oversimplify complex, nonlinear relationships among many risk factors. Even if it might be cumbersome to calculate the risk of POPF using 16 variables in actual clinical care, our AI-driven risk platform better incorporates multiple risk factors and can account for more nuanced relationships between the risk factors and POPF (Figure 2). An example of nonlinearity is shown in Supplement Figure 1, and note that each variable has a variable effect.
This ML algorithms found 16 risk factors for POPF (Figure 1). These risk factors can be categorized into 3 groups: The technically demanding group, intraoperative volume status–related group, and poor general condition group (Figure 2). The risk factors in the technically demanding group (soft pancreas[2,5,6,9], small pancreatic duct[6,9,29], extrapancreatic lesion[6], absence of preoperative pancreatitis or low lipase level[30], absence of preoperative endoscopic biliary decompression, absence of neoadjuvant radiotherapy, and high BMI[7]) indicate potential difficulty in reconstructing the pancreatic-enteric anastomosis, which could cause POPF. Patients with pancreatic cancer, chronic pancreatitis, or neoadjuvant treatment have increased pancreatic fibrosis and a lower incidence of POPF than other PD patients[30,31]. Also, it is well-known that preoperative endoscopic biliary drainage is frequently associated with procedure-related pancreatitis[32]. As a result, those procedures might reduce the risk of POPF. The risk factors in the intraoperative volume status–related group (large intraoperative fluid administration, concomitant portal vein-superior mesenteric vein resection, and low platelet count) could cause ischemia and poor healing of the pancreatic-enteric anastomosis, which is compounded by tissue edema from aggressive volume replacement in a rebound fashion[6,33]. The resultant swelling of the anastomosis can cause duct occlusion or suture disruption. The risk factors in the poor general condition group (old age, underlying heart disease, low preoperative serum albumin level, and low ASA score) could be related to poor nutritional status, which is considered to correlate with a high risk of POPF[34,35].
Notably, a high probability of POPF in patients characterized by only 1 or 2 classical fistula risk factors could not be determined. In this study, we found 16 risk factors by using AI algorithms (Figure 1), but controversy remains about the true risk factors for POPF. The varying results from different studies could be influenced by study design, the composition of the patient populations, or statistical methods. For example, there is still debate about whether intraoperative volume status, such as intraoperative blood loss, transfusion, or the amount of fluid administration, are risk factors for POPF[6,9,33,36]. Some reports suggest intraoperative volume status as an independent risk factor for POPF[6,33] because of pancreatic parenchymal and intestinal edema from aggressive volume replacement, but other studies have denied its adverse effect because estimation of blood loss during surgery is unreliable and inaccurate[3,9]. We think this discrepancy about the prognostic value of different risk factors for POPF could be a fundamental interpretation error caused by the assumption of linearity and an attempt to simplify what isn't actually simple. Therefore, ML algorithms such as those used in the study will be an important tool for POPF risk assessments.
Other recently proposed risk prediction models[6,9,10] have the advantage of being easily performed because they use only 3–6 variables. However, previously unknown risk factors for POPF are still being newly identified. For example, preoperative sarcopenia, an age-related decrease in muscle mass, has been identified as a risk factor for POPF[37-39]. Existing models cannot reflect new factors for POPF as they emerge but must be re-analyzed and developed from scratch. To make matters worse, as the number of potential risk factors increases, the complexity of the conventional models can cause over-fitting, yielding implausible results. However, we addressed that possibility by using active and appropriate choices in pre-training, hyper-parameter selection, and regularization in our AI-driven algorithms[19]. Because AI is scalable, there is no need to develop a new model; even if many new variables affecting POPF are introduced, it is possible to just continue adding them to the original model. As the amount of pancreatectomy data continues to grow, the creation and deployment of learning systems as accessible tools could significantly enhance the prognosis and management of POPF. New learning algorithms and architectures that are currently being developed, such as convolution[40] or recurrent[41] NNs, will accelerate this progress.
This AI-driven risk prediction platform for POPF could assist the drive toward personalized medicine by better tailoring risk management to individual patients. For example, after a risk evaluation, high-risk patients could be selected for a multiple-drain strategy and postoperative prophylactic octreotide use. In this way, we expect our platform to help select patients who need more intense therapy and establish effective (and cost-effective) treatment strategies for POPF. Various mitigation strategies have been proposed to reduce the occurrence and morbidity of POPF, including technical variations, such as, pancreaticogastrostomy reconstruction[2,42], dunking/invaginating anastomosis[1,43,44], absorbable mesh patches[45,46], and the use of intraperitoneal drains[29], anastomotic stents[47], and prophylactic somatostatin analogues[4,48,49]. As a part of those efforts, we have an ongoing trial of this risk score wherein we are applying a somatostatin analogue during postoperative days 0–3 in high-risk patients. Future prospective studies could stratify treatments based on the outcome of this platform and provide comprehensive treatment algorithms.
This study has several limitations. First, Co-existing pancreatitis, which is bound to be subjective, was defined as classic feature of pancreatitis on preoperative CT scan or intraoperative findings. Also, the input data used to develop risk prediction platform was pre- and intraoperative variables. In practice, it is important for both pre- and intraoperative variables to enter the algorithms in order to improve the predictability of POPF, but clinically, it may be helpful for only preoperative variables to enter the algorithms. Therefore, we will sooner or later conduct modeling for only preoperative variables using multicenter data. Second, the NN’s output response to changing each input variable partially revealed the variables’ nonlinear relationships to POPF risk. Therefore, the pattern of the output response should not be understood as a direct relationship between an input variable and POPF risk. Nevertheless, this process could help inform further explorations of diverse predictive risk factors and the future development of new risk prediction approaches and algorithms. Finally, the study, though a large institution, was not only conducted on patients in single center but also had the disadvantage of not performing external validation. As a result, the follow-up study will be conducted by performing an external validation on patients in multiple institutions.
In conclusion, ML algorithms are promising tools for the prediction of POPF that can be integrated into clinically useful decision tools. Compared with established POPF risk prediction methods, our ML algorithms better predict the POPF risk correctly. After external validation, this new platform could be used to select patients who need more intense therapy and to preoperatively establish an effective treatment strategy.
Despite advancements in operative technique and improvements in postoperative managements, postoperative pancreatic fistula (POPF) is a life-threatening complication following pancreatoduodenectomy (PD). Artificial intelligence (AI) technology is being actively used in the medical field, but few studies have reported applying it to outcomes after PD.
There are some reports to predict POPF preoperatively or intraoperatively, but the accuracy of those is questionable. Compared with established POPF risk prediction methods, we expect that our ML algorithms can better predict the POPF risk correctly.
This study aimed to develop a risk prediction platform for POPF with single center dataset using an AI model.
A total of 38 variables from 1769 patients who underwent PD from 2007 to 2016 at Samsung Medical Center were inserted into AI-driven algorithms. The algorithms tested to make the risk prediction platform were random forest (RF) and a neural network (NN) with or without recursive feature elimination (RFE). These algorithms can better incorporate multiple risk factors and account for more nuanced relationships between the risk factors and POPF. The median imputation method was used for missing values.
The number of POPFs was 221 (12.5%) according to the International Study Group of Pancreatic Fistula definition 2016. The maximal AUC using NN with RFE was 0.74. Sixteen risk factors for POPF were identified by AI algorithm: Pancreatic duct diameter, body mass index, preoperative serum albumin, lipase level, amount of intraoperative fluid infusion, age, platelet count, extrapancreatic location of tumor, combined venous resection, co-existing pancreatitis, neoadjuvant radiotherapy, American Society of Anesthesiologists’ score, sex, soft texture of the pancreas, underlying heart disease, and preoperative endoscopic biliary decompression. We developed a web-based POPF prediction platform available at https://popfrisk.smchbp.org.
This study is the first to predict POPF with multiple risk factors using AI. This platform is reliable (AUC 0.74), so it could be used to select patients who need especially intense therapy and to preoperatively establish an effective treatment strategy.
This study developed a risk prediction platform for POPF with single center dataset using an AI model. The follow-up study will be conducted by performing an external validation on patients in multiple institutions. Also, future prospective studies could stratify treatments based on the outcome of this platform and provide comprehensive treatment algorithms.
We thank the study research nurse, Hyemin Kim, for her tremendous work in the acquisition of data, and we thank Sang Eun Lee for dedicated support with medical illustration at Samsung Medical Information & Media Services at Samsung Medical Center.
Manuscript source: Unsolicited manuscript
Specialty type: Gastroenterology and hepatology
Country/Territory of origin: South Korea
Peer-review report’s scientific quality classification
Grade A (Excellent): A
Grade B (Very good): 0
Grade C (Good): C
Grade D (Fair): 0
Grade E (Poor): 0
P-Reviewer: Bian AZL, Cloyd JM S-Editor: Wang DM L-Editor: A P-Editor: Zhang YL
1. | Lyu Y, Li T, Wang B, Cheng Y, Zhao S. Selection of pancreaticojejunostomy technique after pancreaticoduodenectomy: duct-to-mucosa anastomosis is not better than invagination anastomosis: A meta-analysis. Medicine (Baltimore). 2018;97:e12621. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 17] [Cited by in F6Publishing: 14] [Article Influence: 2.3] [Reference Citation Analysis (1)] |
2. | Lyu Y, Li T, Cheng Y, Wang B, Chen L, Zhao S. Pancreaticojejunostomy Versus Pancreaticogastrostomy After Pancreaticoduodenectomy: An Up-to-date Meta-analysis of RCTs Applying the ISGPS (2016) Criteria. Surg Laparosc Endosc Percutan Tech. 2018;28:139-146. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 27] [Cited by in F6Publishing: 40] [Article Influence: 6.7] [Reference Citation Analysis (34)] |
3. | Chen BP, Chen M, Bennett S, Lemon K, Bertens KA, Balaa FK, Martel G. Systematic Review and Meta-analysis of Restrictive Perioperative Fluid Management in Pancreaticoduodenectomy. World J Surg. 2018;42:2938-2950. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 19] [Cited by in F6Publishing: 14] [Article Influence: 2.3] [Reference Citation Analysis (0)] |
4. | Garg PK, Sharma J, Jakhetiya A, Chishi N. The Role of Prophylactic Octreotide Following Pancreaticoduodenectomy to Prevent Postoperative Pancreatic Fistula: A Meta-Analysis of the Randomized Controlled Trials. Surg J (N Y). 2018;4:e182-e187. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 6] [Cited by in F6Publishing: 10] [Article Influence: 1.7] [Reference Citation Analysis (0)] |
5. | Kambakamba P, Mannil M, Herrera PE, Müller PC, Kuemmerli C, Linecker M, von Spiczak J, Hüllner MW, Raptis DA, Petrowsky H, Clavien PA, Alkadhi H. The potential of machine learning to predict postoperative pancreatic fistula based on preoperative, non-contrast-enhanced CT: A proof-of-principle study. Surgery. 2020;167:448-454. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 21] [Cited by in F6Publishing: 39] [Article Influence: 7.8] [Reference Citation Analysis (0)] |
6. | Callery MP, Pratt WB, Kent TS, Chaikof EL, Vollmer CM. A prospectively validated clinical risk score accurately predicts pancreatic fistula after pancreatoduodenectomy. J Am Coll Surg. 2013;216:1-14. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 653] [Cited by in F6Publishing: 847] [Article Influence: 70.6] [Reference Citation Analysis (2)] |
7. | Belyaev O, Herzog T, Kaya G, Chromik AM, Meurer K, Uhl W, Müller CA. Pancreatic surgery in the very old: face to face with a challenge of the near future. World J Surg. 2013;37:1013-1020. [PubMed] [Cited in This Article: ] |
8. | Pecorelli N, Capretti G, Sandini M, Damascelli A, Cristel G, De Cobelli F, Gianotti L, Zerbi A, Braga M. Impact of Sarcopenic Obesity on Failure to Rescue from Major Complications Following Pancreaticoduodenectomy for Cancer: Results from a Multicenter Study. Ann Surg Oncol. 2018;25:308-317. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 55] [Cited by in F6Publishing: 55] [Article Influence: 7.9] [Reference Citation Analysis (0)] |
9. | Mungroop TH, van Rijssen LB, van Klaveren D, Smits FJ, van Woerden V, Linnemann RJ, de Pastena M, Klompmaker S, Marchegiani G, Ecker BL, van Dieren S, Bonsing B, Busch OR, van Dam RM, Erdmann J, van Eijck CH, Gerhards MF, van Goor H, van der Harst E, de Hingh IH, de Jong KP, Kazemier G, Luyer M, Shamali A, Barbaro S, Armstrong T, Takhar A, Hamady Z, Klaase J, Lips DJ, Molenaar IQ, Nieuwenhuijs VB, Rupert C, van Santvoort HC, Scheepers JJ, van der Schelling GP, Bassi C, Vollmer CM, Steyerberg EW, Abu Hilal M, Groot Koerkamp B, Besselink MG; Dutch Pancreatic Cancer Group. Alternative Fistula Risk Score for Pancreatoduodenectomy (a-FRS): Design and International External Validation. Ann Surg. 2019;269:937-943. [PubMed] [DOI] [Cited in This Article: ] |
10. | You Y, Han IW, Choi DW, Heo JS, Ryu Y, Park DJ, Choi SH, Han S. Nomogram for predicting postoperative pancreatic fistula. HPB (Oxford). 2019;21:1436-1445. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 14] [Cited by in F6Publishing: 24] [Article Influence: 4.8] [Reference Citation Analysis (0)] |
11. | Ryu Y, Shin SH, Park DJ, Kim N, Heo JS, Choi DW, Han IW. Validation of original and alternative fistula risk scores in postoperative pancreatic fistula. J Hepatobiliary Pancreat Sci. 2019;26:354-359. [PubMed] [DOI] [Cited in This Article: ] |
12. | Kang JS, Park T, Han Y, Lee S, Kim JR, Kim H, Kwon W, Kim SW, Heo JS, Choi SH, Choi DW, Kim SC, Hong TH, Yoon DS, Park JS, Park SJ, Han SS, Choi SB, Kim JS, Lim CS, Jang JY. Clinical validation of scoring systems of postoperative pancreatic fistula after pancreatoduodenectomy: applicability to Eastern cohorts? Hepatobiliary Surg Nutr. 2019;8:211-218. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 12] [Cited by in F6Publishing: 13] [Article Influence: 2.6] [Reference Citation Analysis (0)] |
13. | LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436-444. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 36149] [Cited by in F6Publishing: 18330] [Article Influence: 2036.7] [Reference Citation Analysis (0)] |
14. | Hashimoto DA, Rosman G, Rus D, Meireles OR. Artificial Intelligence in Surgery: Promises and Perils. Ann Surg. 2018;268:70-76. [PubMed] [DOI] [Cited in This Article: ] |
15. | Bihorac A, Ozrazgat-Baslanti T, Ebadi A, Motaei A, Madkour M, Pardalos PM, Lipori G, Hogan WR, Efron PA, Moore F, Moldawer LL, Wang DZ, Hobson CE, Rashidi P, Li X, Momcilovic P. MySurgeryRisk: Development and Validation of a Machine-learning Risk Algorithm for Major Complications and Death After Surgery. Ann Surg. 2019;269:652-662. [PubMed] [DOI] [Cited in This Article: ] |
16. | Karhade AV, Thio QCBS, Ogink PT, Shah AA, Bono CM, Oh KS, Saylor PJ, Schoenfeld AJ, Shin JH, Harris MB, Schwab JH. Development of Machine Learning Algorithms for Prediction of 30-Day Mortality After Surgery for Spinal Metastasis. Neurosurgery. 2019;85:E83-E91. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 70] [Cited by in F6Publishing: 101] [Article Influence: 25.3] [Reference Citation Analysis (0)] |
17. | Menden MP, Iorio F, Garnett M, McDermott U, Benes CH, Ballester PJ, Saez-Rodriguez J. Machine learning prediction of cancer cell sensitivity to drugs based on genomic and chemical properties. PLoS One. 2013;8:e61318. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 321] [Cited by in F6Publishing: 284] [Article Influence: 25.8] [Reference Citation Analysis (0)] |
18. | Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform. 2002;35:352-359. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 1196] [Cited by in F6Publishing: 735] [Article Influence: 33.4] [Reference Citation Analysis (0)] |
19. | Weng SF, Reps J, Kai J, Garibaldi JM, Qureshi N. Can machine-learning improve cardiovascular risk prediction using routine clinical data? PLoS One. 2017;12:e0174944. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 789] [Cited by in F6Publishing: 545] [Article Influence: 77.9] [Reference Citation Analysis (0)] |
20. | Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA. 2016;316:2402-2410. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 3669] [Cited by in F6Publishing: 3129] [Article Influence: 391.1] [Reference Citation Analysis (0)] |
21. | Kassambara A. Machine Learning Essentials: Practical Guide in R. sthda, 2018. [Cited in This Article: ] |
22. | Potdar K, Pardawala T, Pai C. A Comparative Study of Categorical Variable Encoding Techniques for Neural Network Classifiers. IJCA. 2017;175:7-9. [DOI] [Cited in This Article: ] [Cited by in Crossref: 128] [Cited by in F6Publishing: 129] [Article Influence: 18.4] [Reference Citation Analysis (0)] |
23. | Bassi C, Marchegiani G, Dervenis C, Sarr M, Abu Hilal M, Adham M, Allen P, Andersson R, Asbun HJ, Besselink MG, Conlon K, Del Chiaro M, Falconi M, Fernandez-Cruz L, Fernandez-Del Castillo C, Fingerhut A, Friess H, Gouma DJ, Hackert T, Izbicki J, Lillemoe KD, Neoptolemos JP, Olah A, Schulick R, Shrikhande SV, Takada T, Takaori K, Traverso W, Vollmer CR, Wolfgang CL, Yeo CJ, Salvia R, Buchler M; International Study Group on Pancreatic Surgery (ISGPS). The 2016 update of the International Study Group (ISGPS) definition and grading of postoperative pancreatic fistula: 11 Years After. Surgery. 2017;161:584-591. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 2615] [Cited by in F6Publishing: 2600] [Article Influence: 371.4] [Reference Citation Analysis (35)] |
24. | Kim D, You S, So S, Lee J, Yook S, Jang DP, Kim IY, Park E, Cho K, Cha WC, Shin DW, Cho BH, Park HK. A data-driven artificial intelligence model for remote triage in the prehospital environment. PLoS One. 2018;13:e0206006. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 37] [Cited by in F6Publishing: 35] [Article Influence: 5.8] [Reference Citation Analysis (0)] |
25. | Cho BH, Yu H, Kim KW, Kim TH, Kim IY, Kim SI. Application of irregular and unbalanced data to predict diabetic nephropathy using visualization and feature selection methods. Artif Intell Med. 2008;42:37-53. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 94] [Cited by in F6Publishing: 64] [Article Influence: 3.8] [Reference Citation Analysis (0)] |
26. | Cho BH, Yu H, Lee J, Chee YJ, Kim IY, Kim SI. Nonlinear support vector machine visualization for risk factor analysis using nomograms and localized radial basis function kernels. IEEE Trans Inf Technol Biomed. 2008;12:247-256. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 46] [Cited by in F6Publishing: 46] [Article Influence: 2.9] [Reference Citation Analysis (0)] |
27. | Larochelle H, Erhan D, Courville A, Bergstra J, Bengio Y. An empirical evaluation of deep architectures on problems with many factors of variation. Proceedings of the 24th international conference on Machine learning. 2007: 473-480 Available from: https://doi.org/10.1145/1273496.1273556. [Cited in This Article: ] |
28. | Dias FM, Antunes A, Vieira J. On-line Training of Neural Networks: A Sliding Window Approach for the Levenberg-Marquardt Algorithm. Springer, Berlin, Heidelberg, 2005. [Cited in This Article: ] |
29. | Zhang W, He S, Cheng Y, Xia J, Lai M, Cheng N, Liu Z. Prophylactic abdominal drainage for pancreatic surgery. Cochrane Database Syst Rev. 2018;6:CD010583. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 20] [Cited by in F6Publishing: 21] [Article Influence: 3.5] [Reference Citation Analysis (0)] |
30. | Mathur A, Pitt HA, Marine M, Saxena R, Schmidt CM, Howard TJ, Nakeeb A, Zyromski NJ, Lillemoe KD. Fatty pancreas: a factor in postoperative pancreatic fistula. Ann Surg. 2007;246:1058-1064. [PubMed] [DOI] [Cited in This Article: ] |
31. | Takahashi H, Ogawa H, Ohigashi H, Gotoh K, Yamada T, Ohue M, Miyashiro I, Noura S, Kishi K, Motoori M, Shingai T, Nakamura S, Nishiyama K, Yano M, Ishikawa O. Preoperative chemoradiation reduces the risk of pancreatic fistula after distal pancreatectomy for pancreatic adenocarcinoma. Surgery. 2011;150:547-556. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 53] [Cited by in F6Publishing: 57] [Article Influence: 4.4] [Reference Citation Analysis (0)] |
32. | Mine T, Morizane T, Kawaguchi Y, Akashi R, Hanada K, Ito T, Kanno A, Kida M, Miyagawa H, Yamaguchi T, Mayumi T, Takeyama Y, Shimosegawa T. Clinical practice guideline for post-ERCP pancreatitis. J Gastroenterol. 2017;52:1013-1022. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 57] [Cited by in F6Publishing: 56] [Article Influence: 8.0] [Reference Citation Analysis (0)] |
33. | Han IW, Kim H, Heo J, Oh MG, Choi YS, Lee SE, Lim CS. Excess intraoperative fluid volume administration is associated with pancreatic fistula after pancreaticoduodenectomy: A retrospective multicenter study. Medicine (Baltimore). 2017;96:e6893. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 21] [Cited by in F6Publishing: 21] [Article Influence: 3.0] [Reference Citation Analysis (0)] |
34. | Gianotti L, Besselink MG, Sandini M, Hackert T, Conlon K, Gerritsen A, Griffin O, Fingerhut A, Probst P, Abu Hilal M, Marchegiani G, Nappo G, Zerbi A, Amodio A, Perinel J, Adham M, Raimondo M, Asbun HJ, Sato A, Takaori K, Shrikhande SV, Del Chiaro M, Bockhorn M, Izbicki JR, Dervenis C, Charnley RM, Martignoni ME, Friess H, de Pretis N, Radenkovic D, Montorsi M, Sarr MG, Vollmer CM, Frulloni L, Büchler MW, Bassi C. Nutritional support and therapy in pancreatic surgery: A position paper of the International Study Group on Pancreatic Surgery (ISGPS). Surgery. 2018;164:1035-1048. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 102] [Cited by in F6Publishing: 144] [Article Influence: 24.0] [Reference Citation Analysis (0)] |
35. | Kim E, Kang JS, Han Y, Kim H, Kwon W, Kim JR, Kim SW, Jang JY. Influence of preoperative nutritional status on clinical outcomes after pancreatoduodenectomy. HPB (Oxford). 2018;20:1051-1061. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 28] [Cited by in F6Publishing: 28] [Article Influence: 4.7] [Reference Citation Analysis (0)] |
36. | Grendar J, Jutric Z, Leal JN, Ball CG, Bertens K, Dixon E, Hammill CW, Kastenberg Z, Newell PH, Rocha F, Visser B, Wolf RF, Hansen PD. Validation of Fistula Risk Score calculator in diverse North American HPB practices. HPB (Oxford). 2017;19:508-514. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 27] [Cited by in F6Publishing: 36] [Article Influence: 5.1] [Reference Citation Analysis (0)] |
37. | Jang M, Park HW, Huh J, Lee JH, Jeong YK, Nah YW, Park J, Kim KW. Predictive value of sarcopenia and visceral obesity for postoperative pancreatic fistula after pancreaticoduodenectomy analyzed on clinically acquired CT and MRI. Eur Radiol. 2019;29:2417-2425. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 40] [Cited by in F6Publishing: 47] [Article Influence: 9.4] [Reference Citation Analysis (0)] |
38. | El Amrani M, Vermersch M, Fulbert M, Prodeau M, Lecolle K, Hebbar M, Ernst O, Pruvot FR, Truant S. Impact of sarcopenia on outcomes of patients undergoing pancreatectomy: A retrospective analysis of 107 patients. Medicine (Baltimore). 2018;97:e12076. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 26] [Cited by in F6Publishing: 29] [Article Influence: 4.8] [Reference Citation Analysis (0)] |
39. | Nishida Y, Kato Y, Kudo M, Aizawa H, Okubo S, Takahashi D, Nakayama Y, Kitaguchi K, Gotohda N, Takahashi S, Konishi M. Preoperative Sarcopenia Strongly Influences the Risk of Postoperative Pancreatic Fistula Formation After Pancreaticoduodenectomy. J Gastrointest Surg. 2016;20:1586-1594. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 78] [Cited by in F6Publishing: 91] [Article Influence: 11.4] [Reference Citation Analysis (0)] |
40. | Kawahara J, Brown CJ, Miller SP, Booth BG, Chau V, Grunau RE, Zwicker JG, Hamarneh G. BrainNetCNN: Convolutional neural networks for brain networks; towards predicting neurodevelopment. Neuroimage. 2017;146:1038-1049. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 250] [Cited by in F6Publishing: 221] [Article Influence: 27.6] [Reference Citation Analysis (0)] |
41. | Beiroti A, Aghasadeghi MR, Hosseini SN, Norouzian D. Application of recurrent neural network for online prediction of cell density of recombinant Pichia pastoris producing HBsAg. Prep Biochem Biotechnol. 2019;49:352-359. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 9] [Cited by in F6Publishing: 8] [Article Influence: 1.6] [Reference Citation Analysis (0)] |
42. | Topal B, Fieuws S, Aerts R, Weerts J, Feryn T, Roeyen G, Bertrand C, Hubert C, Janssens M, Closset J; Belgian Section of Hepatobiliary and Pancreatic Surgery. Pancreaticojejunostomy versus pancreaticogastrostomy reconstruction after pancreaticoduodenectomy for pancreatic or periampullary tumours: a multicentre randomised trial. Lancet Oncol. 2013;14:655-662. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 210] [Cited by in F6Publishing: 222] [Article Influence: 20.2] [Reference Citation Analysis (0)] |
43. | Berger AC, Howard TJ, Kennedy EP, Sauter PK, Bower-Cherry M, Dutkevitch S, Hyslop T, Schmidt CM, Rosato EL, Lavu H, Nakeeb A, Pitt HA, Lillemoe KD, Yeo CJ. Does type of pancreaticojejunostomy after pancreaticoduodenectomy decrease rate of pancreatic fistula? A randomized, prospective, dual-institution trial. J Am Coll Surg. 2009;208:738-47; discussion 747-9. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 267] [Cited by in F6Publishing: 279] [Article Influence: 18.6] [Reference Citation Analysis (0)] |
44. | El Nakeeb A, El Hemaly M, Askr W, Abd Ellatif M, Hamed H, Elghawalby A, Attia M, Abdallah T, Abd ElWahab M. Comparative study between duct to mucosa and invagination pancreaticojejunostomy after pancreaticoduodenectomy: a prospective randomized study. Int J Surg. 2015;16:1-6. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 54] [Cited by in F6Publishing: 49] [Article Influence: 5.4] [Reference Citation Analysis (0)] |
45. | Jang JY, Shin YC, Han Y, Park JS, Han HS, Hwang HK, Yoon DS, Kim JK, Yoon YS, Hwang DW, Kang CM, Lee WJ, Heo JS, Kang MJ, Chang YR, Chang J, Jung W, Kim SW. Effect of Polyglycolic Acid Mesh for Prevention of Pancreatic Fistula Following Distal Pancreatectomy: A Randomized Clinical Trial. JAMA Surg. 2017;152:150-155. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 52] [Cited by in F6Publishing: 62] [Article Influence: 8.9] [Reference Citation Analysis (0)] |
46. | Kang JS, Han Y, Kim H, Kwon W, Kim SW, Jang JY. Prevention of pancreatic fistula using polyethylene glycolic acid mesh reinforcement around pancreatojejunostomy: the propensity score-matched analysis. J Hepatobiliary Pancreat Sci. 2017;24:169-175. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 13] [Cited by in F6Publishing: 16] [Article Influence: 2.3] [Reference Citation Analysis (0)] |
47. | Jang JY, Chang YR, Kim SW, Choi SH, Park SJ, Lee SE, Lim CS, Kang MJ, Lee H, Heo JS. Randomized multicentre trial comparing external and internal pancreatic stenting during pancreaticoduodenectomy. Br J Surg. 2016;103:668-675. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 48] [Cited by in F6Publishing: 51] [Article Influence: 6.4] [Reference Citation Analysis (0)] |
48. | Gurusamy KS, Koti R, Fusai G, Davidson BR. Somatostatin analogues for pancreatic surgery. Cochrane Database Syst Rev. 2013;4:CD008370. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 64] [Cited by in F6Publishing: 75] [Article Influence: 6.8] [Reference Citation Analysis (0)] |
49. | Allen PJ, Gönen M, Brennan MF, Bucknor AA, Robinson LM, Pappas MM, Carlucci KE, D'Angelica MI, DeMatteo RP, Kingham TP, Fong Y, Jarnagin WR. Pasireotide for postoperative pancreatic fistula. N Engl J Med. 2014;370:2014-2022. [PubMed] [DOI] [Cited in This Article: ] [Cited by in Crossref: 283] [Cited by in F6Publishing: 257] [Article Influence: 25.7] [Reference Citation Analysis (1)] |