Diagnostic accuracy of endoscopic ultrasound in pancreatic neuroendocrine tumors: A systematic review and meta analysis

Abstract

AIM: To detect pancreatic neuroendocrine tumors (PNETs) has been varied. This study is undertaken to evaluate the accuracy of endoscopic ultrasound (EUS) in detecting PNETs.

METHODS: Only EUS studies confirmed by surgery or appropriate follow-up were selected. Articles were searched in Medline, Ovid journals, Medline nonindexed citations, and Cochrane Central Register of Controlled Trials and Database of Systematic Reviews. Pooling was conducted by both fixed and random effects model).

RESULTS: Initial search identified 2610 reference articles, of these 140 relevant articles were selected and reviewed. Data was extracted from 13 studies (n = 456) which met the inclusion criteria. Pooled sensitivity of EUS in detecting a PNETs was 87.2% (95%CI: 82.2-91.2). EUS had a pooled specificity of 98.0% (95%CI: 94.3-99.6). The positive likelihood ratio of EUS was 11.1 (95%CI: 5.34-22.8) and negative likelihood ratio was 0.17 (95%CI: 0.13-0.24). The diagnostic odds ratio, the odds of having anatomic PNETs in positive as compared to negative EUS studies was 94.7 (95%CI: 37.9-236.1). Begg-Mazumdar bias indicator for publication bias gave a Kendall’s tau value of 0.31 (P = 0.16), indication no publication bias. The P for χ² heterogeneity for all the pooled accuracy estimates was > 0.10.

CONCLUSION: EUS has excellent sensitivity and specificity to detect PNETs. EUS should be strongly considered for evaluation of PNETs.

Key Words: Endoscopic ultrasound; Ultrasound; Endosonography; Pancreatic mass; Neuroendocrine tumors; Sensitivity; Specificity; Positive predictive value; Negative predictive value

Core tip: The published data on the diagnostic accuracy of endoscopic ultrasound (EUS) for detection of pancreatic neuroendocrine tumors is varied. We conducted a comprehensive review of the published literature to assess the diagnostic accuracy of EUS in this setting. Our systematic review and meta-analysis has demonstrated an excellent sensitivity and specificity of EUS in this setting compared to previously published literature of other imaging modalities such as transabdominal ultrasound, computed tomography, and magnetic resonance imaging.

INTRODUCTION

Neuroendocrine tumors of the gastrointestinal tract are rare, accounting for less than 1% of all malignancies with an estimated annual incidence of 1-4 per 100000; however, they may lead to significant morbidity and mortality[1,2]. These tumors are difficult to diagnosis, treat, and have a propensity for metastasis prior to their diagnosis given that many to not become clinically apparent until late in their course. These tumors may be found throughout the gastrointestinal tract; however, the pancreas is an area where neuroendocrine tumors are commonly discovered[3,4].

Neuroendocrine tumors of the pancreas (PNETs) may be functional or non-functional and are mostly sporadic, although some are associated with other genetic diseases[1]. Functional PNETs often secrete active substances, such as insulin, somatostatin, gastrin, glucagon, or vasoactive intestinal peptide, which may allow them to be discovered earlier[1,5]. However, some of these PNETs are non-functional, secreting non-active substances, such as chromogranin A[1]. Serological tests have been used to determine levels of these compounds, leading to an enhanced ability to diagnosis PNET. However, these tumors tend to have metastasized by the time they are diagnosed, especially in non-functioning PNETs. Many imaging modalities have been utilized for PNETs, including trans-abdominal ultrasound (US), computed tomography (CT), and magnetic resonance imaging (MRI) with significant limitations. These imaging techniques are able to detect PNETs in 9%-48% with an estimated sensitivity of 29%-60%[6-8]. Given the need for improved imaging techniques, endoscopic ultrasound (EUS) has been evaluated as a possible diagnostic tool for PNETs.

Since its early introduction in the early 1990’s, EUS has emerged as a safe and accurate technique for the diagnosis, stage, and treat a variety of lesions. A particularly useful aspect of EUS is the enhanced imaging of the pancreas. There are currently several reports of EUS in correctly detecting PNETs. However, the accuracy of these results varies across centers. To the best of our knowledge, a meta-analysis summarizing these results has not been performed. The purpose of this investigation is to review the world literature regarding the accuracy of EUS in detecting PNET.

MATERIALS AND METHODS

Study selection criteria

Studies evaluating the use of EUS to characterize pancreatic neuroendocrine tumors with a gold standard (either confirmed by surgery or appropriate follow-up) were selected. From this pool, only studies from which a 2 × 2 table could be constructed for true positive, false negative, false positive and true negative values were included.

Data collection and extraction

Articles were searched in MEDLINE (through PubMed, an electronic search engine for published articles and Ovid), PubMed, Ovid Journals, EMBASE, Cumulative Index for Nursing and Allied Health Literature, ACP Journal Club, DARE, International Pharmaceutical Abstracts, old Medline, Medline non-indexed citations, OVID Healthstar, and Cochrane Central Register of Controlled Trials and Database of Systematic Reviews (Central). The search was performed from January 1966 to January 2012. The terms used for search were endoscopic ultrasound, EUS, ultrasound, endosonography, pancreatic mass, neuroendocrine tumors, sensitivity, specificity, positive predictive value, and negative predictive value. Study authors were contacted when the required data could not be determined from the publications. Two by two tables were constructed with the data extracted from each study. Two authors (Puli SR, Bechtold ML) independently searched and extracted the data using an abstraction form. Any differences were resolved by mutual agreement. The agreement between reviewers for the collected data was quantified using the Cohen’s κ[9].

Quality of studies

Clinical trials designed with control and treatment arms can be assessed for quality of the study. A number of criteria have been used to assess the quality of a study (e.g., randomization, selection bias of the arms in the study, concealment of allocation, and blinding of outcome)[10,11]. There is no consensus on how to assess studies designed without a control arm. Hence, these criteria do not apply to studies without a control arm[11]. Therefore, for this meta-analysis and systematic review, studies were selected based on completeness of the data and inclusion criteria. Completeness was defined as data available for true positive, false negative, false positive and true negative values of the diagnostic test (EUS). Quality Assessment of Studies of Diagnostic Accuracy Included in Systematic Reviews (QUADAS) criteria has been proposed to evaluate quality of diagnostic studies[12,13]. This was used to evaluate the studies on 14 items described in the QUADAS criteria.

Statistical analysis

Meta-analysis for the accuracy of EUS in diagnosing PNETs was performed by calculating pooled estimates of sensitivity, specificity, likelihood ratios, and diagnostic odds ratios. Pooling was conducted using both Mantel-Haenszel Method (fixed effects model) and DerSimonian Laird Method (random effects model). The confidence intervals were calculated using the F distribution method[14]. Forrest plots were drawn to examine how the point estimates in each study related to the summary pooled estimate. For 0 value cells, a 0.5 was added as described by Cox[15]. The heterogeneity of the sensitivities and specificities were tested by applying the likelihood ratio test[16]. The heterogeneity of likelihood ratios and diagnostic odds ratios were tested using Cochran’s Q test based upon inverse variance weights[17]. Heterogeneity among studies was also tested by using summary receiver operating characteristic (SROC) curves. SROC curves were used to calculate the area under the curve (AUC). The affect of publication and selection bias on the summary estimates was tested by Egger bias indicator[18]. Also, funnel plots were constructed to evaluate potential publication bias using the standard error and diagnostic odds ratio[19,20].

RESULTS

Initial search identified 2610 reference articles, of these 140 relevant articles were selected and reviewed. Data was extracted from 13 studies[21-33] (n = 456) which met the inclusion criteria. Search results are shown in Figure 1. All the pooled estimates given are estimates calculated by the fixed effects model. The change adjusted agreement analysis between the reviewers for data collected separately gave a kappa value of 1.0. QUADAS criteria to evaluate the quality of studies demonstrated that all the studies fulfilled 4 to 5 out of 14 described criterion.

Figure 1 Search results. EUS: Endoscopic ultrasound; PNETs: Pancreatic neuroendocrine tumors.

Accuracy of EUS to detect PNETs

Pooled sensitivity of EUS in detecting PNETs was 87.2% (95%CI: 82.2-91.2). Forrest plot in Figure 2A shows the sensitivity of EUS in individual included studies. EUS had a pooled specificity of 98.0% (95%CI: 94.3-99.6). Figure 2B shows the specificity from various studies. The positive likelihood ratio (+LR) of EUS was 11.1 (95%CI: 5.34-22.8) and negative likelihood ratio (-LR) was 0.17 (95%CI: 0.13-0.24). The diagnostic odds ratio (DOR), the odds of having anatomic PNETs in positive as compared to negative EUS studies was 94.7 (95%CI: 37.9-236.1). All the pooled estimates calculated by fixed and random effect models were similar. SROC curves showed an area under the curve of 0.94. Figure 3 shows SROC curve and the area under the curve. Egger bias indicator for publication bias gave a value of 1.39 (95%CI: -1.52-4.32, P = 0.31), indication no publication bias. Funnel plot in Figure 4 also shows that there is no publication bias in the included studies. The P for chi-squared heterogeneity for all the pooled accuracy estimates was > 0.10.

Figure 2 Forrest plot. A: Forrest plot showing sensitivity of endoscopic ultrasound (EUS) to detect pancreatic neuroendocrine tumor; B: Forrest plot showing specificity of EUS to detect pancreatic neuroendocrine tumors.

Figure 3 Summary receiver operating characteristic curves of endoscopic ultrasound to detect pancreatic neuroendocrine tumors. AUC: Area under the curve; SROC: Summary receiver operating characteristic.

Figure 4 Funnel plot showing no publication bias in the studies included in this meta-analysis.

Subgroup analysis was performed to see how EUS performs in detecting an Insulinoma or a Gastrinoma in the pancreas.

Accuracy of EUS to detect an insulinoma in the pancreas

Data was extracted from 9 studies (n = 242) which met the inclusion criteria. Pooled sensitivity of EUS in detecting a Pancreatic Insulinoma was 87.5% (95%CI: 81.2-92.3). EUS had a pooled specificity of 97.4% (95%CI: 90.8-99.7). The +LR of EUS was 8.2 (95%CI: 3.7-18.3) and -LR was 0.17 (95%CI: 0.12-0.26). The DOR, the odds of having anatomic Pancreatic Insulinoma in positive as compared to negative EUS studies was 67.6 (95%CI: 22.7-200.9). All the pooled estimates calculated by fixed and random effect models were similar. SROC curves showed an area under the curve of 0.94. Egger bias indicator for publication bias gave a value of -0.05 (95%CI: -4.13-4.04, P = 0.98), indication no publication bias. The P for chi-squared heterogeneity for all the pooled accuracy estimates was > 0.10.

Accuracy of EUS to detect gastrinoma in the pancreas

Five EUS studies (n = 122) looked at detecting Gastrinomas in the pancreas. Pooled sensitivity of EUS in detecting a gastrinoma in the pancreas was 84.5% (95%CI: 72.6-92.7). EUS had a pooled specificity of 95.3% (95%CI: 86.9-99.0). The positive likelihood ratio of EUS was 10.5 (95%CI: 4.3-25.5) and negative likelihood ratio was 0.25 (95%CI: 0.13-0.47). The diagnostic odds ratio, the odds of having anatomic gastrinoma in positive as compared to negative EUS studies was 57.3 (95%CI: 15.1-217.2). All the pooled estimates calculated by fixed and random effect models were similar. SROC curves showed an area under the curve of 0.94. Egger bias indicator for publication bias gave a value of -0.74 (95%CI: -15.19-13.72, P = 0.88), indication no publication bias. The P for χ² heterogeneity for all the pooled accuracy estimates was > 0.10.

DISCUSSION

Localizing or detecting a neuroendocrine neoplasm in the pancreas helps not only with the planning of treatment but also when detected early might improve overall prognosis. Over the past sixteen years since the introduction of EUS, this minimally invasive technique has emerged as the premiere modality to confirm pancreatic neoplasms. In this meta-analysis we sought to pool and compare the findings of 13 high quality studies concerned with the performance of EUS in the evaluation of PNETs.

The strengths of this analysis are that the literature was reviewed and data was extracted independently by two independent reviewers. Comparison of their analyses indicates excellent agreement. This meta-analysis and systematic review was written in accordance with the proposal for reporting by the Quality of Reporting of Meta-analyses statement[34]. A standardized extraction algorithm was applied and only studies which fulfilled at least four of the QUADAS criterion were included. Additionally, extensive efforts were made to ensure that the true positive, false positive, true negative and false negative results for all studies were either gleaned from the literature or acquired via direct communication with the investigators. Egger bias estimates as well as funnel plots were performed and both methods suggest that there was no significant publication bias. Since this manuscript looks at diagnostic accuracy of a test, the study design for this meta-analysis and systematic review followed the guidelines of Standards for Reporting of Diagnostic Accuracy initiative[35].

A core finding of this meta-analysis is that in patients with symptomatic PNETs, EUS had high sensitivity (88%) and specificity (98%) in localizing the lesion to the pancreas. EUS as a diagnostic test has a very high DOR to detect PNETs (about 95 times). If EUS localizes the lesion to the pancreas, the odds of having the correct histological neuroendocrine tumor in the pancreas is 95 times.

Additional performance characteristics for EUS were assessed in this meta-analysis which demonstrate a high +LR and low -LR. The higher the positive likelihood ratio, the better the diagnostic test performs in correctly identifying the true disease state. On the flip side, negative likelihood ratio of a diagnostic test is a measure of how well the test correctly excludes a disease stage. The diagnostic tests ability to exclude a disease state is better if the negative likelihood ratio is lower. For PNETs, EUS has a high positive likelihood ratio and a low negative likelihood ratio. This indicates that EUS performs well in excluding as well as correctly localizing neuroendocine tumor within the pancreas.

In a subgroup analysis to look at performance of EUS to correctly diagnose Insulinomas, EUS had high sensitivity (88%) and specificity (97%). Also, EUS had high sensitivity (85%) and specificity (95%) to detect Gastrinomas in pancreas.

A strength of this meta-analysis is that there was no heterogeneity among the studies included in this analysis. Heterogeneity among different studies was evaluated not only with test of heterogeneity but also by drawing SROC curves and finding the AUC. An AUC of 1 for any diagnostic test indicates that the test is excellent. SROC curves for EUS showed that the value of AUC was very close to 1, indicating that EUS is an excellent diagnostic test to detect PNETs.

One limitation of this meta-analysis is that none of the included studies were multicentre or randomized trials. The included studies were small in size indicating the low incidence of neuroendocrine tumors among general population. Studies on EUS with statistical significance tend to be published and cited leading to publication bias. Additionally smaller studies may show larger treatment effects due to fewer case-mix differences (e.g., patients with only early localized vs late metastatic disease) than larger studies. This publication and selection bias may affect the summary estimates in any meta-analysis. This bias can be estimated by Egger bias indicators and construction of Funnel plots. In our meta-analysis and systematic review, bias calculations using Harbord-Egger bias indicator[18] showed no statistically significant bias. Furthermore, analysis using Funnel plots showed no significant publication bias among the studies included in this analysis.

At this time the role of EUS to detect PNETs is as an adjunct to the imaging modalities such as CT scans. This is especially true when they are functional and when present in an early localized stage. More recently a single center study undertaken by Khashab et al[36] showed the despite improvement in CT technology which has increased detection rates, they still missed PNETs that were smaller in size.

A subgroup analysis to see if FNA could improve the diagnostic accuracy could not be performed as there were only two studies[28,33] that included FNA data for PNETs. In a meta-analysis of 41 studies by Puli et al[37], the accuracy of EUS-FNA in the setting of solid pancreatic mass was analysed which showed a pooled sensitivity and specificity of 86.9% (95%CI: 85.5-87.9) and 95.8% (95%CI: 94.6-96.7) respectively. Given these findings it would make sense to probably conclude that EUS-FNA could replicate similar diagnostic characteristics in PNETs. In additional, Khashab et al[36] also reported increased diagnostic accuracy of EUS in detected lesions to the pancreas when the were smaller in size. This was especially true for functional PNETs which tend to present early due to active peptides.

In conclusion, EUS has excellent sensitivity and specificity to localize PNETs approaching 100%. In a subgroup analysis, EUS had high sensitivity and specificity to detect Insulinoma or Gastrinoma in the pancreas. Though the studies in literature are small studies, EUS should be strongly considered for evaluation of PNETs.

COMMENTS

Background

Biochemically active neuroendocrine tumors often arise from the pancreas and are preceded by hormone related symptoms before metastasis. They are often small early on and could be missed on traditional imaging such as abdominal ultrasound, computed tomography (CT), and magnetic resonance imaging. However, the performance characteristics of endoscopic ultrasound (EUS) from previously published studies have demonstrated varying results.

Research frontiers

To our knowledge there is no published meta-analysis that has reported the diagnostic accuracy of EUS in neuroendocrine tumor of pancreas (PNETs). Several small studies have demonstrated varying results. This study is undertaken to assess pooled estimates on the diagnostic accuracy of EUS in early PNETs.

Innovations and breakthroughs

EUS has excellent sensitivity and specificity in detecting PNETs both approaching close to 100%. A subgroup analysis is also performed for pancreatic functional PNETs i.e., gastrinoma and insulinoma which showed high sensitivity and specificity. This gives additional diagnostic option in patients undergoing conventional imaging such as CT scan and scintigraphy with higher diagnostic accuracy compared to previously published data of the former tests.

Applications

EUS can be used to identify pancreatic PNETs with high degree of diagnostic accuracy.

Terminology

EUS has a very high sensitivity and specificity in PNETs especially in early stages aiding in early diagnosis and potential treatment.

Peer review

The current paper of systematic review and meta-analysis investigates the diagnostic accuracy of EUS in diagnosis of PNETs. The statistical analysis performed in this study produced reliable results.