Validity of studies suggesting preoperative chemotherapy for resectable thoracic esophageal cancer: A critical appraisal of randomized trials

Abstract

BACKGROUND

In 2015, Kidane published a Cochrane review and meta-analysis to summarise the impact of preoperative chemotherapy versus surgery alone on survival for resectable thoracic esophageal cancer. The authors concluded that preoperative chemotherapy improved overall survival (OS).

AIM

The aim of this article was to assess the validity of the three most powerful studies included in the Cochrane review and the meta-analysis supporting the advantage of preoperative chemotherapy and to investigate the impact of an exclusion of these three studies on the result of the meta-analysis.

METHODS

OS was selected as the endpoint of interest. Among the ten included papers which analysed this endpoint, we identified the three publications with the highest weights influencing the final result. The validity of these papers was analysed using the CONSORT checklist for randomized controlled trials. We performed a new meta-analysis without the three studies to assess their impact on the general result of the original meta-analysis.

RESULTS

The three analysed studies revealed several inconsistencies. Inappropriate answers were found in up to one third of the items of the CONSORT checklist. Missing information about sample-size calculation and power, unclear or inadequate randomisation, and missing blinded set-up were the most common findings. When the three criticized studies were excluded in the meta-analysis, preoperative chemotherapy showed no benefit in OS.

CONCLUSION

The three most powerful publications in the Cochrane review show substantial deficits. After the exclusion of these studies from the meta-analysis, preoperative chemotherapy does not seem to result in an advantage in survival. We suggest a more critical appraisal regarding the validity of single studies.

Key Words: Esophageal cancer, Assessment of validity, Meta-analysis, CONSORT, Overall survival

Core tip: The quality of single studies is crucial in order to perform valid meta-analyses that are often used as basis for guideline recommendations. We critically analysed a recent Cochrane meta-analysis that supports the use of preoperative chemotherapy for resectable thoracic esophageal cancer in order to improve overall survival. The most powerful included studies showed several inconsistencies according to the requirements of the Consort checklist for randomized controlled trials. After the exclusion of these studies from the meta-analysis, preoperative chemotherapy does not seem to result in an advantage in survival. We suggest a more critical appraisal regarding the validity of single studies.

INTRODUCTION

Esophageal cancer is the eighth most common cancer in the world and the sixth most common cause of death from cancer with an overall ratio of mortality of 0.88[1]. Although it accounts for only 3.2% of all cancers, the incidence of esophageal cancer is increasing with an incidence of 572/100000 new cases/year in 2018[2].

Surgery is the treatment of choice for localized esophageal cancer[3-4] with a potential to provide loco-regional control, as well as long-term survival[5]. Curative resection is possible in only 15% to 39% of the cases[6-9]. Surgery is the only curative treatment, but it alone often fails to overcome the natural history of the disease owing to the presence of occult micrometastases, and fatal distant and loco-regional disease relapse is common. Median survival after esophagectomy with curative intention is 15 to 18 mo with a 5-year survival rate of 20% to 25%[5]. Therefore, clinicians are now inclined to use some form of multidisciplinary treatment including surgery as a standard of care for locally advanced esophageal cancer, which is defined as disease restricted to the esophagus or resectable periesophageal tissue (T2-T4) and/or lymph-node involvement (N1-N3) in the absence of distant metastasis[10].

The optimal multimodality treatment is still controversial. Potential contentious issues exist regarding the (1) ideal preoperative, perioperative or postoperative approach and (2) ideal combination of radiotherapy (RTx), chemotherapy (CTx) or concurrent chemoradiation. Various randomized and non-randomized trials and several meta-analyses have been conducted to address this topic, but established standard guidelines still vary considerably or even fail to propose a specific treatment regime[11]. Preoperative (radio-)chemotherapy aims to exterminate micro-metastases, enhance resectability by down-staging the tumour, improve loco-regional control and provide relief of dysphagia[11,12].

Several studies have investigated whether preoperative CTx leads to improved cure rates, but reports remain conflicting. The initial Cochrane review of preoperative CTx for resectable esophageal cancer[13] concluded that no survival advantage was associated with CTx. The same result was found by Urschel et al[14] after inclusion of 11 randomized trials in a meta-analysis. Ychou et al[15], Boonstra et al[16] and MRC Allum et al[17] subsequently reported a survival benefit for patients receiving neoadjuvant CTx. After inclusion of these last three studies, the updated Cochrane Review and meta-analysis by Kidane et al[18] on the same topic found an improvement in overall survival (OS) [hazard ratio (HR): 0.88; 95% confidence interval (CI): 0.80 to 0.96] for patients receiving preoperative CTx. A total of ten randomized controlled studies with OS as the primary endpoint were included in this meta-analysis.

The aim of our study was to assess the validity of the studies by Ychou et al[15], Boonstra et al[16] and MRC Allum et al[17] included in the updated Cochrane Review and the meta-analysis by Kidane et al[18], which confirmed the benefit of preoperative CTx on survival for resectable thoracic esophageal cancer with the intention to invite everyone to critically interpret not only the results, but also the methodology by which the results were achieved. We performed a variety of meta-analyses excluding or including studies depending on their validity and attributed power and discuss those findings in regard to current recommendations of esophageal carcinoma guidelines.

MATERIALS AND METHODS

The meta-analysis by Kidane et al[18] included a total of ten studies. Four (40%) (Ychou et al[15], Boonstra et al[16], MRC Allum et al[17], and Law et al[19]) found a statistically significant advantage in survival in patients after preoperative CTx for resectable thoracic esophageal cancer (HR: < 1 with a significant 95%CI). All the other six included studies (60%) were not statistically significant[20-25].

In the first part of the results section we assessed the validity of the three most powerful studies included in the Cochrane review by Kidane et al[18], which found a statistically significant advantage in survival in patients receiving preoperative CTx before resection for thoracic esophageal cancer. These studies were those of Ychou et al[15], Boonstra et al[16] and MRC Allum et al[17].

In the second part of the results section, we performed a new meta-analysis without these aforementioned three studies. Among the three analysed studies, Boonstra et al[16] had the higher validity, so we performed another meta-analysis assuming that this study is valid enough to be included in the meta-analysis.

Finally, we present the results of the meta-analysis excluding the four statistically significant studies confirming the survival advantage for patients treated with preoperative CTx. In this last case, only statistically non-significant studies were included in the meta-analysis.

Selection of the studies and assessment of their validity

We used the same methodology as described in our previous publication[26] to analyse the validity of the Cochrane review. From the several endpoints investigated in the Cochrane review by Kidane et al[18], we identified OS as a major endpoint of interest. Among the ten studies identified by the authors of the Cochrane review investigating OS, we selected the three most powerful studies as weighted by the review’s authors which support the advantage of preoperative CTx: Ychou et al[15], Boonstra et al[16] and MRC Allum et al[17]. The weights assigned to these three studies by the authors of the systematic review according to their sample size, precision of the estimates and width of the confidence intervals were 24.5%, 24.1% and 20.5%, respectively. We then assessed the validity of these studies using the CONSORT checklist 2010[27], which is a validated instrument for the evaluation of randomized controlled trials (RCTs) and contains a total of 37 items. The checklist with all items and their precise description is available in the Appendix of our previous publication[26]. We then asked whether the positive result in the Cochrane review is supported by sufficient validity. Figure 1 illustrates our methodology. Two independent review authors (UK and GM) assessed the validity of each of the three publications.

Figure 1 Four steps to the analysis of validity of a systematic review according to our previous work[26]. We identified the endpoint of interest (overall survival) and selected the three most powerful studies addressing this endpoint on the basis of the assigned weights by the authors of the systematic review, as these studies contributed essentially to the positive result of the systematic review. We finally assessed the validity of these studies by using the CONSORT checklist.

Meta-analysis

We repeated the meta-analysis without the three analysed studies (n = 7) and compared the result with the original meta-analysis comprising ten studies. Since Boonstra et al[16] has the higher validity among the analysed studies, we then conducted a second meta-analysis only excluding Ychou et al[15] and MRC Allum et al[17]. In a next step, we assumed that all single studies with a statistically significant benefit of preoperative CTx for thoracic resectable esophageal cancer (n = 4) were not valid enough and performed a second meta-analysis with the remaining six studies. The results were compared with the original meta-analysis (n = 10 studies). The meta-analyses were performed with R, version 3.2.0, with the package “meta” (http:// www.r-project.org/foundation).

RESULTS

Assessment of the validity of the studies

Table 1 presents a summary of the three analysed papers. The results are reported for each of the three included studies. Table 2 summarizes all the items present in the CONSORT checklist showing how the studies dealt with them. In this section, we describe the problems of each study. Eleven of the 34 validity criteria (32.4%) were not met in the study by Ychou et al[15]. Three items were not applicable. The randomisation occurred by phone call through a centralized randomisation system, and then the assignment was stratified according to centre, performance status, and tumour site using the minimisation procedure. Due to the use of the minimisation method, allocation concealment was not maintained. Blinding was not possible in this study, as the control group did not receive any preoperative treatment. Inclusion of untreated controls limits the interpretation of the study. Specifically, the difference between the intervention and control group may be caused by a non-specific effect, such as a placebo effect. Moreover, 50% of the patients in the intervention group also received postoperative CTx. Regarding sample size, in the methods section the authors described that 250 patients (178 deaths) were required to achieve the needed power. The trial was closed earlier due to difficulties in patient recruitment. At the closure time, a total of 224 patients (156 deaths) had been included, raising the question of whether the power was sufficient. Moreover, patients with stomach adenocarcinoma were also included in the study at a later time after changing the inclusion criteria. Taken together, these issues lead to insufficient validity of the report; therefore the described effect cannot be considered as clinically relevant.

Table 1 Summary of the three analysed studies.

Study (year)	Boonstra et al[16]	MRC Allum et al[17]	Ychou et al[15]
Number of included patients (intervention vs control)	85 vs 84	400 vs 402	113 vs 111
Inclusion criteria	100% squamous-cell cancer of thoracic oesophagus (upper, middle and lower third), T1-3, any N, M0 (M1a eligible if distal oesophageal cancer and suspected celiac nodes) < 80 yr of age, Karnofsky > 70	Squamous-cell cancer, adenocarcinoma, undifferentiated, upper, middle and lower thirds of oesophagus, as well as the gastric cardia	Resectable adenocarcinoma of the lower third of the oesophagus or gastro-oesophageal junction or stomach 18-75 years of age, WHO performance status 0 or 1, adequate renal (Cr < 120 mol/L) and hematologic functions
Intervention group	Preop. CTxa: Cisplatin, Etoposid iv. po. + surgery	Preop.CTx: Cisplatin, 5-FU + preop.radiotherapy + surgery	Preop.CTx: 5-FU, Cisplatin + surgery
Control group	Surgery	Preop. radiotherapy + surgery	Surgery
Outcome (intervention vs control)	Median overall survival 16 mo vs 12 mo, P = 0.03, by the log-rank test, HRb: 0.71; (95%CIc: 0.51-0.98)	Overall survival is significantly greater in CS group (HR: 0.84, 95%CI: 0.72-0.98, P = 0.03)	Overall survival significantly higher in CS group (HR for death 0.69, 95%CI: 0.50-0.95, P = 0.02) 5-year survival: 38% (95%CI: 29%-47%) in the CS group vs 24% (95%CI: 26%-44%) in the S group
Weight assigned in the Cochrane review (%)	24.1	20.5	24.5

^aCTx: chemotherapy,

^bHR: hazard ratio,

^cCI: Confidence interval. HR: Hazard ratio; CS: Chemotherapy + surgery; WHO: World Health Organization.

Table 2 Assessment of validity of the three analysed studies according to the CONSORT checklist (REF).

Section/Topic	Item number	Boonstra et al[16]	MRC Allum et al[17]	Ychou et al[15]
Title and Abstract	1a	Yes	Yes	No
	1b	Yes	Yes	Yes
Introduction
Background and objectives	2a	Yes	Yes	Yes
	2b	Yes	Yes	Yes
Methods
Trial design	3a	Yes	Yes	Yes
	3b	Not applicable	Not applicable	Yes
Participants	4a	Yes	Yes	Yes
	4b	Yes	No	No
Interventions	5	Yes	No	Yes
Outcomes	6a	Yes	Yes	Yes
	6b	Not applicable	Not applicable	Not applicable
Sample size	7a	Yes	No	Yes
	7b	Not applicable	Not applicable	Yes
Randomisation
-Sequence generation	8a	No	Yes	Yes
	8b	No	No	No
-Allocation concealment mechanism	9	No	No	No
- Implementation	10	No	No	No
Blinding	11a	No	Yes	No
	11b	Yes	No	No
Statistical methods	12a	Yes	Yes	Yes
	12b	Yes	Yes	Not applicable
Results
Participant flow	13a	Yes	Yes	Yes
	13b	Yes	Yes	Yes
Recruitment	14a	Yes	Yes	Yes
	14b	Not applicable	Not applicable	Nes
Baseline data	15	Yes	Yes	Yes
Numbers analysed	16	Yes	Yes	Yes
Outcomes and estimation	17a	Yes	Yes	Yes
	17b	Yes	Yes	Yes
Ancillary analysis	18	Yes	Yes	Not applicable
Harms	19	Yes	No	Yes
Discussion
Limitations	20	Yes	Yes	Yes
Generalisability	21	No	No	No
Interpretation	22	Yes	Yes	Yes
Other information
Registration	23	No	No	No
Protocol	24	No	No	No
Funding	25	Yes	Yes	No

In Boonstra et al[16], we identified poor validity in 8 of the 33 validity criteria (24.2%). Four items were not applicable. Again, as in the previous study, the use of untreated controls limits the interpretation of the study. Blinding was not possible in this work either, as the control group did not receive any preoperative treatment. Central randomisation was performed, but the process is not clearly described. Therefore, it is not possible to ascertain whether allocation concealment was maintained or not. Additionally, random assignment was stratified by age, gender, weight loss and tumour length. As also pointed out by the authors of the Cochrane review, it is unclear whether an intention-to-treat (ITT) analysis had been performed, as information on withdrawals was missing or unclear.

As the validity of the report is not sufficient, the described effect cannot be considered as clinically relevant.

MRC Allum et al[17] described the long-term results of a previously published study by the same group in 2002. If information was not found in the last studies, we checked if the needed information was available in the first publication[28]. Taking this into consideration, 11 of the 33 validity criteria were not met (33.3%) by MRC Allum et al[17]. Four items were not applicable. As in the previous study, the use of untreated controls limits the interpretation of the study. Blinding was also not possible because the control group did not receive any preoperative treatment. Due to the use of the minimisation method, allocation concealment is not maintained. A power calculation is missing.

In the previous publication by the same group in 2002[28], the sponsor appointed the writing committee, which interpreted data, wrote the report and submitted it for publication. The risk profiles of the two groups are slightly different with a certain probability of unbalanced risk distribution in favour of the intervention group regarding age and degree of dysphagia.

As the validity of the report is not sufficient, the described effect cannot be considered as clinically relevant.

Meta-analyses

Figure 2 shows the result of the meta-analysis when the three analysed studies were excluded. A total of seven studies were included. One study (Law et al[19]) showed a positive and statistically significant result in favour of the use of preoperative CTx before resection of thoracic esophageal cancer. Six of the included studies were not statistically significant by themselves. The new meta-analysis estimate had a HR of 0.94 with a 95%CI (0.81; 1.09) under assumption of a fixed-effect model and a HR of 0.92 with a 95%CI (0.75; 1.13) under assumption of a mixed-effect model. Regardless of the assumed model, the new estimate does not confirm the advantage of preoperative CTx for resectable thoracic esophageal cancer. The estimate of the original meta-analysis was 0.88 with a 95%CI (0.80; 0.96). The exclusion of the three studies completely changed the result of the meta-analysis. In Boonstra et al[16], only 24.2% of the items on the CONSORT checklist were inappropriately answered, so we assumed that the validity of this study was enough to be included in the meta-analysis. We performed a new meta-analysis excluding only Ychou et al[15] and MRC Allum et al[17] (Figure 3). We found a HR of 0.90 with a 95%CI (0.81; 1.00) under assumption of a fixed-effect model and a HR of 0.90 with a 95%CI (0.78; 1.05) under assumption of a mixed-effect model. Again, regardless of the assumed model, the new estimate does not confirm the advantage of preoperative CTx for resectable thoracic esophageal cancer.

Figure 2 Meta-analysis of seven studies after excluding the three analysed studies. HR: Hazard ratio; N(T): Number of patients in the experimental group; N(C): Number of patients in the control group; W(fixed): Weight assigned to the study by using a fixed-effect model; W(random): Weight assigned to the study by using a random-effect model.

Figure 3 Meta-analysis of eight studies after excluding the studies by Ychou and MRC Allum. HR: Hazard ratio; N(T): Number of patients in the experimental group; N(C): Number of patients in the control group; W(fixed): Weight assigned to the study by using a fixed-effect model; W(random): Weight assigned to the study by using a random-effect model.

Finally, we performed a second meta-analysis (Figure 4) also excluding Law et al[19], which found a positive and statistically significant result as well. After the exclusion of all four studies with positive and statistically significant results, the new meta-analysis consisted of only six statistically non-significant studies. The new meta-analysis estimate was HR 1.04 with a 95%CI (0.88; 1.22), confirming the lack of a survival advantage for patients undergoing preoperative CTx before resection of the thoracic esophageal cancer.

Figure 4 Meta-analysis of six studies after excluding all studies which found a statistically significant survival advantage in the experimental group. HR: hazard ratio; N(T): Number of patients in the experimental group; N(C): Number of patients in the control group; W(fixed): Weight assigned to the study by using a fixed-effect model; W(random): Weight assigned to the study by using a random-effect model.

DISCUSSION

In the present manuscript, we assessed the validity of three studies included in the meta-analysis by Kidane et al[18], which supports the results of improved survival in patients treated with preoperative CTx for resectable thoracic esophageal cancer. It is important to identify possible bias in the three studies which support the result of the meta-analysis because bias jeopardizes validity. We demonstrated that these three studies are not valid enough to be included in a Cochrane review. When excluded from the meta-analysis, the overall result of the meta-analysis is no longer significant.

We will first illustrate the problems we discovered in the three mathematically most influential studies supporting the conclusions and, in a second step, discuss our findings after performing the new meta-analyses.

Common problems in all studies

The lack of a placebo-controlled and blinded study affects the validity of the three studies and, consequently, the validity of the review. As discussed in our previous work[26], without a placebo control, it is impossible to differentiate between specific pharmacological and placebo effects. A placebo effect is defined as the “…response of a subject to a substance or any procedure known to be without specific therapeutic effect for the condition being treated[29].” Several studies demonstrated that perceptual characteristics of drugs[30], the route of administration[31], laboratory tests[32], diagnosis[33] and the doctor-patient relationship play an important role in the outcome of an illness[34-37]. Information regarding treatment or no treatment alone is sufficient to elicit a placebo effect[38]. Moreover, patients’ and doctors’ preferences could also have influenced the results in an open study[39]. Patients assigned to the control group feel disadvantaged because they expect to be treated. Furthermore, when there is no concealment of treatment allocation, the randomisation procedure is compromised because of conscious or subconscious bias[40]. It is important to perform an ITT analysis to maintain the balance distribution of risk factors between groups achieved by a randomisation procedure. A correct ITT analysis was only conducted in the studies by MRC Allum et al[17] and Ychou et al[15]. These aspects collectively affect the validity of the reports and, therefore, the described effects cannot be considered as clinically relevant.

Specific problems of the study by Ychou et al[15]

In the study by Ychou et al[15], a minimisation method is used. Minimisation[41-44], a type of dynamic allocation, is gaining popularity especially in clinical cancer trials. In this design, the new subject’s treatment assignment is determined by evaluating the potential covariate imbalance that would result if he or she were assigned to the treatment or to the control group[45]. Minimisation aims at achieving balance over a large number of prespecified prognostic factors simultaneously. We raise concerns over this design, as it compromises adequate generation of an allocation sequence and concealment in this study. Investigators using minimisation can actually determine the group to which a prospective subject would be allocated and then decide whether this is positive or negative in terms of creating an imbalance in some key predictor of outcome not considered in the imbalance function. Despite adding randomisation, so that the treatment that minimises the imbalance function for a given patient is not necessarily allocated to that patient, there is a high probability of this being the case[46]. The European Medicines Agency’s Committee[47] states that “dynamic allocation is strongly discouraged”.

Specific problems of the study by Boonstra et al[16]

In this study, as in the study by Ychou et al[15], the randomisation process is not exhaustively described; they only mentioned that a central randomisation took place. A description of the randomisation process is completely lacking. Aside from this problem, which is extremely relevant, we find that this study was conducted well in comparison to the other two.

Specific problems of the study by MRC Allum et al[17]

This study reports long follow-up results of a previously published study by the same authors (2002)[28]. As in the study by Ychou et al[15], minimisation was used, raising the same concerns as previously described. A power calculation is completely missing. Finally, a sponsor-related conflict of interest was identified by our analysis.

As recently shown by Shnier et al[48], financial conflicts of interest and relationships between guideline authors and drug companies are common and represent a source of bias in studies. As authoritative value is assigned to guidelines, it is important to develop formal policies to limit the potential influence of any conflict of interest on guideline recommendations. This is the only way to improve the quality of medical publications. Only valid studies are reliable studies. For an expert pool aiming to publish guidelines, it is necessary to scrutinise the validity of single studies and of meta-analyses as well, as low-quality studies can lead to a distortion of the summary-effect estimate[49].

In the second part of our analysis we performed the meta-analysis first without the three analysed studies and showed that the result of the meta-analysis is no longer significant. This result coincides with previous big studies and the original meta-analysis by Malthaner et al[13]. Moreover, as we find that the study by Boonstra et al[16] was quite well done in comparison to the other two, we performed a new meta-analysis excluding only the studies of Ychou et al[15] and MRC Allum et al[17]. The estimate also showed no benefit of preoperative CTx before surgical resection. As expected, when all studies with positive results are eliminated from the meta-analysis, the estimate is not significant.

Implications for practice

According to the results of the Cochrane review, preoperative CTx should be used for patients with resectable thoracic esophageal cancer. However, it is important to note that some of the included trials contain limitations so that definitive assessments of this topic should be delayed until future trials are properly developed. The three analysed studies that were chosen because of their attributed weights are not sufficiently valid to be included in a meta-analysis, which is also true for most of the other studies included.

Despite finding several inconsistencies and substantial deficits in the included high-power studies, the aim of this work is not primarily to identify the best therapeutic treatment for esophageal cancer, but to increase awareness of the quality of studies and their impact on medical treatment when used in meta-analyses or Cochrane reviews. Especially studies that were performed before implementation of the CONSORT checklist show a variety of inconsistencies that would exclude publication according to current quality standards. High-quality RCTs decrease the risk of inherent bias and therefore receive higher attributed weight in meta-analyses. The inclusion of several low-power studies with serious deficits can overpower well conducted studies and change the outcome.

The analysed Cochrane review was published in 2015; only three included studies were performed after 2009, but seven before 2001, some even dating back to before the 1990s. At that time, no standardised reporting procedure, like the CONSORT checklist, existed. Therefore, the findings are quite heterogeneous. The three most powerful studies were the last ones published and still show a substantial lack in standardisation according to the CONSORT checklist, which was first published in 1996 and revised in 2001 and 2010.

As the incidence of esophageal carcinoma is relatively low, studies often include adenocarinoma and squamous-cell carcinoma without discrimination. Even worse, in some of the studies adjuvant treatment was not only CTx, but sometimes also RCTx for squamous-cell carcinoma. Both inherently different carcinoma types with different neoadjuvant treatment regimens were included in a single group. To analyse the role of neoadjuvant CTx in this context, two groups needed to be established: RTx alone vs RCTx as neoadjuvant therapy as performed by Herskovic et al[50]. In this paper, adenocarcinoma and squamous-cell carcinoma of the esophagus were also put into one group.

Multimodale therapy in patients with esophageal cancer is now the standard treatment in most centres today and is recommended in several national guidelines[51-52].

In Germany, S3 guidelines for esophageal carcinoma were updated in 2018[51]. Several newer publications, usually multicentric randomised controlled studies, were taken into account.

The evaluated Cochrane review by Kidane is not mentioned in the current German S3 guideline for the standardised treatment of esophageal carcinoma. However, the analysed studies by Ychou et al[15], Boonstra et al[16] and MRC Allum et al[17] with observed inconsistencies are mentioned and included. Thanks to the authors of the German S3 guideline, the current data is critically presented and not all study results are included in the recommendation for standardised treatment: “In squamous cell carcinoma, no consistent increase in survival after CTx alone – despite the positive study by Boonstra – could be observed by metaanalyses.” (page 101 German S3 guidelines AWMF-Registernummer: 021/023OL).

In conclusion, multimodal therapy of advanced esophageal carcinoma represents the current gold standard for treatment. We observed several deficits of the analysed studies in the Cochrane review by Kidane. Interestingly, this review was not taken into account in the current german S3 guideline for treatment of esophageal carcinoma, and the analyzed single studies are there critically reviewed and set in context with similar research papers. Well performed (multicentric) randomized controlled studies are needed to be analysed together in a meta-analyse. High-quality single studies are required, as they determine the outcome of meta-analyses that can influence the recommendations of national guidelines.