Article Text

Download PDFPDF

Validity of studies suggesting postsurgical chemotherapy for resectable gastric cancer: critical appraisal of randomised trials
  1. Giulia Manzini,
  2. Doris Henne-Bruns,
  3. Michael Kremer
  1. Department of General and Visceral Surgery, Universitat Ulm, Ulm, Germany
  1. Correspondence to Dr Giulia Manzini; giulia.manzini{at}


Background and aim In 2013, Diaz-Nieto et al published a Cochrane review to summarise the impact of postsurgical chemotherapy versus surgery alone on survival for resectable gastric cancer. The authors concluded that postsurgical chemotherapy showed an improvement in overall survival. The aim of this article was to assess the validity of four studies included in the Cochrane review and to investigate the impact of an exclusion of these four studies on the result of the meta-analysis.

Methods Overall survival was selected as endpoint of interest. Among the 34 included papers which analysed this endpoint, we identified the four publications which have the highest weights to influence the final result. The validity of these papers was analysed using the CONSORT (Consolidated Standards of Reporting Trials) checklist for randomised controlled trials. We performed a new meta-analysis without the four studies in order to assess their impact on the general result of the original meta-analysis.

Results The analysed four studies revealed several inconsistencies: inappropriate answers were found in up to 77% of the items of the CONSORT checklist. Unclear or inadequate randomisation, missing blinded set-up, conflict of interest and lacking intention-to-treat analysis were the most common findings. When performing a meta-analysis excluding the four criticised studies, postsurgical chemotherapy still showed a significant improvement in overall survival. Even when excluding all single studies with a statistically significant outcome by themselves and performing a meta-analysis on the remaining 26 studies, the result remains statistically significant.

Conclusion The four most powerful publications in the Cochrane review show substantial deficits. We suggest a more critical appraisal regarding the validity of single studies. However, after the exclusion of these four studies, the result of the meta-analysis did not change.

  • gastric cancer
  • assessment of validity
  • placebo effect
  • meta-analysis

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • Contributors GM, DHB and MK contributed substantially to conception and design of the study. GM and MK contributed to analysis and interpretation of the data and drafted the article. GM performed the meta-analysis. All authors gave the final approval of the version to be published.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.