Background and aim In 2013, Diaz-Nieto et al published a Cochrane review to summarise the impact of postsurgical chemotherapy versus surgery alone on survival for resectable gastric cancer. The authors concluded that postsurgical chemotherapy showed an improvement in overall survival. The aim of this article was to assess the validity of four studies included in the Cochrane review and to investigate the impact of an exclusion of these four studies on the result of the meta-analysis.

Methods Overall survival was selected as endpoint of interest. Among the 34 included papers which analysed this endpoint, we identified the four publications which have the highest weights to influence the final result. The validity of these papers was analysed using the CONSORT (Consolidated Standards of Reporting Trials) checklist for randomised controlled trials. We performed a new meta-analysis without the four studies in order to assess their impact on the general result of the original meta-analysis.

Results The analysed four studies revealed several inconsistencies: inappropriate answers were found in up to 77% of the items of the CONSORT checklist. Unclear or inadequate randomisation, missing blinded set-up, conflict of interest and lacking intention-to-treat analysis were the most common findings. When performing a meta-analysis excluding the four criticised studies, postsurgical chemotherapy still showed a significant improvement in overall survival. Even when excluding all single studies with a statistically significant outcome by themselves and performing a meta-analysis on the remaining 26 studies, the result remains statistically significant.

Conclusion The four most powerful publications in the Cochrane review show substantial deficits. We suggest a more critical appraisal regarding the validity of single studies. However, after the exclusion of these four studies, the result of the meta-analysis did not change.