Background Circulating tumour DNA from colorectal cancer (CRC) is a biomarker for early detection of the disease and therefore potentially useful for screening. One such biomarker is the methylated SEPT9 (mSEPT9) gene, which occurs during CRC tumourigenesis. This systematic review and meta-analysis aims to establish the sensitivity, specificity and accuracy of mSEPT9 tests for the early diagnosis of CRC.
Methods A systematic search of the relevant literature was conducted using Medline and Embase databases. Data were extracted from the eligible studies and analysed to estimate pooled sensitivity, specificity and diagnostic test accuracy.
Results Based on 19 studies, the pooled estimates (and 95% CIs) for mSEPT9 to detect CRC were: sensitivity 69% (62–75); specificity 92% (89–95); positive likelihood ratio 9.1 (6.1–13.8); negative likelihood ratio 0.34 (0.27–0.42); diagnostic OR 27 (15–48) and area under the curve 0.89 (0.86–0.91). The test has a positive predictive value of 2.6% and negative predictive value of 99.9% in an average risk population (0.3% CRC prevalence), and 9.5% (positive predictive value) and 99.6% (negative predictive value) in a high-risk population (1.2% CRC prevalence).
Conclusion The mSEPT9 test has high specificity and moderate sensitivity for CRC and is therefore a potential alternative screening method for those declining faecal immunochemical test for occult blood (FIT) or other screening modalities. However, it is limited by its poor diagnostic performance for precancerous lesions (advanced adenomas and polyps) and its relatively high costs, and little is known about its acceptability to those declining to use the FIT.
- colorectal cancer
- cancer epidemiology
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What is already known about this subject?
The methylation of the SEPT9 (mSEPT9) test has shown promise as a screening test for early detection of colorectal cancers.
What are the new findings?
This updated meta-analysis estimates the sensitivity to be 69% and specificity of 92% and adds to current evidence by including new studies, evaluating the covariate effects and how these interact with pooled estimates of diagnostic performance.
While diagnostic performance seems promising for colorectal cancers, the mSEPT9 test has poorer performance for detecting precancerous lesions (advanced adenomas and polyps) and is more expensive.
How might it impact on clinical practice in the forseeable future?
The clinically relevant findings of this study posits the potential use of an alternative screening method for those that decline the faecal immunochemical test (FIT) to provide cover to individuals who would otherwise miss out on the benefits offered by colorectal cancer screening.
Further studies are needed to test whether mSEPT9 is acceptable for those that decline the FIT and whether this strategy would be cost-effective.
Colorectal cancer (CRC) survival is highly dependent on stage and therefore methods for early detection are critical to reduce mortality. In the USA, only 40% of CRC cases are diagnosed at an early stage; the 5-year relative survival for stage 1 is 89.9%, stage 2 is 71.1%, stage 3–4 is 14.2%.1
Liquid biopsy screening tests are blood-based tests for early detection of cancer. They examine peripheral blood for circulating tumour DNA (ctDNA), which can be present in the earliest stages of tumourigenesis.2 ctDNA is released into the bloodstream from malignant cells during apoptosis. These fragments of DNA are representative of cancer cells of the tumour. Commonly, the DNA of tumour cells and therefore the ctDNA is methylated at the 5’-Cytosine-phosphate-Guanine-3’ sites.2 Therefore, detection of methylated ctDNA is used as a biomarker for the detection of asymptomatic cancer including CRC.
For the early detection of CRC, methylation of the SEPT9 (mSEPT9) gene located on chromosome 17 at q25.33 has shown promise as a ctDNA biomarker as CRC tumours have an increased likelihood of exhibiting methylation of the promoter region of the SEPT9 gene.3 For this test, a purified DNA sample obtained from peripheral blood is evaluated using duplex real-time PCR to identify methylation in the v2 region of the SEPT9 gene,3 which is suggestive of CRC.
A previous meta-analysis of diagnostic performance of mSEPT94 reported estimates of the sensitivity of mSEPT9 for CRC ranging from 37% up to 88%. However, most of the studies within this meta-analysis have not assessed the effect of covariates on sensitivity and specificity (such as the study sample size, region of recruitment of participants, total number of CRC cases and the distribution of stage). Additionally, four studies included in this meta-analysis were assessed as having a high probability of patient selection bias and could not include research reported since 2017. Two other meta-analyses5 6 did not conduct a comprehensive analysis of diagnostic accuracy of mSEPT9 tests: the first study5 considered covariates to assess sources of heterogeneity but provided limited evidence of the effect of those covariates on the diagnostic efficacy of mSEPT9. The second study6 only provided estimates on sensitivity and specificity. Finally, few studies have estimated the accuracy of the newer and commonly used mSEPT9 test, Epi-ProColon V.2.0 to the commonly used screening method for CRC, the faecal immunochemical tests (FIT), thereby limiting understanding of its clinical applicability. Only one previous meta-analysis4 assessed the impact of the threshold level for the number of PCR results that constitute a positive test, which is a major issue affecting the accuracy of mSEPT9 test for CRC.7
The aim of this systematic review was to conduct an updated meta-analysis stratified by the region of recruitment, sample size and stage, and to evaluate the accuracy of a range of positivity thresholds for the mSEPT9 test.
This systematic review and meta-analysis was conducted as per Guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis.8
Medline and Embase databases were searched for relevant literature. The search was divided into three phases. All articles included in these databases from January 1946 to May 2018 were considered. The search terms for the first phase were: (‘colon’ or ‘colorectal’) and (‘cancer’ or ‘carcinoma’) and (‘serum’ or ‘plasma’) and ‘sensitivity’ and ‘specificity’. The search terms for the second phase were: (‘SEPT9’ or ‘SEPT9’ or ‘Septin-9’ or ‘mSEPT9’) and (‘colon’ or ‘colorectal’) and (‘methylated’ or ‘methylation’). The search terms for the third phase were: ‘Epi procolon’ and ‘SEPT9’ and ‘diagnosis’. The articles from all three phases were combined and duplicate articles removed. A detailed search strategy is provided in online supplementary file 1.
The inclusion criteria for the papers were: full-text available, English language, human studies, case-control study design, CRC diagnosis either prior or following mSEPT9 test, report of sufficient data to enable construction of 2×2 tables of mSEPT9 result and CRC status, and the use of a mSEPT9 single gene assay for both cases and controls. Exclusion criteria were: articles that were meta-analysis, conference and meeting abstracts, studies that used a multigene ctDNA panel; and studies with <20 CRC case participants.
Endnote X9 was used to manage citations. Each article was assigned to one or more of the following reference categories: ‘articles with SEPT9 information’, ‘potential candidates for meta-analysis’, ‘irrelevant to this study’ and ‘selected studies’ by the following method. Articles that matched the inclusion criteria, identified by the three search phases were assigned to ‘article with SEPT9 information’. The abstracts of these papers were scanned for validity. Those meeting all inclusion criteria were assigned to ‘potential candidates for meta-analysis’. The full-text of these articles were evaluated, and the applicable studies for the meta-analysis were assigned to ‘selected studies’. Articles such as meeting abstracts, conference notes, meta-analysis and those studies which matched one exclusion criteria were assigned to ‘irrelevant for this study’.
Quality assessment of individual studies
The two authors (RH and MJ) independently carried out quality assessment of each study to assess risk of bias and applicability concerns using the Quality Assessment of Diagnostic Accuracy Studies V.2.0 tool.9 Risk of bias and applicability concerns of the study methodology was assessed under four domains: ‘patient selection’, ‘index test (mSEPT9)’, ‘reference standard (histopathology or colonoscopy to define CRC)’. The ‘flow and timing’ domain was omitted when assessing ‘applicability concerns’ as instructed by the tool. Each domain comprised two or three signalling questions, for which each reviewer answered ‘yes’, ‘no’ or ‘unclear’. Based on this, each study was graded ‘high’, ‘low’ or ‘unclear’ in each of the four domains, and the assessment results were then graphed (see online supplementary table 1, online supplementary figure 1). Inter-rater reliability/agreeability was measured using Cohen’s kappa statistic.
From each of the articles, the following results were abstracted: the number of true positives (tp)=participants who had CRC at the time of testing and were positive for the mSEPT9 test; true negatives (tn)=participants who did not have CRC at the time of testing and were negative for the mSEPT9 test; false positives (fp)=participants who did not have CRC and were positive for the mSEPT9 test and false negatives (fn)=participants who had CRC and were negative for the mSEPT9 test. All data were extracted by the author of this systematic review and was doubled checked to ensure accuracy.
Calculations of the following measures of accuracy of mSEPT9 were conducted: sensitivity [tp/(tp+fn)], specificity [tn/(tn+fp)], positive likelihood ratio [sensitivity/(1−specificity)], negative likelihood ratio [(1−sensitivity)/specificity] and diagnostic ORs [(tp/fn)/(fp/tn)] as well as their respective 95% CIs. As a secondary measure, positive predictive value (PPV) (tp/tp+fp) and negative predictive value (NPV) (tn/tn+fn) was also calculated assuming a CRC prevalence of 0.3% for an average risk population (consistent with having no first-degree relatives with CRC) and a CRC prevalence of 1.2% for a high-risk population (consistent with having two first-degree relatives with CRC). The range of threshold levels signifying a positive test were considered using either “one positive count out of three PCR results (1/3 algorithm)’ or ‘two positive counts out of three PCRs (2/3 algorithm)’,7 for the stratified subgroup analysis and meta-regression.
To investigate the relationship between performance indicators and the covariates that affect them, we used mixed effect bivariate and hierarchical statistical models.10 We estimated accuracy of diagnostic tests, how this accuracy varies with alterations in clinical and methodological practices and the comparison of accuracy of two or more tests.10
Forest plots were synthesised for the pooled sensitivity and specificity estimates. A stratified analysis (and subgroup meta-regression) was conducted based on country of recruitment, sample size, recruiting more stage 1 over stage 4 CRC participants and threshold algorithm. Heterogeneity from the pooled estimates of sensitivity and specificity was evaluated via the I2 statistic. To determine whether varying threshold levels to qualify as a positive test was a potential source for heterogeneity, an overall Spearman’s coefficient was estimated between sensitivity and specificity and respectively for studies sorted by region of recruitment and CRC case sample sizes (online supplementary table 2). A positive correlation between sensitivity and specificity is indicative of heterogeneity due to varying thresholds for positivity chosen for each study.
A hierarchical summary operator receiver curve (HSROC) model was used to evaluate the effects of covariates simultaneously on the pooled estimates of mSEPT9.10 The HSROC plot provides estimates of diagnostic performance around a summary point and is suited to evaluate the effects on accuracy and positive test cut-off thresholds. The area under the curve (AUC) of the HSROC was calculated to provide evidence for diagnostic validity of the mSEPT9 tests. Pooled positive likelihood ratio and pooled negative likelihood ratio was used to assess diagnostic test accuracy. A high pooled positive likelihood ratio combined with a low pooled negative likelihood ratio provides evidence of validation of mSEPT9 tests as an early detection test for CRC. To illustrate the performance of the test in terms of predicting presence of CRC, a Fagan Nomogram was generated to convert a pretest probability of CRC to post-test probability of CRC depending on whether the mSEPT9 test was positive or negative. All statistical analysis were computed with STATA V.13.1.
Based on the selected search terms 287 articles were identified, of which 20 were duplicates, leaving a total of 267 (figure 1). Of these, 239 studies were excluded after screening the abstracts. The full text of the 28 papers selected were read and the references from each article screened for other eligible studies. From the pool of 28 eligible studies, 11 studies were excluded (2 meta-analyses, 3 insufficient information to produce and 6 reviews). In the references of the eligible studies, 2 more eligible studies were identified, leaving a final total of 19 eligible studies (table 1).
Quality assessment of individual studies
Of the 19 studies,127 12–22 were screening studies (blood sample for mSEPT9 test taken prior to CRC diagnosis) and 723–29 were clinical studies (blood sample for mSEPT9 test after CRC diagnosis). As the mSEPT9 test is being evaluated for its screening ability, the majority being screening studies is a strength of this meta-analysis. Fifteen studies7 12–19 21–24 26 27 collected plasma samples from participants for the mSEPT9 test prior to starting neoadjuvant therapy, adjuvant chemotherapy, radiotherapy or surgical resection and 17 studies,7 12–24 26 27 29 confirmed CRC status by gold standard histopathology.
Three studies were assessed as having a high risk of bias due to patient selection; six studies were assessed as having ‘unclear’ risk of bias due to the conduct of the index test and seven studies were assessed as ‘unclear’ risk of bias due to flow and timing. Five studies were assessed as having ‘high’ applicability concerns due to patient selection, and three studies were assessed as having ‘high’ applicability concerns due to the index test. One study was assessed as having high applicability concerns and two ‘unclear’ for the reference standard domain. There was good inter-rater agreeability for risk of bias (k=0.75) and excellent agreeability (k=1) for applicability concerns between the quality assessors.
The 19 selected studies had a total of 7629 cases and controls from 7 countries, in North America, Europe and East/South-East Asia. All CRC cases were diagnosed and staged before treatment and controls were individuals with no evidence of CRC, most (94.3%) confirmed by prior colonoscopy.
The pooled sensitivity of mSEPT9 in detecting CRC was 0.69 (95% CI 0.62 to 0.75) and the pooled specificity was 0.92 (95% CI 0.89 to 0.95) (table 2). There was considerable heterogeneity for both sensitivity (I2=88.37%) and specificity (I2=96.29%) (figure 2). The pooled positive likelihood ratio of 9.1 (95% CI 6.1 to 13.8) indicates that CRC cases were ninefold more likely to have a positive mSEPT9 test compared with controls (table 2). The negative likelihood ratio of 0.33 (95% CI 0.22 to 0.50) indicates that CRC cases were one-third less likely to have a negative mSEPT9 test compared with controls (table 2). The overall pooled diagnostic OR for mSEPT9 was 27 (95% CI 15 to 48)(table 2), which indicates that a case had 27-fold greater odds of being positive than a control, or to put it another way, a positive mSEPT9 result was associated with a 27-fold increase in odds of the person having CRC. Pooled specificity and positive likelihood ratio of North American studies was lower compared with Asian and European studies (p<0.001).
The 2/3 algorithm for positivity threshold appeared to have the same sensitivity as the 1/3 threshold (approximately 0.69), but had a higher specificity and thereby a higher positive likelihood ratio and diagnostic OR, with a lower negative likelihood ratio. Sensitivity and specificity appeared similar for large (>100 participants) and small (<100 participants) studies.
The Spearman’s correlation coefficient for all studies was 0.19 (p=0.4) indicating no evidence that the heterogeneity was due to differences in the definition of threshold used by each study (online supplementary table 2) or of a monotonic relationship between sensitivity and specificity. The effect investigated here was the algorithm chosen by the study as 2/3 PCR results. While some studies clearly stated the use of the 1/3 algorithm, other studies which did not explicitly state this were considered within the 1/3 PCR result subgroup, as this is the minimum threshold to establish a positive result (online supplementary table 2). There was some evidence that the threshold used contributed to the heterogeneity of Asian studies (Spearman’s coefficient 0.793, p=0.02), but no such evidence was observed for European or North American studies or large or small studies (online supplementary table 2).
There was marginal evidence (p=0.06) for the heterogeneity in sensitivity being attributable to differences in sensitivity in North American studies and studies that recruited more cases with stage 1 CRC than cases with stage 4 CRC (p=0.06) (see online supplementary table 3, online supplementary figure 2). There was strong evidence (p<0.01) that the heterogeneity in specificity was largely affected by size of study, and whether it was conducted in Asian (p<0.01) or North American (p<0.001) regions. Studies with a high proportion of stage 1 CRCs had a lower sensitivity (p=0.06) than those with a high proportion of stage 4 CRCs (see online supplementary table 3, online supplementary figure 2).
There was marginal evidence of publication bias (p=0.05) (see online supplementary figure 3).
Hierarchical summary operator receiver curve
The 95% prediction contour of the HSROC encompassed all eligible studies indicating the likely region of sensitivity and specificity of mSEPT9 (figure 3) . The AUC of 0.89 (95% CI 0.86 to 0.91) confirms the mSEPT9 test as being diagnostically valid with moderate-to-high accuracy. The β-coefficient of the HSROC, which describes a symmetrical distribution of data points (studies), is not significantly different from zero (0.37; 95% CI −0.25 to 0.79) and therefore the lambda coefficient is a reasonable measure of the test’s accuracy; 3.01 (95% CI 2.45 to 3.73). The theta coefficient of −0.65 (95% CI –1.09 to –0.202) represents the positivity threshold cut-off on a log scale (0.522 on the linear scale).
The sensitivity and specificity was more efficient for the positivity threshold chosen in the Asian and European studies compared with the North American studies (see online supplementary figure 4a–c). The diagnostic performance for European studies was similar to Asian studies, both with AUC 0.96 (95% CI 0.92 to 0.97), which were higher compared with North American studies 0.80 (95% CI 0.76 to 0.83).
Visual assessment of pretest and post-test probability and likelihood ratios
The summary points from this meta-analysis (pooled positive likelihood ratio=9.4 and cut-off exclusion level or negative likelihood ratio=0.3) falls just below the cut-off confirmation of diagnostic validity, defined as a test with a positive likelihood ratio >10 (confirmatory role) and negative likelihood ratio of <0.1 (exclusory role) (see online supplementary figure 5a–c).
Positive and negative predictive values
A mSEPT9 test with sensitivity of 69% and specificity of 92% would accurately diagnose 21 of 30 CRC cases present in a sample of 10 000 people at average risk. For this sample, the PPV is 2.6% and NPV is 99.9%.
For the same sensitivity and specificity, a mSEPT9 test would accurately diagnose 83 of 120 CRC cases in a sample of 10 000 people at high risk. For this sample, the PPV is 9.5% and NPV is 99.6%.
In this meta-analysis, mSEPT9 has an estimated sensitivity of 0.69 (95% CI 0.62 to 0.75) and specificity of 0.92 (95% CI 0.89 to 0.95), AUC of 0.89 (95% CI 0.86 to 0.91) and positive and negative likelihood ratios of 9.1 and 0.33, respectively. For a screened population of 10 000 people, of which 0.3% (30 cases) have CRC, that is, typical of an average risk population aged 50–74 years, 21 of the 30 CRC cases will be detected by the mSEPT9 test and 9172 of the 9970 healthy individuals would be correctly classified by the test as not having CRC. A person who tests positive for the mSEPT9 test has a 2.6% chance of having CRC (PPV). A negative mSEPT9 test virtually rules out CRC with an NPV of 99.9%. For a population at high risk, for example, with a strong family history putting them at four-fold increased risk, the corresponding PPV and NPV would be 9.5% and 99.6%.
A previous meta-analysis of mSEPT9 in 2017 (which included 14 of the 19 studies we included in this meta-analysis) reported a pooled sensitivity of 0.71, specificity of 0.92 and AUC 0.88, which are in line with previous diagnostic test performance indicators.4 Previous meta-analyses reported that estimates of sensitivity and specificity were highly heterogeneous, probably due to differences in test performance by geographic region of recruitment of participants. We also observed marginal evidence that some of heterogeneity could be due to varying ctDNA positivity thresholds inferring that test accuracy depended on the positivity algorithm, with the optimum threshold level being the 2/3 algorithm. The North American studies had overall lower pooled diagnostic accuracy indicators potentially due to different positivity threshold used in that region. As reported by Nian et al,4 we found marginal evidence for the sensitivity being lower for studies with a higher proportion of early stage disease is expected given earlier stage CRC would be expected to produce lower amounts of circulating mSEPT9 given fewer cancer cells at the site of the original tumour and no metastases.
Limitations of this meta-analysis included inability to investigate other non-threshold effects on heterogeneity or variations in PCR methodology and technology.
Many CRC screening programmes use the FIT to detect occult blood in faeces. The FIT has a sensitivity of 79% (95% CI 69 to 86) and specificity of 94% (95% CI 92 to 95),30 that is, is more sensitive and specific than mSEPT9. For a screened population of 10 000 people, at average CRC risk, 24 of the 30 CRC cases will be detected by FIT and 9372 of the 9970 healthy individuals would be correctly classified by FIT as not having CRC. The PPV for this population would be 3.9% and NPV would be 99.9%. In the same screened population, for individuals at high risk of CRC, the corresponding PPV and NPV would be 13.8% and 99.7%. Therefore, on a purely performance-based comparison, mSEPT9 is not a replacement test for FIT. However, as with all cancer screening programmes, there is incomplete uptake of screening by FIT, particularly so for some countries where less than half of the eligible population are participating in population-based organised programmes, for example, Australia, France, Ireland, Italy, Republic of Korea and Singapore.31 One reason (of many) for this non-participation being some people find talking about and collecting/storing faecal samples unpleasant and embarrassing.32 For these people, a blood test may be more appealing and result in higher participation. To support this strategy, studies need to be done on decliners of FIT screening to determine their uptake in an alternative screen via the mSEPT9 test. Such studies would indicate whether decliners of the FIT might be amenable to an alternative test based on a blood sampled test such as mSEPT9, and therefore mSEPT9 might be a viable alternative screening test for those who would otherwise miss out on the protection that CRC screening affords. A recent study reported that 94% of average risk patients who declined a FIT test, opted for mSEPT9 testing; however, the study sample only included people who attended a screening clinic and therefore does not represent all decliners of FIT screening.33 One argument against the strategy of offering mSEPT9 as an alternative to FIT screening is the high cost of the mSEPT9 test (€15015), which means that screening an average risk population via mSEPT9 is twice as expensive as screening via FIT, even under the assumption of equivalent sensitivity and specificity of the tests.34 Another argument against this strategy is that mSEPT9 might not be sensitive for precancerous lesions: advanced adenomas (15%) and polyps (5%).4 Because of these limitations of mSEPT9, investment in strategies to facilitate participation in FIT remains important.
This analysis of data from 19 studies finds that the mSEPT9 test has potential for a population screening of CRC with the ability to detect approximately two-thirds of CRC cases and ruling out 92% of non-cases. While mSEPT9 has diagnostic performance similar (although lower) to the FIT, it is currently unsuitable as its replacement in screening programmes. However, it may be a valid alternative to the FIT, when considering those unwilling to participate in the home faecal test. Further screening studies are required to assess the uptake of mSEPT9 tests when offered to FIT decliners in lieu of FIT as well as an economic evaluation that includes the costs of the test.
Contributors RH conceptualised the study, carried out literature search and selected the studies, provided quality assessment, extraction of data, conducted data analysis, conceptualised the design and draft of the manuscript and approved the final manuscript. MJ contributed to the conception of the paper, supported in selection of papers and provided quality assessment, supported in data analysis and design and draft of the manuscript, proofread and approved the final manuscript.
Funding This study was funded by National Health and Medical Research Council (Grant number: APP1117611).
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information. All data for this analysis are included in the article or uploaded supplementary material.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.