Article Text
PDF
Abstract
Objective To identify the optimal incentive protocol for maximising participation while managing study costs during the Voyage trial.
Design Prospective cohort (Voyage trial) of colorectal cancer (CRC) incidence and mortality outcomes in individuals screened with multitarget stool DNA (mt-sDNA) served as the population. A subset was randomised to receive postage stamps as a pre-consent incentive, or as a post-consent incentive after completion of the consent and questionnaire. Descriptive statistics from year 1 are reported.
Results During year 1 of the Voyage trial, a total of 600 258 individuals with mt-sDNA orders received at Exact Sciences Laboratories were randomly selected and invited to participate. Of those, 26 429 (4.4%) opted in, 14 365 of whom (54.3%) consented. The opt-in and consent samples were similar to the target population with respect to sex but differed by geographic residence and age (p<0.001). For the embedded incentive experiment, 2333 were randomised to the pre-incentive arm, while 2342 were randomised to the post-incentive arm. Overall consent rate in the incentive trial was 56.4% (60.9% for the pre-consent incentive arm (1421/2333) vs 52.0% for the post-consent incentive arm (1217/2342), p<0.001). Cost reduction was observed for the pre-consent incentive group, and higher response rates were seen among older versus younger individuals.
Conclusions Pre-consent incentive option was associated with a higher participation rate and lower costs and was used for the remainder of study recruitment. CRC incidence and mortality vary with age; thus, adjusting for differential participation by age and region will be important in analyses of Voyage data.
Trial registration number NCT04124406.
- COLORECTAL CANCER
- COLORECTAL CANCER SCREENING
- STOOL MARKERS
Data availability statement
Data are available on reasonable request to the corresponding author.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
When considering implementing an incentive, the question is often how to do so in a cost-effective manner. The Voyage Study is a collaboration between Mayo Clinic and Exact Sciences Laboratories (ESL), with some recruitment activities managed by ESL and others by Mayo Clinic. This study was necessary to identify the optimal incentive strategy given the recruitment workflow that is specific to the Voyage Study.
WHAT THIS STUDY ADDS
We determined that providing the incentive at the time of mailing the consent form and baseline survey was more effective than mailing an incentive after receiving a completed survey and research consent form, with respect to both increasing participation rate while minimising cost. By reviewing the year 1 data, we also identified groups of individuals who are less likely to participate in the Voyage Study.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This manuscript can help researchers by illustrating how to plan and conduct an investigation to identify the optimal recruitment and incentive protocol for their studies.
Introduction
Colorectal cancer (CRC) is the third most commonly diagnosed cancer and the third leading cause of cancer death in the USA,1 with an estimated 152 810 diagnoses and 53 010 fatal cases in 2024.2 Regular screening and early detection reduces CRC incidence and improves overall survival.3 4 However, in 2018, just under 70% of US adults were up to date with recommended CRC screening,5 falling short of the CRC screening goal of 80% adherence set forth by the National Colorectal Cancer Roundtable.6
Self-administered CRC screening test options have the potential to improve screening rates by reducing barriers associated with bowel preparation and office visits. The multitarget stool DNA assay4 7 8 (mt-sDNA; Exact Sciences Corporation, Madison, Wisconsin, USA) is one such test.9 10 Previous studies have elucidated the sensitivity and specificity performance characteristics of mt-sDNA in the detection of colorectal neoplasia and advanced precancerous lesions11 12; however, long-term outcomes and the real-world impact of mt-sDNA screening on CRC incidence and mortality have yet to be investigated.
To address this gap, Mayo Clinic and Exact Sciences Laboratories (ESL) are collaborating on the Voyage Study, a prospective cohort study of CRC outcomes in individuals with mt-sDNA screening test orders.13 The Voyage Study aims to enrol 150 000 participants to accurately estimate CRC risk factors and outcomes in persons screened with mt-sDNA testing. Through periodic contacts, the consented participants were prospectively followed to track health outcomes. It is necessary to understand the characteristics and behaviour of the overall cohort to enable adjustment for potential non-response bias in future analyses and improve participation rates.14–17 Therefore, in this study, we addressed the following three questions1: Are those who opt in to receive more information about the Voyage Study representative of all individuals who received a mt-sDNA screening test order?2; are those who consent to participate in the Voyage Study representative of all individuals who received a mt-sDNA screening test order?3; and which strategy is more cost-effective at increasing participation, incentive included in initial survey packet (pre-incentive) or sent after survey completion (post-incentive)? Herein, we describe the methodological approach used to evaluate the representativeness of participants of the sampling frame population and to evaluate the impact of incentive strategies on response rates in year 1 of the Voyage Study.
Methods
The Voyage Study13 was developed to review the real-world impact of mt-sDNA on long-term clinical outcomes, such as CRC incidence and mortality, while examining use of CRC-related outcomes over time in 150 000 participants (NCT04124406). Recruitment of participants began on 17 October 17 2019, and enrolment ended on 17 October 2023. A detailed description of the methodological approach has been published elsewhere.13
Design, study population and data acquisition
The Voyage cohort was prospectively designed to enrol persons who had mt-sDNA for CRC screening ordered by a healthcare provider. All participants must reside within the 50 states of the USA or Puerto Rico. Individuals with mt-sDNA orders received at ESL (‘target population’) were randomly selected on a weekly basis and invited to participate in the Voyage Study through an initial welcome letter mailed by ESL. The welcome letter included instructions to return an opt-in form to Voyage Study research staff at Mayo Clinic. The opt-in form included a bar code embedded with the name and address of the person who received the mt-sDNA order. Interested individuals returned the opt-in form to the Mayo Clinic study team, and those who opted in received a consent packet which contained a cover letter, consent form, baseline enrolment questionnaire and a postage-paid return envelope. Individuals who did not return a completed consent form and enrolment questionnaire within 30 days of the initial packet were mailed a reminder packet. No further attempts to enrol the individual were made after the reminder packet.
Incentive options and protocols for delivery were constrained by what was feasible in a study of this scale. Cash and gift card incentives were not allowed within the requirements of Mayo Clinic’s research participant remuneration policy. Token gifts such as a pen or notepad branded with a Mayo Clinic or Exact Sciences Corporation logo are bulky and increase postage costs and were also not feasible. Given these constraints, we chose to offer a book of 20 first class ‘forever’ postage stamps (current approximate value of US$11) as the incentive as it was easy to add to the consent packet and would not increase the packet weight. The value of the incentive selected is commensurate with values reported in the literature.18 19
During the initial rollout phase (from 6 November 2019 to 11 February 2020), individuals who returned both a completed consent and questionnaire were subsequently mailed a book of stamps. An embedded incentive experiment was then implemented to determine the optimal survey administration protocol that would maximise consent rates. As part of this phase, individuals who returned the opt-in form to the Mayo Clinic study team between 11 February 2020 and 9 June 2020 were randomised into two arms, in equal numbers. Individuals randomised to the pre-incentive arm received a book of stamps with the initial consent packet, whereas individuals randomised to the post-incentive arm were sent their incentive after the completed questionnaire and consent form were returned.
The cost of print materials and postage was calculated for each incentive group. Response rates were used to extrapolate future cost for the entire cohort and to determine whether offering a pre-incentive increased the consent rate enough to offset the initial cost of the incentive by reducing overall printing and mailing costs.
Statistical analysis
To address the research questions regarding whether individuals who opt in or give consent are representative of all individuals who received a mt-sDNA screening test order, we assembled three cohorts for comparison. The ‘target population’ consisted of all randomly selected individuals who received an mt-sDNA order and were mailed an opt-in form from ESL from 17 October 2019 to 30 September 2020. From the target population, the ‘opt-in sample’ was considered as those who returned the interest form to the study team at Mayo Clinic by 31 December 2021. Individuals who returned a completed enrolment questionnaire and a signed consent form by 31 December 2021 were considered as the ‘consent sample’. Aggregated counts for the ‘target population’ were provided by ESL separately by state, sex and age groups (<50, 50–54, 55–59, 60–64, 65–69, 70–74, 75, 79 and 80 years and older). Geographic regions of the USA were then categorised according to the US Census regions as either Midwest, Northeast, West and South, not including Puerto Rico. Distributions of age group, sex and geographic region for the opt-in and consent samples were compared with those of the target population using a one-sample χ2 test.
Consent rate was determined separately for the initial rollout phase and for the two arms in the embedded experiment phase. For each phase, analyses were restricted to individuals who opted in within 180 days of being sent the invitation letter. Consent rate was calculated as the percentage who returned a completed enrolment questionnaire and a signed consent form within 180 days of opting in. Among the individuals included in the embedded experiment phase, the consent rate was compared between the pre-incentive and post-incentive arms using a χ2 test. To assess whether the incentive approach influenced the characteristics of consent sample, we compared the distributions of sex, geographic region and age in the pre-incentive and post-incentive consent samples from the embedded trial using a χ2 test. Statistical analysis was performed using SAS V.9.4 (SAS Institute). All calculated p values were two-sided.
The cost of each individual opt-in form and each consent packet was calculated and used to project the total cost for all mailings required to reach the target cohort populate for each incentive group. For the pre-incentive group, the value of sent incentives that did not return a consent form was calculated and extrapolated to the full cohort. The overall costs of the two groups were compared to determine which was most cost-effective.
The parent study was powered to assess the impact of mt-sDNA on CRC-related incidence and mortality.13 No a priori sample size was established for the embedded incentive trial. Rather, the pre-incentive and post-incentive approaches were implemented concurrently for 4 months. At 4 months, the response rates for the two arms were calculated and determined to be statistically significantly different and the optimal incentive approach was identified; thus, the embedded trial was halted.
Results
During year 1 of the Voyage Study, 600 258 individuals were randomly selected from individuals with mt-sDNA orders received at ESL and invited to participate (target population). Within the target population, 59.2% were women. The largest 5-year age group consisted of those 50–54 years of age (21.8%), and 42.3% of the orders were for individuals residing in states in the South. Of those randomly selected, 26 429 (4.4%) opted in (opt-in sample). Of the individuals who opted in, 14 365 consented (consent sample). A comparison of the cohorts (target population, opt-in sample and consent sample) by sex, age and geographic region is presented in table 1. The opt-in and consent samples were similar to the target population with respect to sex, each with 59%–60% women. Although the distribution of both age and geographic region in the consent sample mirrored that of the opt-in sample, these distributions differed from the target population (p<0.001). In terms of the geographic region, 36.3% of the consent sample resided in the South compared with 42.3% of the target population. By contrast, there was an over-representation in the consent sample of individuals from the West (17.6% vs 14.1%) and Midwest (29.2% vs 26.2%). Compared with the target population, younger individuals were under-represented in the consent sample (aged 50–54 years, 16.6% vs 21.8%; aged 55–59, 12.1% vs 17.0%), whereas older individuals were over-represented in the consent sample (aged 65–69, 21.2% vs 18.3%; aged 70–74, 17.4% vs 13.5%; and aged 75–79, 9.6% vs 6.2%).
Representativeness of opt-in and consent samples during the 1st year of the Voyage Study
A comparison of the pre-incentive and post-incentive consent samples by sex, geographic region and age is presented in table 2. No statistically significant differences were observed across the two incentive approaches.
Characteristics of embedded trial participants consented through pre-incentive and post-incentive approaches
The consent rate during the initial rollout phase was 52.0% (903/1738). During the next 4 months, all individuals who returned their opt-in form were included in the embedded incentive experiment; 2333 were randomised to the pre-incentive arm, while 2342 were randomised to the post-incentive arm. The numbers for each group are unequal due to days when the total was an odd number, one more subject was randomised on those odd days to the post-consent incentive arm than to the pre-consent incentive arm; thus, one group wound up with slightly more than the other. The overall consent rate observed in the embedded incentive trial was 56.4%, with the higher consent rate observed in the pre-incentive versus the post-incentive arm (60.9% vs 52.0%, p<0.001) (figure 1). Differences in consent rates between the pre-incentive and post-incentive groups were also observed following the initial mailing of the consent packet with consent rates of 42.4% versus 34.1%, respectively (p<0.001). Among the non-respondents who were mailed a reminder packet after failing to respond within 30 days of the initial consent packet, the consent rates for the pre-incentive and post-incentive groups were 33.2% and 27.5%, respectively (p=0.001).
Embedded randomised incentive experiment flow diagram. mt-sDNA, multitarget stool DNA.
A 2.5% reduction in cost was calculated in the pre-incentive group compared with the post-incentive group for printing, postage and incentive value. The study team did not further calculate the cost differential for personnel effort as they were confident it would be higher in the post-incentive arm and would only further strengthen the conclusion that the pre-incentive approach was optimal.
Discussion
The ambitious Voyage Study, with a target enrolment of 150 000 individuals, is a seminal study on the effectiveness of mt-sDNA screening with respect to CRC incidence and mortality reduction. As with any large epidemiologic study, understanding the study population is critical for appropriate interpretation of outcomes. The choice to participate in research is largely driven by the salience of the research objectives to the prospective participant and the perceived opportunity cost associated with participation.20 While these vary by study, systematic reviews have shown that participation in survey-based epidemiologic research can be associated with demographic characteristics.21 Of particular interest when interpreting results from a prospective cohort study is whether characteristics that are associated with participation are also associated with the outcomes of interest.
Using data from year 1 of the Voyage Study, we compared demographic characteristics of the opt-in and consent samples to the target population. The distribution of men and women was the same across the opt-in sample, consent sample and target population. The opt-in and consent samples were similar to each other but differed from the target population with respect to age and region of residence. We observed higher response rates among older versus younger individuals in the consent sample, which reflects a pattern previously reported in some healthcare-related survey research.22–25 As CRC incidence and mortality vary with age,26 adjusting for differential response by age will be important in analyses of Voyage data. Regional differences in opt-in and consent rates, such as lower participation rates in the South, are consistent with patterns observed in US Census participation.5 Age-adjusted incidence rates for CRC vary by region of the USA27 28; thus, careful adjustment for differential response by region will also be important to ensure representativeness. The consent approach did not induce appear to influence self-selection as the pre-incentive and post-incentive consent samples had comparable distributions of sex, geographic region and age.
When deciding whether to complete a survey research study, ‘leverage-salience’ theory posits that individuals consider features of the request including the perceived burden, interest in the topic and familiarity with and/or trust of the sponsor. Incentives can be valuable for tipping the scale towards co-operation.20 We demonstrated that a modest incentive valued at US$11 yielded a higher consent rate when delivered to the prospective participant as a pre-incentive rather than as a post-incentive conditional on participation, consistent with other studies.14 18 20 Furthermore, the pre-incentive approach was more cost-effective than a post-incentive after considering material costs. Once the cost difference related to materials was calculated, there was sufficient savings to continue with the pre-incentive approach without the need to calculate the savings in personnel time, which would have only amplified the savings since the post-incentive protocol was more labour intensive. While seemingly counterintuitive, the cost-effectiveness of incentivising the entire sample rather than only those who complete a survey has been previously reported.18 20 Based on the results of the embedded incentive experiment, the pre-incentive option was used for the remainder of recruitment to the study. In addition, using multiple contacts has long been recommended for maximising response rates in surveys.16 In our study, reminder mailings increased consent by roughly 18% regardless of the timing of the incentive (pre-incentive vs post-incentive).
Limitations and bias considerations
Evaluation of the extent to which our sample cohorts are representative of our target population is constrained by the limited availability of sociodemographic information for the target population. We were only able to compare our sample cohorts in terms of sex, age and geographic location; therefore, systematic differences may exist between groups for additional unmeasured sociodemographic characteristics, such as race/ethnicity and socioeconomic status.
We observed that the pre-incentive option was associated with a higher participation rate and lower costs during the embedded incentive trial; thus, it was used for the remainder of the study’s recruitment. Because of this insight, multiple contact attempts with an incentive included in the initial consent packet (pre-incentive) was the protocol applied to the opt-in sample for the remainder of enrolment to the Voyage Study. Given that CRC incidence and mortality vary with age, we plan to adjust for differential participation by age in future analyses of Voyage data.
Data availability statement
Data are available on reasonable request to the corresponding author.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved on 5 August 2019 by the Mayo Clinic institutional review board (application no. 19-000091). This manuscript is an analysis of the data from year 1 of the Voyage Study. The Voyage protocol study was conducted in accordance with GCP, the provisions specified in Title 21 Parts 11, 50, 54, 56 and 812 of the US Code of Federal Regulations, the International Ethical Guidelines for Biomedical Research Involving Human Subjects, the Declaration of Helsinki and applicable legal and regulatory requirements. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
The authors would like to thank Amy L. Weaver, formerly of the Mayo Clinic, for the statistical work she performed on this study. Medical writing and editorial support were provided by William K. Johnson, PhD, an employee of Exact Sciences Corporation.
References
Footnotes
Contributors Conceptualisation: KJY, JEO, LJFR, JLSS, BK, JJL and CRK; data curation: REC and JAG; formal analysis: REC, KJY and JAG; methodology: JEO, LJFR, JLSS, KJY and EJK; project administration: JEO, LJFR and EJK; and writing (original draft and review and editing) and interpretation of the data: all authors. KJY is the guarantor.
Funding This study was funded by Exact Sciences Corporation (Madison, Wisconsin, USA). Exact Sciences Laboratory and Mayo Clinic were responsible for the design and conduct of the study.
Competing interests KJY, REC, CRK, EJK, JLSS, JAG and JEO are employees of Mayo Clinic and received funding for this study through a grant from Exact Sciences Corporation. BK and JJL are employees and shareholders of Exact Sciences. At the start of the work herein, LJFR was an employee of the Mayo Clinic, whereby she offered scientific input to Exact Sciences for research studies through a contracted services agreement between Mayo Clinic and Exact Sciences Corporation. During the writing of this manuscript, she became a full-time employee of Exact Sciences Corporation. Currently, she is an employee of Mayo Clinic only.
Provenance and peer review Not commissioned; externally peer reviewed.