You are here

Article Text

Article menu
Endoscopy
The Newcastle ENDOPREM™: a validated patient reported experience measure for gastrointestinal endoscopy
  1. Laura J Neilson1,2,
  2. Linda Sharp2,3,
  3. Joanne M Patterson4,
  4. Christian von Wagner5,
  5. Paul Hewitson6,
  6. Lesley M McGregor7,
  7. Colin J Rees1,2,3
  1. 1Department of Gastroenterology, South Tyneside and Sunderland NHS Foundation Trust, South Shields, UK
  2. 2Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
  3. 3Newcastle University Centre for Cancer, Newcastle University, Newcastle upon Tyne, UK
  4. 4School of Health Sciences, University of Liverpool, Liverpool, UK
  5. 5Health Behaviour Research Centre, University College London, London, UK
  6. 6Nuffield Department of Population Health, University of Oxford, Oxford, UK
  7. 7Division of Psychology, University of Stirling, Stirling, UK
  1. Correspondence to Dr Laura J Neilson; l.neilson{at}nhs.net

Abstract

Objectives Measuring patient experience of gastrointestinal (GI) procedures is a key component of evaluation of quality of care. Current measures of patient experience within GI endoscopy are largely clinician derived and measured; however, these do not fully represent the experiences of patients themselves. It is important to measure the entirety of experience and not just experience directly during the procedure. We aimed to develop a patient-reported experience measure (PREM) for GI procedures.

Design Phase 1: semi-structured interviews were conducted in patients who had recently undergone GI endoscopy or CT colonography (CTC) (included as a comparator). Thematic analysis identified the aspects of experience important to patients. Phase 2: a question bank was developed from phase 1 findings, and iteratively refined through rounds of cognitive interviews with patients who had undergone GI procedures, resulting in a pilot PREM. Phase 3: patients who had attended for GI endoscopy or CTC were invited to complete the PREM. Psychometric properties were investigated. Phase 4 involved item reduction and refinement.

Results Phase 1: interviews with 35 patients identified six overarching themes: anxiety, expectations, information & communication, embarrassment & dignity, choice & control and comfort. Phase 2: cognitive interviews refined questionnaire items and response options. Phase 3: the PREM was distributed to 1650 patients with 799 completing (48%). Psychometric properties were found to be robust. Phase 4: final questionnaire refined including 54 questions assessing patient experience across five temporal procedural stages.

Conclusion This manuscript gives an overview of the development and validation of the Newcastle ENDOPREM™, which assesses all aspects of the GI procedure experience from the patient perspective. It may be used to measure patient experience in clinical care and, in research, to compare patients’ experiences of different endoscopic interventions.

  • endoscopy
  • colonoscopy
  • gastrointesinal endoscopy
  • gastroscopy

Data availability statement

No data are available.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

http://dx.doi.org/10.1136/bmjgast-2021-000653

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Summary box

    What is already known about this subject?

    • Patient experience is a key dimension of high-quality clinical care.

    • Positive patient experience is an important determinant of participation in screening programmes and repeat attendance for surveillance procedures.

    • Current measures of patient experience in gastrointestinal (GI) endoscopy are largely clinician derived and measured and may not address the priorities of patients themselves.

    What are the new findings?

    • The Newcastle ENDOPREM™ is the first fully patient-derived patient-reported experience measure (PREM) for GI endoscopy.

    • This is a comprehensive measure, which captures experience across the entire patient journey, specifically elements of patient experience as prioritised by patients.

    • The psychometric properties of the Newcastle ENDOPREM™ have been demonstrated to be robust.

    How might it impact on clinical practice in the foreseeable future?

    • The Newcastle ENDOPREM™ is an important tool for use both in research and evaluation of routine care.

    • Use of the Newcastle ENDOPREM™ will enable units to measure experience provided by units and endoscopists, to allow comparison of experience across different endoscopy modalities (eg, as part of clinical trials of devices) and will identify specific areas, which can be targeted to improve patient experience.

    • The PREM is now being adapted for use internationally and for measuring experience of more novel technologies

    Introduction

    Around one-third of the population will undergo gastrointestinal (GI) endoscopy during their lifetime, with approximately 1.5 million endoscopies performed annually in England and 17 million in the USA.1–3 CT Colonography (CTC) is performed in many countries as an alternative to colonoscopy or where colonoscopy is incomplete. There is significant variation in CTC use between countries with around 150 000 CTCs undertaken in England annually, and although less widely used in the USA, it is performed in at least 700 centres.4 5

    Consistent high-quality endoscopy is essential to ensure that procedures are safe and that lesions are not missed, however, there is significant and unacceptable variation in practice and quality.6–10 Patient experience is one of the key dimensions of high-quality clinical care. Increasing emphasis on patient experience as a marker of quality has emerged in recent years because of the recognition that positive patient experience is associated with better patient outcomes.11 In England, a report into a failing hospital found that losing sight of patients at the centre of healthcare was a key component of system failures.12 As a result, patient experience is now routinely assessed across many aspects of primary and secondary care services in the English National Health Service. Similarly, in the USA, there is increasing use of experience measures (such as the Hospital Consumer Assessment of Healthcare Providers and Systems Survey) to evaluate healthcare providers.13 The value of such information is that it lets healthcare providers identify which aspects of a service may need attention or require improvement.

    Within the field of endoscopy, there is evidence that colonoscopists who deliver high-quality procedures also deliver better patient experience.14 Patient experience is a key determinant of participation in bowel cancer screening programmes and influences whether patients attend for repeat disease surveillance procedures.11 15

    Current measures of patient experience within the field of GI endoscopy focus almost completely on procedural aspects such as pain or discomfort but ignore elements such as preprocedural communication, anxiety and preparation for procedures.16 Additionally, almost all measures are clinician derived and measured. For example, the Gastrointestinal Endoscopy Satisfaction Questionnaire focuses on satisfaction and did consult with patients during question development, however, these questions were developed based on literature review and were not generated by the patients themselves following patient interviews using best practice for development of measures of patient experience.17 It has been reported that patients and clinicians prioritise different aspects of experience with clinicians prioritising the procedure itself, including aspects such as comfort, whereas communication and interaction with the endoscopist are more important to patients.18 Elements of the endoscopy experience other than just those at the time of the procedure, for example, bowel preparation and preprocedural anxiety, are also important. Preprocedural anxiety has been found to be a key determinant of pain and discomfort experience during other clinical procedures.19 Moreover, measuring only procedural comfort scores gives information on only a very small part of the overall patient experience. Patient experience of CTC is not routinely measured in clinical practice and its measurement in research settings tends to focus on comfort and bowel preparation.20 21

    Endoscopy organisations advise measuring patient experience but provide little advice on how this should be done. The European Society of Gastrointestinal Endoscopy (ESGE) and British Society of Gastroenterology (BSG) have highlighted the importance of measuring and improving patient experience and recommend measuring self-reported patient experience using a ‘validated scale;’ however, they acknowledge the lack of a standardised approach.22 23 Patient-reported experience measures (PREMs) are tools to measure patients’ experience of healthcare and should address those aspects of care defined by patients as being important to them; thus, patients should be involved in developing such measures and in defining what they measure.24 25 The systematic development and validation of PREMs have been undertaken in a range of specialties and disease conditions, for example, cancer care, paediatric emergency care and sickle cell disease but not in the field of GI endoscopy or CTC.26–28 The approach to devising a PREM includes literature review to identify current tools and aspects contributing to patient experience; a qualitative stage involving relevant patients to identify areas of care important to them; questionnaire design informed by the qualitative work; cognitive testing of the questionnaire and field testing to assess validity.25 29 Tools to measure the total, or overall, experience of GI endoscopy do not exist. Those tools which have been developed to measure some elements of experience have not been systematically developed with patients. This paper describes the rigorous process of developing the first patient-derived, validated, PREM for GI endoscopy.

    Methods

    The study was undertaken in the North-East of England. The development of the PREM involved four phases and was undertaken according to the established COSMIN (COnsensus-based Standards for the selection of health status Measurement INstruments) criteria for development of health measurement tools.30

    Phase 1: concept elicitation

    This phase provided in-depth and detailed identification and description of the experiences of patients who had undergone oesophagogastroduodenoscopy (OGD), colonoscopy or CTC and has been reported in detail elsewhere.31 Between February 2016 and April 2017, patients aged ≥18 years who had undergone one of these procedures for symptoms or surveillance (but not within the national bowel cancer screening programme) in one hospital were invited to participate in one-to-one semi-structured interviews. Eligible patients on a series of endoscopy lists were approached in the department before their procedure. Purposive sampling was used to ensure a range of procedures, age and sex among participants. These were guided by a topic guide developed from literature review and expert opinion; the guide was used flexibly so that any new issues identified by patients were added to the guide to be explored in subsequent interviews. Interviews were audio-recorded and transcribed. Recruitment continued until data saturation. Thematic analysis was undertaken to identify over-arching themes.32 This phase identified aspects of experience that mattered to patients to inform questionnaire content. It was also used to identify whether it was possible to develop a PREM that would apply across different GI procedures.

    Phase 2: development and content validity

    This phase was undertaken between April and August 2017 and involved iterative development of the pilot PREM. Informed by the qualitative interviews and a focused literature review, a question bank was generated. This involved decisions on topics to be included, their order, question format and response format. Several rounds of revision and review were undertaken by the research team.

    The draft questionnaire—designed for patient self-completion—was then developed and pretested, using several rounds of face-to-face cognitive interviews with patients who had undergone GI procedures, from June to July 2017. Interviewees were invited to ‘talk aloud’ as they completed the questionnaire, with verbal probing used to clarify any problems or issues.33 Different styles of question format and layout were tested. Interviews were audio-recorded and transcribed. Analysis after each round used a systematic approach to identify problems.34 The questionnaire was refined after each round and the refined draft tested in the subsequent round. Once no new issues arose in interviews, the pilot PREM was agreed. This process ensured patient comprehension of the questions and face and content validity of the questionnaire (ie, confirmed relevance and comprehensiveness of the questions).

    Phase 3: psychometric properties and validation

    This phase evaluated the psychometric properties of the pilot questionnaire and validated it in a large cohort. Between October 2017 and September 2018, the pilot PREM was given to patients following GI endoscopy or CTC in four English NHS hospital Trusts. The sample size was based on the number of questions included in the pilot PREM and exceeded the recommended 15 completed questionnaires per item for studies of psychometric qualities. Patients were asked to take it home and complete and return it in the prepaid envelope provided. A patient information sheet (PIS) was included with the questionnaire pack and it was made clear to participants that return of a completed questionnaire was deemed to signify consent.

    Statistical analysis focused on response rates and patterns, missing values, ‘floor’ and ‘ceiling’ effects (ie, propensity of respondents to endorse the extreme ends of the response scale) and correlations between question responses (to identify agreement between items and question redundancy) and with the total score (sum of all the experience questions). Analysis was performed using IBM® SPSS® V.24 (IBM Corp. Released 2016. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp).

    Following completion of Phase 3, all questions were reviewed systematically alongside the Phase 3 data to determine where refinements in wording or layout were required. Those questions considered to be redundant were removed, for example, those which correlated poorly with others. This resulted in the Newcastle ENDOPREM™

    Results

    Phase 1: concept elicitation

    Thirty-five patients (15 OGD, 10 colonoscopy, 10 CTC) were interviewed. Characteristics of these patients are listed in table 1. When invited to discuss their experience, a few participants focused on a single aspect of the experience which was most important to them: for example, the taste and volume of the bowel preparation or the effect of good communication from staff. However, most participants described their experience in chronological order, that is, as stages of a process, starting with the referral process (henceforth, called ‘before attending for the test’), then visiting the hospital (‘at the hospital, before the test’), undergoing the procedure itself (‘during the test’) and what happened afterwards (‘after the test’). Six themes emerged: anxiety, expectations, information & communication, embarrassment & dignity, comfort and choice & control. All six themes were demonstrated in more than one procedural stage. All themes and procedural stages emerged in relation to all three GI procedures.

    View this table:
    Table 1

    Phase 1 participant characteristics

    The themes were organised by procedural stage, as described in phase 1. Questions were developed for each relevant theme and stage; where possible, language echoed that used by patients in phase 1. Most questions were in the form of statements (eg, ‘during the test, my dignity was maintained at all times’) with a five-level Likert-type response scale, ranging from ‘strongly agree’ to ‘strongly disagree’. To minimise response set bias (where respondents endorse the same response option for all questions), both positive and negative questions were included (ie, some statements were phrased such that ‘strongly agree’ would indicate a ‘positive’ experience, and others phrased so that ‘strongly disagree’ would indicate a ‘positive’ experience).

    Ten rounds of review and revision were conducted by the study team. Five rounds of cognitive testing were then undertaken, with three endoscopy patients in each round (15 participants in total, table 2). Focused literature review found no additional aspects of patient experience not covered by the pilot PREM.

    View this table:
    Table 2

    Phase 2 participant characteristics

    The pilot PREM thus comprised 59 questions which spanned patient experience; plus a series of questions on respondent characteristics (eg, age, ethnicity) as these have been found to be associated with patient experience in many clinical contexts; plus three questions on potential ‘explanatory’ factors (which might affect patient experience; eg, endoscopist gender); and a further two questions on ‘overall’ experience. The sections of the pilot PREM were section A—completing this survey (10 questions), section B—before coming to hospital for your test (16 questions), section C—preparing for your test (six questions), section D—at the hospital, before the test (five questions), section E—during the test (21 questions), section F—after the test (11 questions) and section G—overall experience (three questions).

    Phase 3: psychometric properties and validation

    The PREM underwent multisite validation and was given to 1650 eligible patients of whom 799 responded (response rate=48.4%). The response rate was higher in older patients and those undergoing lower GI procedures were more likely to respond (table 3). As patients took the PREM away to complete, the time to complete it was not measured; however, in phase 2, the interview time was 10–15 min. Respondents’ ages ranged from 18 to 95, with a mean age of 65.3 (SD 12.6). 43.3% of respondents were male and the majority (98.4%) were of white British ethnicity. 41.1% of respondents underwent OGD (including 0.8% who underwent transnasal endoscopy), 43.3% underwent colonoscopy, 1.3% underwent both OGD and colonoscopy on the same day (referred to as ‘OGD & colonoscopy’ henceforth) and 14.4% underwent CTC.

    View this table:
    Table 3

    Phase 3 response rates according to participant characteristics

    There was notable variation in the numbers of participants recruited per procedure type; 10 patients who had both OGD & colonoscopy performed responded to the questionnaire (of 34 invited) and 115 CTC participants (of 209 invited). Far fewer CTCs are done at each of the sites than OGD and colonoscopies, and this was reflected in the number of participants recruited in this group.

    There was a discrepancy between the procedure the patient reported that they had attended for and the procedure confirmed by the clinical team in 53 cases (6.6%). The section that should have been completed by those undergoing colonoscopy and CTC only (covering issues around bowel preparation) was completed by 96.1% of participants to whom it applied and 18.0% of those to whom it did not.

    Completion rates of individual questions were high; for only three questions, more than 5% of people failed to provide an answer. The oldest age group (>75 years) was significantly more likely to miss questions.

    In terms of potential question redundancy, two pairs of questions correlated strongly (rs>0.8) and four questions poorly correlated with any others (rs<0.3). Eight questions, including the four which poorly correlated with any others, had poor corrected item-total correlation (ITC <0.3), meaning that they did not correlate well with the overall sum of all questions.

    Twenty-five questions had a ceiling effect (>40% of respondents endorsed the ‘best’ response). No questions had floor effects (>40% choosing the ‘worst’ option).

    Phase 4: item reduction and refinement

    Five questions were removed as they were considered redundant because of poor inter-item correlation and ITC. Two items (which were strongly correlated) were merged; they asked a similar question with slightly different time points. One question was added by the study team to simplify an item and one explanatory question was rephrased with a further explanatory question added for clarity.

    The final Newcastle ENDOPREM™ includes 10 demographic/patient characteristic questions, 54 patient experience and four explanatory questions (online supplemental file 1). A shorter version will be available for specific procedures, for example, without the bowel preparation section for upper GI endoscopy.

    Discussion

    Understanding and improving patient experience are fundamental to delivering high-quality GI endoscopy. We give an overview of a systematic and rigorous approach to developing the first validated and fully patient-derived PREM for GI endoscopy, the Newcastle ENDOPREM™, conducted according to the COSMIN criteria.

    The Newcastle ENDOPREM™ is robust and comprehensive and in this form is designed to capture experience across the entire patient journey, not simply during the procedure itself. In its current form, it is applicable to upper GI endoscopy, colonoscopy and CTC. The various elements of the patient journey—including how the patient received their appointment, how they experienced the facilities in which they waited for the procedure, how results were given and what would happen next were conveyed—were all things that patients raised in the qualitative work as being important to them.31

    A range of measures are available to measure individual components of the endoscopy process. For example, a number of procedural comfort scores exist and some of these have been validated.35 However, almost all measures are clinician or expert designed rather than being developed based on what patients report as being important. The Global Rating Scale patient experience domain is used internationally and was derived from literature review and expert opinion.36 The Gastrointestinal Endoscopy Satisfaction Questionnaire was developed by including the ‘most relevant’ questions from existing measures.17 Although patients were involved to ensure face and content validity, they were not involved in initially defining the questions. While it covers some preprocedural and postprocedural elements, items about preparation for the test or later results are not covered in detail. The Gastronet is a clinician-derived questionnaire for assessing patient experiences of endoscopy in Europe.37 It includes three questions about discomfort and a question about satisfaction with the information given about the test and results, which patients complete the day following their procedure. In the USA, the American Society for Gastrointestinal Endoscopy recommends that a general satisfaction scale is used, modified for endoscopic procedures.38 39 This is not specific to GI procedures and was developed without patient input, nor does it assess comfort. The Newcastle ENDOPREM™ addressed the assessment of patient experience differently by first establishing the aspects of experience that mattered to endoscopy patients, then systematically developing and testing a tool to measure all of those aspects of experience, while adhering to best practice in health measurement development according to the established COSMIN checklist.31 Patients were also involved in the process of refining and improving the instrument and in the psychometric validation study. Thus, the instrument is both grounded in patient experience and seeks to measure that experience.

    The length of this questionnaire is an important issue to be considered. The PREM is necessarily long as it covers the entirety of the patient experience and covers upper and lower GI procedures with and without sedation. However, despite this length, the completion time was around 10–15 min in the cognitive interview round. In the validation survey, the response rate of 48% is comparable to similar self-completion questionnaires in other areas of clinical practice.40 Completion rates might be affected by when the questionnaire is administered. We took the decision to ask patients to complete the questionnaire after leaving the department to avoid the possibility that their ratings of experience would be influenced either by sedation or by still being present in the hospital within the department that had provided them with care. There are different possible modes and timing of questionnaire administration (eg, paper vs digital, provided at end of clinic appointment vs sent by post after discharge) and these will be the focus of further work. Were the questionnaire too long then the completion rate of questions might be expected to be impacted, however, the completion of individual questions was excellent with very low levels of missing data. Moreover, there was no evidence that the number of questions missed increased in later parts of the questionnaire. The full questionnaire covers upper and lower GI endoscopy and CTC. At the outset, it was unclear whether it would be possible to develop a PREM that was applicable to patients who had undergone different endoscopic procedures. The work here demonstrates that the issues that matter to patients are very similar across GI procedures. The main difference—as would be expected—is the issue of bowel preparation for colonoscopy, which patients described in rich and varied ways in the qualitative interviews.31 We designed the instrument so that the questions relating to bowel preparation were in a contained section, however, we note that this section was completed by 18% of those to whom it did not apply. When applied to clinical practice or research, only the relevant parts of the questionnaire will be distributed and, thus, the PREM will be significantly shorter and clearer.

    The PREM will also need to be modified where clinical practice varies. In the UK, conscious sedation is used for colonoscopy. In other countries, for example, USA and Australia, deep sedation is used. Therefore, non-relevant sections of the PREM, for example, those covering intraprocedural pain or comfort, will not be completed where they are not needed. While redundant sections of the PREM will not be given to particular patients and in particular settings, it is important that this is based on only removing those irrelevant sections and not simply removing sections that clinicians or researchers consider of lesser importance.

    The inclusion of CTC within this PREM was to allow a comparator from a non-endoscopy GI investigation, which may be considered an alternative to colonoscopy. Much of the argument for the role of CTC relates to experience, so developing a PREM to allow this to be measured accurately was one of the goals of this research.

    A potential weakness of the study is the study population. Phases 1 and 2 were undertaken in an expert centre with a strong track record in endoscopy and endoscopy research. Phase 3 was undertaken across four sites to increase diversity of patients and experiences, but these were all in the North-East of England, an area with very limited ethnic diversity. Very few participants reported their ethnic group as anything other than White British (and only 0.5% of the participants in phases 1 and 2 were non-White British).41 This means we cannot be certain that the performance and properties of the questionnaire are the same in non-White British ethnic groups.

    Further work is planned to test the PREM further in international settings and settings of wider ethnic diversity. We also currently lack information on the suitability of the questionnaire for patients who had other types of endoscopic procedures (eg, flexible sigmoidoscopy); testing this is also important.

    There are two possible uses of the Newcastle ENDOPREM™. As a patient experience measure, the detail and granularity of the questions across the journey can help healthcare providers identify which specific aspects of their service may benefit from improvement. The instrument may also be used to compare patient experiences of different endoscopic interventions in research studies. Detailed understanding of the different components of endoscopy and how these affect patient experience would be valuable in the context of such head-to-head comparisons. While the Newcastle ENDOPREM™ was developed within the context of the NHS in the UK, relatively few of the questions are specific to the processes and organisation of care in the NHS. We would expect, therefore, that the instrument would be applicable internationally with revisions related to service context. We are currently investigating this in a rigorous international validation study across Europe, USA and Australia using the European Organisation for Research and Treatment of Cancer approach to international translation of patient-reported outcome measures, with the PREM now having been translated into Polish, Norwegian, Spanish, Italian, Dutch and French.42

    The Newcastle ENDOPREM™ is currently being adapted for other GI procedures, including capsule endoscopy and CytoSponge. These procedures are being introduced into routine clinical practice, in some instances as an alternative to endoscopy, and, therefore, the Newcastle ENDOPREM™ will be available to compare experiences of these procedures, both in the clinical and research setting.

    The Newcastle ENDOPREM™ has been developed through a robust and comprehensive pathway. The themes are likely to remain constant despite the changing endoscopic landscape. This PREM was developed prior to the novel COVID-19 pandemic and while delivery of endoscopy may change, and consequently how patients rate their experience, this tool should remain a valid way to measure that experience.

    Conclusion

    The Newcastle ENDOPREM™ is the first patient-derived PREM that can be used to assess experience of patients who have undergone different GI procedures. It is now available for use in GI endoscopy research or evaluation of routine care.

    Data availability statement

    No data are available.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Ethical approval was obtained through the NRES Committee London-Stanmore (IRAS ID: 14689, National Institute for Health Research UKCRN ID 18749). Participants provided informed consent before interview.

    References

    1. ↵
      1. Seeff LC,
      2. Manninen DL,
      3. Dong FB, et al
      . Is there endoscopic capacity to provide colorectal cancer screening to the unscreened population in the United States? Gastroenterology 2004;127:1661–9.doi:10.1053/j.gastro.2004.09.052pmid:http://www.ncbi.nlm.nih.gov/pubmed/15578502
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Department of Health Knowledge and Intelligence Team
      . Further analysis of HES endoscopy data. London: Department of Health Knowledge and Intelligence Team, 2010.
    3. ↵
      1. Peery AF,
      2. Crockett SD,
      3. Murphy CC, et al
      . Burden and cost of gastrointestinal, liver, and pancreatic diseases in the United States: update 2018. Gastroenterology 2019;156:254–72.doi:10.1053/j.gastro.2018.08.063pmid:http://www.ncbi.nlm.nih.gov/pubmed/30315778
      OpenUrlCrossRefPubMed
    4. ↵
      1. Obaro AE,
      2. Plumb AA,
      3. Fanshawe TR, et al
      . Post-imaging colorectal cancer or interval cancer rates after CT colonography: a systematic review and meta-analysis. Lancet Gastroenterol Hepatol 2018;3:326–36.doi:10.1016/S2468-1253(18)30032-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/29472116
      OpenUrlPubMed
    5. ↵
      1. American College of Radiology
      . American College of Radiology Website- my CT colonography, 2021. Available: https://www.acr.org/Clinical-Resources/Colon-Cancer-Screening-Resources/My-CT-Colonography
    6. ↵
      1. Gavin DR,
      2. Valori RM,
      3. Anderson JT, et al
      . The National colonoscopy audit: a nationwide assessment of the quality and safety of colonoscopy in the UK. Gut 2013;62:242–9.doi:10.1136/gutjnl-2011-301848pmid:http://www.ncbi.nlm.nih.gov/pubmed/22661458
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Bowles CJA,
      2. Leicester R,
      3. Romaya C, et al
      . A prospective study of colonoscopy practice in the UK today: are we adequately prepared for national colorectal cancer screening tomorrow? Gut 2004;53:277–83.doi:10.1136/gut.2003.016436pmid:http://www.ncbi.nlm.nih.gov/pubmed/14724164
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Oppong KW,
      2. Romagnuolo J,
      3. Cotton PB
      . The ERCP quality network benchmarking project: a preliminary comparison of practice in UK and USA. Frontline Gastroenterol 2012;3:157–61.doi:10.1136/flgastro-2011-100099pmid:http://www.ncbi.nlm.nih.gov/pubmed/28839658
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Rembacken B,
      2. Hassan C,
      3. Riemann JF, et al
      . Quality in screening colonoscopy: position statement of the European Society of gastrointestinal endoscopy (ESGE). Endoscopy 2012;44:957–68.doi:10.1055/s-0032-1325686pmid:http://www.ncbi.nlm.nih.gov/pubmed/22987217
      OpenUrlCrossRefPubMed
    10. ↵
      1. Kaminski MF,
      2. Regula J,
      3. Kraszewska E, et al
      . Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 2010;362:1795–803.doi:10.1056/NEJMoa0907667pmid:http://www.ncbi.nlm.nih.gov/pubmed/20463339
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Black N,
      2. Jenkinson C
      . Measuring patients’ experiences and outcomes. BMJ 2009;339:b2495. doi:10.1136/bmj.b2495
    12. ↵
      1. Francis R
      . Report of the Mid-Staffordshire NHS Foundation trust public inquiry and the government's response.. London The Stationery Office; 2013. https://www.gov.uk/government/publications/report-of-the-mid-staffordshire-nhs-foundation-trust-public-inquiry
    13. ↵
      1. Agency for Healthcare Research and Quality
      . CAHPS®: assessing health care quality from the patient’s perspective 2014 https://www.ahrq.gov/cahps/about-cahps/index.html
    14. ↵
      1. Ekkelenkamp VE,
      2. Dowler K,
      3. Valori RM, et al
      . Patient comfort and quality in colonoscopy. World J Gastroenterol 2013;19:2355–61.doi:10.3748/wjg.v19.i15.2355pmid:http://www.ncbi.nlm.nih.gov/pubmed/23613629
      OpenUrlCrossRefPubMed
    15. ↵
      1. Doyle C,
      2. Lennox L,
      3. Bell D
      . A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open 2013;3:e001570. doi:10.1136/bmjopen-2012-001570pmid:http://www.ncbi.nlm.nih.gov/pubmed/23293244
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Brown S,
      2. Bevan R,
      3. Rubin G, et al
      . Patient-Derived measures of Gi endoscopy: a meta-narrative review of the literature. Gastrointest Endosc 2015;81:1130–40.doi:10.1016/j.gie.2014.11.047pmid:http://www.ncbi.nlm.nih.gov/pubmed/25864891
      OpenUrlPubMed
    17. ↵
      1. Hutchings HA,
      2. Cheung W-Y,
      3. Alrubaiy L, et al
      . Development and validation of the gastrointestinal endoscopy satisfaction questionnaire (GESQ). Endoscopy 2015;47:1137–43.doi:10.1055/s-0034-1392547pmid:http://www.ncbi.nlm.nih.gov/pubmed/26349066
      OpenUrlPubMed
    18. ↵
      1. Denters MJ,
      2. Deutekom M,
      3. Derkx B, et al
      . Patient satisfaction with the colonoscopy procedure: endoscopists overestimate the importance of adverse physical symptoms. Frontline Gastroenterol 2012;3:130–6.doi:10.1136/flgastro-2012-100150pmid:http://www.ncbi.nlm.nih.gov/pubmed/28839653
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Baser E,
      2. Togrul C,
      3. Ozgu E, et al
      . Effect of pre-procedural state-trait anxiety on pain perception and discomfort in women undergoing colposcopy for cervical cytological abnormalities. Asian Pac J Cancer Prev 2013;14:4053–6.doi:10.7314/APJCP.2013.14.7.4053pmid:http://www.ncbi.nlm.nih.gov/pubmed/23991951
      OpenUrlPubMed
    20. ↵
      1. Hansmann A,
      2. Burling D
      . Essential requirements of a CT colonography service. Eur J Radiol 2013;82:1187–91.doi:10.1016/j.ejrad.2012.03.018pmid:http://www.ncbi.nlm.nih.gov/pubmed/22537980
      OpenUrlPubMed
    21. ↵
      1. Zueco Zueco C,
      2. Sobrido Sampedro C,
      3. Corroto JD, et al
      . CT colonography without cathartic preparation: positive predictive value and patient experience in clinical practice. Eur Radiol 2012;22:1195–204.doi:10.1007/s00330-011-2367-0pmid:http://www.ncbi.nlm.nih.gov/pubmed/22246146
      OpenUrlCrossRefPubMed
    22. ↵
      1. Kaminski MF,
      2. Thomas-Gibson S,
      3. Bugajski M, et al
      . Performance measures for lower gastrointestinal endoscopy: a European Society of gastrointestinal endoscopy (ESGE) quality improvement initiative. Endoscopy 2017;49:378–97.doi:10.1055/s-0043-103411pmid:http://www.ncbi.nlm.nih.gov/pubmed/28268235
      OpenUrlPubMed
    23. ↵
      1. Rees CJ,
      2. Trebble T,
      3. von Wagner C
      . British Society of gastroenterology position statement on patient experience of GI endsocopy. Gut 2019;66:1181–96.
      OpenUrl
    24. ↵
      1. Coulter A,
      2. Fitzpatrick R,
      3. Cornwell J
      . The point of care. Measures of patients’ experience in hospital: purpose, methods and uses. London: King's Fund, 2009. https://www.kingsfund.org.uk/sites/default/files/Point-of-Care-Measures-of-patients-experience-in-hospital-Kings-Fund-July-2009_0.pdf
    25. ↵
      1. Hodson M,
      2. Andrew S,
      3. Michael Roberts C,
      4. Roberts C
      . Towards an understanding of PREMS and PROMs in COPD. Breathe 2013;9:358–64.doi:10.1183/20734735.006813
      OpenUrlAbstract/FREE Full Text
    26. ↵
      1. Davies F,
      2. Vohra J,
      3. Hopwood B
      . The development of a patient reported experience measure (PREM) survey for children 0-16 years in urgent and emergency care 2011. London Royal College of Paediatrics and Child Health; 2011. https://www.rcpch.ac.uk/sites/default/files/Research_Report_-_PREM_Development_Oct2012.pdf
    27. ↵
      1. Department of Health
      . National cancer patient experience survey, 2011-2012. UK data service, SN: 7134 2012.doi:http://doi.org/10.5255/UKDA-SN-7134-1
    28. ↵
      1. Chakravorty S,
      2. Tallett A,
      3. Witwicki C, et al
      . Patient-Reported experience measure in sickle cell disease. Arch Dis Child 2018;103:1104–9.doi:10.1136/archdischild-2018-314955pmid:http://www.ncbi.nlm.nih.gov/pubmed/30077973
      OpenUrlAbstract/FREE Full Text
    29. ↵
      1. Jenkinson C,
      2. Coulter A,
      3. Bruster S
      . The Picker patient experience questionnaire: development and validation using data from in-patient surveys in five countries. Int J Qual Health Care 2002;14:353–8.doi:10.1093/intqhc/14.5.353pmid:http://www.ncbi.nlm.nih.gov/pubmed/12389801
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      1. Mokkink LB,
      2. Terwee CB,
      3. Patrick DL, et al
      . The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010;19:539–49.doi:10.1007/s11136-010-9606-8pmid:http://www.ncbi.nlm.nih.gov/pubmed/20169472
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      1. Neilson LJ,
      2. Patterson J,
      3. von Wagner C, et al
      . Patient experience of gastrointestinal endoscopy: Informing the development of the Newcastle ENDOPREM™. Frontline Gastroenterol 2020;11:209–17.doi:10.1136/flgastro-2019-101321
      OpenUrlAbstract/FREE Full Text
    32. ↵
      1. Braun V,
      2. Clarke V
      . Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101.doi:10.1191/1478088706qp063oa
      OpenUrlCrossRef
    33. ↵
      1. Willis GB
      . Cognitive interviewing: a tool for improving questionnaire design. London: Sage Publications, 2004: 335.
    34. ↵
      1. Conrad F,
      2. Blair J
      . From impressions to data: increasing the Objectivity of cognitive interviews. In: Proceedings of the Section on Survey Research Methods, American Statistical Association 1996.
    35. ↵
      1. Rostom A,
      2. Ross ED,
      3. Dubé C, et al
      . Development and validation of a nurse-assessed patient comfort score for colonoscopy. Gastrointest Endosc 2013;77:255–61.doi:10.1016/j.gie.2012.10.003pmid:http://www.ncbi.nlm.nih.gov/pubmed/23317691
      OpenUrlCrossRefPubMed
    36. ↵
      1. Joint Advisory Group on Gastrointestinal Endoscopy
      . Joint Advisory group on gastrointestinal endoscopy (JAG) global rating scale (GRS). version for acute services (all nations). London 2016 https://www.thejag.org.uk/Downloads/JAG/Accreditation%20-%20Global%20Rating%20Scale%20(GRS)/Guidance%20-%20acute%20GRS%20standards%20UK.pdf
    37. ↵
      1. Hoff G,
      2. Bretthauer M,
      3. Huppertz-Hauss G, et al
      . The Norwegian Gastronet project: continuous quality improvement of colonoscopy in 14 Norwegian centres. Scand J Gastroenterol 2006;41:481–7.doi:10.1080/00365520500265208pmid:http://www.ncbi.nlm.nih.gov/pubmed/16635918
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. Johanson JF,
      2. Schmitt CM,
      3. Deas TM, et al
      . Quality and outcomes assessment in gastrointestinal endoscopy. Gastrointest Endosc 2000;52:827–30.doi:10.1016/S0016-5107(00)70218-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/11182685
      OpenUrlCrossRefPubMed
    39. ↵
      1. Rizk MK,
      2. Sawhney MS,
      3. Cohen J, et al
      . Quality indicators common to all GI endoscopic procedures. Gastrointest Endosc 2015;81:3–16.doi:10.1016/j.gie.2014.07.055pmid:http://www.ncbi.nlm.nih.gov/pubmed/25480102
      OpenUrlCrossRefPubMed
    40. ↵
      1. NHS England Analytical Team (Medical and Nursing Analytical Unit)
      . Statistical bulletin: overall patient experience scores 2015 adult inpatient survey update NHS England; 2016. https://www.england.nhs.uk/statistics/2016/06/08/overall-patient-experience-scores-2015-adult-inpatient-survey-update/
    41. ↵
      1. Office for National Statistics
      . UK data service census support, 2011 census: Digitised boundary data (England and Wales), 2011. https://borders.ukdataservice.ac.uk
    42. ↵
      1. Kuliś D,
      2. Bottomley A,
      3. Velikova G
      . EORTC quality of life group translation procedure. Fourth Edition. Brussels: EORTC, 2017. https://www.eortc.org/app/uploads/sites/2/2018/02/translation_manual_2017.pdf

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @LauraJNeilson

    • Contributors CJR secured funding. CJR, LJN and LS conceived the study and reviewed the literature. CJR, LS and JMP oversaw data collection, analysis and interpretation. LJN undertook all interviews, analyses and drafted the paper. LMM double coded a proportion of the interview transcripts. CJR, LJN, LS, JMP, CvW, PH and LMM contributed to study design, iteratively refined and approved the final PREM, critically reviewed manuscript drafts and approved the final article for submission. CJR and LJN are guarantors for the data.

    • Funding This study was an investigator led study funded by Aquilant Endoscopy and adopted onto the National Institute for Health Research Portfolio (UKCRN ID 18749). The funder had no role in study design, data collection, data analysis, data interpretation or writing of this manuscript. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit.

    • Competing interests CJR has received grant funding from ARC medical, Norgine, Aquilant and Olympus medical. He was an expert witness for ARC medical. CJR and LS have received research funding from 3D matrix. No other authors have competing interests.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.