Original articleClinical endoscopyThe validity and reliability of a Direct Observation of Procedural Skills assessment tool: assessing colonoscopic skills of senior endoscopists
Section snippets
Background
Several tools focusing on assessment of complex, integrated aspects of clinical practice in the workplace have been developed. These workplace-based assessments (WBAs) have facilitated a move away from the use of numbers-based experience and a broad overview of capability toward a more structured10 and evidenced approach based on competence. Centered on a framework of scoring over a finite number of items, they have been used in the UK Foundation doctors' program and have been trialed in senior
Development of the assessment
An expert group composed of physician, surgeon, and nurse colonoscopists and clinical educationalists was convened. They comprised the national and regional endoscopy training center leads, the chair of the JAG, and the education advisor to the National Endoscopy Program. All had a track record of high-quality endoscopy and experience in training trainees and practitioners in endoscopy, and some had formal educational qualifications. They developed the assessment via a series of iterations.
Assessments
One hundred ninety-three assessments were undertaken by 162 candidates and 28 assessors. Of those, 31 were repeated assessments and were excluded from analysis, as were 14 candidates not eligible for inclusion for the correlations because their polyp detection rates were less than 20%, and 1 candidate was excluded owing to influential outlying data (a lifetime colonoscopy number of over 25,000), leaving 147 candidates. Descriptive statistics for these variables are shown in Table 2.
DOPS validity: questionnaire data
There were
Discussion
The assessment appears to be reliable, valid, and useful for assessing skills in colonoscopy. What are the wider implications? The DOPS can be used in clinical practice to evaluate the skills of trainees and independent practitioners at colonoscopy. Detailed assessment of attending physicians or consultants is likely to meet significant resistance for a variety of reasons. In some circumstances, it could be very valuable: In confirming attainment of competence in trainees, in the assessment of
Conclusion
This is the first specific evaluation of an endoscopic DOPS assessment. It performs well, with good levels of reliability and validity, sufficient to be used in a high-stakes assessment. This approach could be adapted as a quality management process in other specialties.
Acknowledgments
The authors thank all the candidates, the assessors, the Accreditation Panel, and the BCSP for assisting with the development of this process. Particular thanks to the National Endoscopy Programme and the training centre leads for developing the DOPS assessment. Special thanks to Linda Beard for administrative support, Nicola Gardiner for data handling, and Dr Ron Hoogenboom for the statistical analysis.
This study was supported by the English NHS Bowel Cancer Screening Programme.
References (25)
- et al.
Measuring the quality of endoscopy
Gastrointest Endosc
(2006) Trainee certification
- et al.
AGA Institute Future Trends Committee report: the future of gastroenterology training programs in the United States
Gastroenterology
(2008) Renewal of and proctoring for endoscopic privileges
Trust, Assurance and SafetyThe regulation of health professionals in the 21st century
(2007)- et al.
A prospective study of colonoscopy practice in the UK today: are we adequately prepared for national colorectal cancer screening tomorrow?
Gut
(2004) Scoping our practiceThe 2004 Report of the National Confidential Enquiry into Patient Outcome and Death
(2004)- et al.
Ensuring the clinical competence of medical school graduates through standardized patients
Arch Intern Med
(1987) Principles of assessment
- et al.
How well do internal medicine faculty members evaluate the clinical skills of residents?
Ann Intern Med
(1992)
Implementing workplace-based assessment across the medical specialties in the United Kingdom
Med Educ
The mini-CEX: a method for assessing clinical skills
Ann Intern Med
Cited by (108)
Development and initial validation of a video-based peroral endoscopic myotomy assessment tool
2024, Gastrointestinal EndoscopyWorkplace-based assessment - Moving from participation to engagement using direct observation of procedural skills (DOPS)
2023, Journal of Medical Imaging and Radiation SciencesEnsuring Quality in Endoscopic Training: Tools for the Educator and Trainee
2022, Techniques and Innovations in Gastrointestinal EndoscopyTo evaluate the effect of small-group demonstration–based training module in improving clinical skills in undergraduate medical students
2022, Medical Journal Armed Forces IndiaColorectal cancer prevention by a CLEAR principles–based colonoscopy protocol: an observational study
2020, Gastrointestinal EndoscopyExploring Endoscopic Competence in Gastroenterology Training: A Simulation-Based Comparative Analysis of GAGES, DOPS, and ACE Assessment Tools
2024, Advances in Medical Education and Practice
DISCLOSURE: All authors disclosed no financial relationships relevant to this publication.
If you would like to chat with an author of this article, you may contact Dr Barton at [email protected].