You are here

Article Text

Article menu
Histopathology
CD, or not CD, that is the question: a digital interobserver agreement study in coeliac disease
  1. James Denholm1,2,3,
  2. Benjamin A Schreiber1,2,
  3. Florian Jaeckle1,3,
  4. Mike N Wicks4,
  5. Emyr W Benbow5,6,
  6. Tim S Bracey7,8,
  7. James Y H Chan9,
  8. Lorant Farkas10,11,
  9. Eve Fryer12,
  10. Kishore Gopalakrishnan13,
  11. Caroline A Hughes12,
  12. Kathryn J Kirkwood14,
  13. Gerald Langman15,
  14. Betania Mahler-Araujo9,16,
  15. Raymond F T McMahon5,6,
  16. Khun La Win Myint17,
  17. Sonali Natu18,
  18. Andrew Robinson13,
  19. Ashraf Sanduka9,
  20. Katharine A Sheppard12,
  21. Yee Wah Tsang13,
  22. Mark J Arends19,
  23. Elizabeth J Soilleux1,3
  1. 1 Department of Pathology, University of Cambridge, Cambridge, UK
  2. 2 Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, UK
  3. 3 Lyzeum Ltd, Cambridge, UK
  4. 4 Department of Pathology, The University of Edinburgh College of Medicine and Veterinary Medicine, Edinburgh, UK
  5. 5 Division of Medical Education, The University of Manchester, Manchester, UK
  6. 6 Department of Histopathology, Manchester University NHS Foundation Trust, Manchester, UK
  7. 7 Department of Diagnostic and Molecular Pathology, Royal Cornwall Hospitals NHS Trust, Truro, UK
  8. 8 University Hospitals Plymouth NHS Trust, Plymouth, UK
  9. 9 Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK
  10. 10 Department of Pathology, Akershus University Hospital, Nordbyhagen, Norway
  11. 11 Institute of Clinical Medicine, University of Oslo, Nordbyhagen, Norway
  12. 12 Department of Cellular Pathology, Oxford University Hospitals NHS foundation Trust, Oxford, UK
  13. 13 Department of Histopathology, University Hospitals Coventry and Warwickshire NHS Trust, Coventry, UK
  14. 14 Department of Pathology, Western General Hospital, Edinburgh, UK
  15. 15 Department of Cellular Pathology, Heartlands Hospital, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
  16. 16 MRC Institute of Metabolic Science, Wellcome Trust, Cambridge, UK
  17. 17 Department of Pathology, Queen Elizabeth University Hospital, Glasgow, UK
  18. 18 University Hospital of North Tees, North Tees and Hartlepool NHS Foundation Trust, Stockton on Tees, UK
  19. 19 Division of Pathology, University of Edinburgh, Edinburgh, UK
  1. Correspondence to Dr Florian Jaeckle; florian.jaeckle{at}gmail.com

Abstract

Objective Coeliac disease (CD) diagnosis generally depends on histological examination of duodenal biopsies. We present the first study analysing the concordance in examination of duodenal biopsies using digitised whole-slide images (WSIs). We further investigate whether the inclusion of immunoglobulin A tissue transglutaminase (IgA tTG) and haemoglobin (Hb) data improves the interobserver agreement of diagnosis.

Design We undertook a large study of the concordance in histological examination of duodenal biopsies using digitised WSIs in an entirely virtual reporting setting. Our study was organised in two phases: in phase 1, 13 pathologists independently classified 100 duodenal biopsies (40 normal; 40 CD; 20 indeterminate enteropathy) in the absence of any clinical or laboratory data. In phase 2, the same pathologists examined the (re-anonymised) WSIs with the inclusion of IgA tTG and Hb data.

Results We found the mean probability of two observers agreeing in the absence of additional data to be 0.73 (±0.08) with a corresponding Cohen’s kappa of 0.59 (±0.11). We further showed that the inclusion of additional data increased the concordance to 0.80 (±0.06) with a Cohen’s kappa coefficient of 0.67 (±0.09).

Conclusion We showed that the addition of serological data significantly improves the quality of CD diagnosis. However, the limited interobserver agreement in CD diagnosis using digitised WSIs, even after the inclusion of IgA tTG and Hb data, indicates the importance of interpreting duodenal biopsy in the appropriate clinical context. It further highlights the unmet need for an objective means of reproducible duodenal biopsy diagnosis, such as the automated analysis of WSIs using artificial intelligence.

  • COELIAC DISEASE
  • HISTOPATHOLOGY
  • SMALL INTESTINAL BIOPSY
  • MEDICAL STATISTICS
  • GLUTEN SENSITIVE ENTEROPATHY

Data availability statement

No data are available. The raw data, along with the code and instructions for reproducing all of the analysis and figures presented in this work are available in THIS GITLAB REPOSITORY (https://gitlab.developers.cam.ac.uk/path/soilleux/soilleux-group/cd-inter-observer-agreement). We are not at liberty to share the WSIs, however.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjgast-2023-001252

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Data availability statement

    No data are available. The raw data, along with the code and instructions for reproducing all of the analysis and figures presented in this work are available in THIS GITLAB REPOSITORY (https://gitlab.developers.cam.ac.uk/path/soilleux/soilleux-group/cd-inter-observer-agreement). We are not at liberty to share the WSIs, however.

    View Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors JD, FJ and BS conducted and tested all of the analysis. JD and ES drafted the original revision of this manuscript, which was heavily contributed to by FJ, BS and MJA. MW set up and facilitated access virtual access to the WSIs for each participant. All authors, with the exception of JD, BS, MW, FJ, MJA and ES, reported on the 100 duodenal biopsies. ES and MJA conceptualised this study and guided the analysis. BS, FJ, MJA and ES reviewed and proposed revisions to this manuscript, before it was circulated with all co-authors, who then had the opportunity to do the same. JD is the guarantor for this study.

    • Funding JD and ES acknowledge Coeliac UK and Innovate UK (grant INOV03-19 to ES). FJ acknowledges financial support from the Cambridge Centre for Data-Driven Discovery (C2D3). ES acknowledges a Pathological Society Consultant’s PumpPriming grant—01 April 2016; Grant Reference No: 1084 ‘Developing digital analytical algorithms for coeliac disease diagnosis: a paradigm for epithelial/inflammatory pathology’. BS acknowledges a Pathological Society PhD studentship. The Comparative Pathology Workbench has been further developed for use by the Gut Cell Atlas Crohn's Disease Consortium funded by The Leona M. and Harry B. Helmsley Charitable Trust and is supported by a grant from Helmsley to the University of Edinburgh.

    • Competing interests ES and MS are shareholders in Lyzeum Ltd, by whom JD was employed at the time of this work.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.