Article Text
PDF
Abstract
Objective The utility of International Classification of Diseases (ICD) codes relies on the accuracy of clinical reporting and administrative coding, which may be influenced by country-specific codes and coding rules. This study explores the accuracy and limitations of the Australian Modification of the 10th revision of ICD (ICD-10-AM) to detect the presence of cirrhosis and a subset of key complications for the purpose of future large-scale epidemiological research and healthcare studies.
Design/method ICD-10-AM codes in a random sample of 540 admitted patient encounters at a major Australian tertiary hospital were compared with data abstracted from patients’ medical records by four blinded clinicians. Accuracy of individual codes and grouped combinations was determined by calculating sensitivity, positive predictive value (PPV), negative predictive value and Cohen’s kappa coefficient (κ).
Results The PPVs for ‘grouped cirrhosis’ codes (0.96), hepatocellular carcinoma (0.97) ascites (0.97) and ‘grouped varices’ (0.95) were good (κ all >0.60). However, codes under-detected the prevalence of cirrhosis, ascites and varices (sensitivity 81.4%, 61.9% and 61.3%, respectively). Overall accuracy was lower for spontaneous bacterial peritonitis (‘grouped’ PPV 0.75; κ 0.73) and the poorest for encephalopathy (‘grouped’ PPV 0.55; κ 0.21). To optimise detection of cirrhosis-related encounters, an ICD-10-AM code algorithm was constructed and validated in an independent cohort of 116 patients with known cirrhosis.
Conclusion Multiple ICD-10-AM codes should be considered when using administrative databases to study the burden of cirrhosis and its complications in Australia, to avoid underestimation of the prevalence, morbidity, mortality and related resource utilisation from this burgeoning chronic disease.
- hepatic encephalopathy
- peritonitis
- ascites
- epidemiology
- health service research
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Summary box
What is already known about this subject?
The accuracy of International Classification of Diseases (ICD) codes to identify cirrhosis and cirrhosis-related complications has been reported using data from Department of Veterans Affairs and other US health system databases. Variance in reporting due to country-specific codes and coding rules can have an important impact on health departments’ perception of liver disease morbidity/mortality (and subsequent resource allocation), in addition to the accuracy of epidemiological studies and healthcare research.
What are the new findings?
In this first study of the Australian Modification the 10th revision of ICD (ICD-10-AM), we report the accuracy of individual codes and grouped combinations for cirrhosis, ascites, gastro-oesophageal varices, hepatocellular carcinoma, hepatic encephalopathy and spontaneous bacterial peritonitis. We also developed an algorithm of ICD-10-AM codes to improve detection of encounters among patients with cirrhosis from large heath system and administrative databases.
How might it impact on clinical practice in the foreseeable future?
These data on the strengths and limitations of the ICD-10-AM will inform future large-scale epidemiological research and healthcare studies, which may be used to guide delivery of health services and strategies to improve health outcomes for people with cirrhosis in Australia and internationally.
Introduction
International Classification of Diseases (ICD) codes stored in large health system databases are increasingly used in epidemiological studies and health services research to explore the burden of cirrhosis and its complications. These data can provide valuable information about the distribution and determinants of chronic liver disease to guide delivery of health services and strategies to improve health outcomes.1 2 However, the utility of ICD codes relies on the accuracy of clinical reporting and administrative coding, which may be influenced by country-specific codes and coding rules, or changes in codes during ICD revisions.3
The 9th revision of the ICD (ICD-9) and, more recently, selected ICD-10 codes for cirrhosis and related complications have been validated in Department of Veterans Affairs (VA) administrative databases4–7 and other US health system databases.8 9 ICD code lists generated from these studies have formed the basis for case definition of many clinical and epidemiological studies of cirrhosis and its complications.8 10 11 While the Australian Modification of the ICD-10 (ICD-10-AM) is similar to its parent classification, the ICD-10-AM does not have specific codes for certain common cirrhosis complications such as spontaneous bacterial peritonitis (SBP) and hepatic encephalopathy (HE), which may lead to discrepancies in the reporting of these problems.
The aim of this study was to: (i) evaluate the performance of ICD-10-AM codes to detect the presence of cirrhosis and a subset of key complications (hepatocellular carcinoma (HCC), ascites, varices, SBP and HE) in admitted episodes of care for which they are coded; (ii) explore limitations of individual codes to accurately detect cirrhosis and complications and (iii) evaluate (and if needed, revise) an algorithm of ICD-10-AM codes for cirrhosis and related complications to identify admitted episodes of care among patients with known cirrhosis, for the purpose of future large-scale epidemiological research and healthcare studies.
Methods
Sample populations and data collection
A retrospective cohort study was conducted in a sample of admitted patient encounters at a major Australian tertiary level hospital to ascertain the level of concordance between select ICD-10-AM codes and documentation in patients’ medical records. As previously described,1 all public and private hospital admissions were ascertained from the Queensland Hospital Admitted Patient Data Collection registry (QHAPDC) between 1 July 2007 and 31 December 2016 for every patient that had at least one encounter during this timeframe which contained an ICD-10-AM code for cirrhosis or related complications; and/or death with a Principal or Other code of interest as a cause of death (‘parent cohort’).
For each ICD-10-AM code examined in the current study, a random sample of admissions at the Princess Alexandra Hospital were ascertained from the ‘parent cohort’ using SPSS V.20.0 (IBM, Armonk, New York, USA). The index pathway used by coders for coding of cirrhosis and related complications remained consistent during the audited time period. Assignment of ICD-10-AM diagnosis codes was impacted by the Australian Coding Standards, which stipulate the condition be either the chief reason for admission, or required commencement, alteration or adjustment of therapeutic treatment; diagnostic procedures; or increased clinical care and/or monitoring during the encounter.
Four clinicians (EEP, ALJ, NTB, BJM) blinded to QHAPDC coding conducted a comprehensive review of patients’ medical records and extracted data for each audited encounter. The presence of cirrhosis, HCC, ascites, varices, SBP and HE was collected. The relevance of each complication to the audited encounter was also categorised as either pertinent (requiring active management or monitoring) or not a current issue (see online supplementary file for additional information).
Supplemental material
Validation cohort
The accuracy of an algorithm of ICD-10-AM codes to identify admitted episodes of care in people with cirrhosis was validated in an independent cohort of 116 patients with known cirrhosis. Patients within this ‘validation cohort’ were prospectively recruited from general hepatology clinics at the Princess Alexandra Hospital between 25 January 2016 and 17 October 2016 for a randomised controlled trial of a medication education intervention. All participants had experienced ascites, HE or variceal bleeding during the preceding 2 years. Data for all hospital encounters during a 12-month follow-up period were collected, and admissions were categorised as liver or non-liver related and unplanned or elective, as previously described.12
Data analysis
Accuracy of ICD-10-AM codes
The accuracy of ICD-10-AM codes to predict the presence of cirrhosis and cirrhosis-related complications was determined by calculating sensitivity, positive predictive value (PPV) and negative predictive value (NPV) (online supplementary table 1). Concordance between data abstracted from the patient’s medical records (gold standard) and ICD-10-AM data from QHAPDC was assessed using misclassification error (%), and Cohen’s kappa coefficient (κ) of agreement. κ values <0.20 indicated poor agreement, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial and 0.81–1.0 almost perfect agreement between the codes for individual items.13 Where more than one code could be used to identify a condition, the accuracy of ‘grouped’ code combinations was also assessed.
ICD-10-AM cirrhosis algorithm development
To optimise detection of cirrhosis-related encounters, an ICD-10-AM code algorithm was constructed using systematic stepwise inclusion of individual codes based on their accuracy to identify cirrhosis. The performance of two algorithms was measured in an independent sample of encounters among patients with known cirrhosis (the ‘validation cohort’). The proportion (%) of encounters and patients for whom at least one encounter was identified by the algorithms is reported. Statistical differences between proportions were calculated using Pearson’s χ2 test.
Results
A random sample of 542 encounters was ascertained, however, clinical notes were unavailable for two encounters. Therefore, a total of 540 encounters among 406 patients were included in the final audit. Mean age (±SD) at the time of encounter was 55.1 (±12.8) years and most encounters were among men (68.1%).
Eighty-seven encounters (16.1%) were abstracted by ≥2 data collectors during the initial stage to verify internal consistency. For all pairs of data collectors, κ was between 0.82 and 0.94. The team discussed and resolved discrepancies by consensus.
Cirrhosis
A total of 413 encounters had evidence of cirrhosis on medical record review. Accuracy of four individual ICD-10-AM codes (K70.3, K74.4, K74.5, K74.6) to detect the presence of cirrhosis was variable, with PPVs ranging from 0.67 to 1.00 (table 1). No single code could reliably exclude the presence of cirrhosis (all NPVs≤0.35). A combination of the four cirrhosis codes provided a high probability (PPV 0.96) that a patient with ≥1 of these codes had cirrhosis documented in the medical record during that encounter. However, the combination under-detected cirrhosis prevalence (only 336 of 413 encounters in patients with cirrhosis were identified (table 2); sensitivity 81.4%; NPV 0.60).
Concordance between select ICD-10-AM codes for cirrhosis or cirrhosis-related complications and medical chart review
Classification tables for cirrhosis and cirrhosis-related complications
The 77 false-negative encounters included admissions with a decompensation event (n=33), day case admissions for a procedure such as large volume paracentesis or upper endoscopy (n=31), non-liver related admissions (n=8), gastro-oesophageal bleeding unrelated to portal hypertension (n=2), liver transplantation (n=2) and sepsis (n=1). The 13 false-positive encounters included alcoholic hepatitis (n=4), day admission for liver biopsy (n=3), acute liver failure (n=2), Alagille syndrome (n=1), autoimmune hepatitis (n=1), alcohol excess (n=1) and previous liver transplant (n=1).
Cirrhosis-related complications
The PPVs for HCC (0.97), ascites (0.97) and ‘grouped varices’ (0.95) were good (table 1). NPVs were comparable (all ≥0.76) and there was acceptable agreement (κ all >0.60) between codes and patients’ medical records. Individual codes for varices were less concordant than ‘grouped’ codes. Overall accuracy was lower for SBP (‘grouped’ PPV 0.75; κ 0.73) and the poorest for HE (‘grouped’ PPV 0.55; κ 0.21).
Despite the high predictive values, ICD-10-AM codes underestimated the true prevalence of ascites and varices by more than one-third (sensitivity 61.9% and 61.3%, respectively). There were 93 encounters in which evidence of ascites was found in the medical record but not coded for (table 2). Of these, 48.4% were in patients with minimal or small volume ascites that did not require active management during the admission. Among 94 encounters that did not include a code for varices, 94.7% had documentation of known varices that were not pertinent to the admission.
In contrast, among 107 false-negative HE encounters (sensitivity 24.6%), HE was present and required active management in 80.4% of patients. The 29 false-positive encounters included documentation of cerebellar ataxia due to alcohol (n=5), acute confusion (related to alcohol n=4; medication n=1; other/unknown cause n=4), Wernicke-Korsakoff syndrome (n=4), encephalopathy of unknown aetiology (n=2), AIDS-related dementia (n=2) and acute psychosis (n=2). The reason for discrepancy could not be ascertained in five encounters.
‘Grouped SBP’ had a high concordance between codes and patients’ medical records (κ 0.73) but failed to identify 23.9% of patients with SBP (sensitivity 76.1%). The 12 false-positive encounters involved peritoneal dialysis-associated peritonitis (n=2), other abdominal collections (n=6), empirical SBP treatment (n=2), initiation of SBP prophylaxis (n=1) and documentation of previous SBP (n=1).
The overall accuracy of ICD-10-AM codes to identify HCC was very good. Of the two false-positive codes, one patient had cholangiocarcinoma and the other had an undifferentiated malignant lesion. Of the 10 false-negative codes, all were in patients with documentation of known HCC that was not managed during the encounter. Two additional patients had documentation of a previously treated/eradicated HCC (classified as ‘not present’ on medical record review) that was not coded.
ICD-10-AM code algorithms to augment detection of cirrhosis
To optimise detection of cirrhosis, the effect of adding select ICD-10-AM codes (based on individual performance to detect cirrhosis; online supplementary table 2) to the four cirrhosis codes (K70.3, K74.4, K74.5, K74.6; algorithm #1) was examined (table 3). Algorithm #6 produced the highest NPV (0.76) while maintaining a reasonable PPV (0.88), κ (0.56) and misclassification error (14.4%). This extended algorithm identified 391 of 413 encounters in patients with cirrhosis (94.7%), at the expense of an increase in false-positives (n=56; table 2). Non-cirrhotic portal hypertension accounted for 50% (n=28) of false-positives.
Accuracy of ICD-10-AM code algorithms to identify the presence of cirrhosis
Independent validation of cirrhosis code algorithms
Accuracy of cirrhosis algorithms was further assessed using the ‘validation cohort’. Of 116 recruited patients with known cirrhosis, 82 had at least one hospital encounter during a 12-month follow-up period (total n=321 encounters). Algorithms #1 and #6 detected 50.5% and 57.0% of encounters, respectively, including ≥1 encounter among 72.0% and 76.8% of patients. Both algorithms identified a greater proportion of ‘liver-related’ compared with ‘non-liver related’ encounters (algorithm #1, 65.4% vs 21.8%; p<0.001, and algorithm #6, 74.9% vs 22.7%; p<0.001), and ‘unplanned’ compared with ‘elective’ encounters (algorithm #1, 60.8% vs 37.1%; p<0.001, and algorithm #6, 68.0% vs 42.9%; p<0.001).
There were 116 ‘unplanned liver-related’ encounters among 39 patients. Algorithm #1 identified 75.9% and algorithm #6 identified 87.1% of these encounters, representing 92.3% and 100% of patients, respectively. Only one-third of elective day admissions (n=111, including for paracentesis (n=61) and upper endoscopy (n=12)) were identified by the algorithms. The ICD-10-AM code for ascites (R18) was present in 30.6% of elective day admissions in the absence of other codes in algorithm #6 (figure 1).
Heatmap depicting prevalence and clustering of select ICD-10-AM codes in 116 unplanned liver-related encounters (A), 111 elective day admissions (B) and 94 ‘other’ encounters (C). Columns represent individual encounters. K* included at least one of: K70.3, K74.4, K74.5, K74.6, K70.4, K72.9, K76.6; I* included at least one of: I85.0, I85.9, I98.3, I98.2, I86.4. ICD-10-AM, Australian modification of the 10th revision of International Classification of Diseases.
Discussion
Administrative databases such as QHAPDC that store health system data using ICD codes are the primary resource for undertaking large-scale epidemiological studies and healthcare research. To our knowledge, this is the first Australian study to evaluate the performance of ICD-10-AM codes for identifying cirrhosis and a subset of key complications. In this health system dataset, 96.3% of encounters that included an ICD-10-AM code for ‘alcoholic cirrhosis’, ‘secondary biliary cirrhosis’, ‘biliary cirrhosis unspecified’ and/or ‘other and unspecified cirrhosis’ (algorithm #1) were in patients that truly had cirrhosis following blind medical chart review. We also found that due to the diverse nature of disease aetiology and patient encounters, each individual cirrhosis code had a low NPV (0.23–0.35), highlighting the importance of using a combination of codes to identify cirrhosis. Our data support findings from prior US studies that found cirrhosis was documented in the medical records of ≥90% of patients with ICD-9 and ICD-10 codes for cirrhosis selected from VA administrative data.5 6
Despite the high PPV of the combined cirrhosis codes, 77 of 413 encounters in patients with cirrhosis were not detected by algorithm #1. The addition of ICD-10-AM codes for specific cirrhosis-related complications in a stepwise algorithm reduced the potential for missing cases of cirrhosis, at the expense of increasing the number of false-positives (particularly encounters in patients with non-cirrhotic portal hypertension). While cirrhosis code algorithms identified a high proportion of ‘unplanned’ and ‘liver-related’ encounters, detection of elective day case admissions (including for a procedure related to portal hypertension) was poor, likely reflecting the quality of documentation and requirement for active chronic liver disease management. The optimal combination of codes to identify patients from administrative databases in future epidemiological and healthcare studies will vary depending on the goal of each study.
Regardless of the aetiology of cirrhosis, most of the associated morbidity/mortality and high use of hospital services occurs in patients with HCC or portal hypertension-related events. Accurate coding of these is crucial to track patient outcomes, healthcare burden and economic impact of cirrhosis.10 Similar to findings in VA data,5 we found that ICD-10-AM codes for ascites, gastro-oesophageal varices and HCC were relatively accurate. We also found the probability that patients without these codes didn’t have these complications was very high for HCC (97.9%), but substantially lower for ascites (75.8%) and varices (75.5%). Our data suggest that this was likely due to variable clinical documentation and hence coding (in accordance with the Australian Coding Standards) of non-bleeding varices and small volume ascites that did not require management during the encounter. Importantly, accuracy of ICD-10-AM codes for HE and SBP were not as robust as for other cirrhosis-related complications.
Improved clinical documentation (particularly clearer documentation of active problems and reduced use of abbreviations) may improve accuracy of coding for cirrhosis and ascites, as we found over 50% of false-negative encounters for these conditions were in patients who required active management during admission. This would particularly improve accuracy of coding among day case admissions for cirrhosis-related procedures, thereby increasing awareness of health services use among these patients. However, improved documentation may have limited impact on coding accuracy for HE and SBP in the absence of specific codes for these complications. The complexity and lack of consistency in the coding of HE14 is likely to lead to a substantial underestimation of the actual burden of this debilitating complication.15 Similarly, the absence of a specific code for SBP may result in inadequate capture of this information, which is concerning given its association with significant in-hospital mortality and resource utilisation among patients with cirrhosis.16 Recognition of these limitations will be particularly important to improve the classification of SBP and HE in Australia, and to ‘futureproof’ transition to the 11th revision of the ICD, which the WHO proposes for adoption from 2022.17
Strengths of our study include the large cohort of encounters and detailed review of medical records performed by clinicians (including an experienced hepatologist), to evaluate the documentation of cirrhosis, cirrhosis-related complications, and their relevance to the hospital admission. The study was conducted in a single large tertiary hospital in Queensland and quality of clinical documentation (and hence coding) at other Australian hospitals may differ (in particular small hospitals in regional areas). However, internal auditing processes are regularly conducted at a hospital, Health Service and jurisdictional level for quality assurance purposes, which are supported by national data validation activities. As in previous studies,5 a key limitation is that the patient cohort was identified using a list of ICD-10-AM diagnosis codes for cirrhosis and related complications. Therefore, there was a high prevalence of cirrhosis in the cohort, which impacts PPV and NPV.
Nevertheless, we have shown that specific ICD-10-AM codes accurately identified encounters among patients with cirrhosis and related complications (HCC, ascites and varices) admitted to a Queensland hospital. Our data also show the importance of taking a number of codes into consideration when using administrative databases to study the burden of cirrhosis in Australia, to avoid an underestimation of hospital encounters and mortality from this burgeoning chronic disease. We recommend adoption of specific codes for HE and SBP to align Australia’s coding standard with the international community, and improved clinician training about the importance of accurate clinical documentation.
Acknowledgments
The authors acknowledge and thank Ms Catherine Li for her technical assistance during the study.
References
Footnotes
EEP and PCV contributed equally.
Contributors PCV and EEP conceived and planned the study. PCV and KLH obtained ICD-10-AM data from QHAPDC, and KLH and LUH coordinated clinical data collection by ALJ, BJM, NTB and EEP. CL managed the databases and data entry was cross-checked by KLH and ALJ. KLH, GH and VB merged and analysed the data. PCV, EEP, KLH, GH and CM interpreted study findings. PCV, EEP and KLH drafted the manuscript. All authors reviewed and approved the final version for publication and have agreed to be accountable for all aspects of the work.
Funding KLH was supported by a Health Innovation, Investment and Research Office (HIIRO) Clinical Research Fellowship. PCV was supported by an Australian National Health and Medical Research Council (NHMRC) Career Development Fellowship (no. 1083090).
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data have not been made publicly available due to requirements of Research Ethics and Governance approvals. Requests to collaborate and share data may be directed to the corresponding author.