ORIGINAL ARTICLE

Validation of a case definition for osteoporosisdisease surveillance

W. D. Leslie & L. M. Lix & M. S. Yogendran

Received: 27 October 2009 /Accepted: 1 March 2010 /Published online: 11 May 2010# International Osteoporosis Foundation and National Osteoporosis Foundation 2010

AbstractSummary A simple case definition for osteoporosis casediagnosis is feasible based upon administrative health data.This may facilitate implementation of a population-basedosteoporosis surveillance program, providing informationthat could help to inform and guide screening, prevention,and treatment resources.Introduction Our aim was to construct and validate a sim-plified algorithm for osteoporosis case ascertainment fromadministrative databases that would be suitable for diseasesurveillance.Methods Multiple classification rules were applied to differ-ent sets of hospital diagnosis, physician claims diagnosis, andprescription drug variables from Manitoba, Canada. Algo-rithms were validated against results from a regional bonemineral density testing program that identified bone mineraldensity (BMD) measurements in 4,015 women age 50 yearsand older with at least one BMD test between April 1, 2000and March 31, 2001.Results Sensitivity as high as 93.3%was achievedwith 3 yearsof data. Specificity ranged from 50.8% to 91.4% overall, andfrom 81.2% to 99.1% for discriminating osteoporotic fromnormal BMD. Sensitivity and overall accuracy were generally

lower for algorithms based on diagnosis codes alone than foralgorithms that included osteoporosis prescriptions. In thesubgroup without prior osteoporotic fractures or chroniccorticosteroid use, one simple algorithm (one hospital diagno-sis, physician claims diagnosis, or osteoporosis prescriptionwithin 1 year) gave accuracy measures exceeding 90% fordiscriminating osteoporosis from normal BMD across a widerange of disease prevalence.Conclusions A relatively simple case definition for osteopo-rosis surveillance based upon administrative health data canachieve an acceptable level of sensitivity, specificity, andaccuracy. Performance is enhanced when the case definitionincludes osteoporosis medication use in the formulation.

Keywords Administrative data . Bone densitometry .

Osteoporosis . Public health . Surveillance

Introduction

Osteoporosis is a disease characterized by decreased bonemass and increased fracture risk. It represents a significantpopulation health issue because of its known negativeeffects on mortality, quality of life, and other healthoutcomes [1–3], and also because of its increasingprevalence due to an aging population [4, 5]. Population-based surveillance of disease has become an importantcomponent of addressing common chronic diseases. Suchsystems can potentially guide screening, prevention, andtreatment resources. The operational definition of osteopo-rosis developed by the World Health Organization (WHO)is based on the measurement of bone mineral density(BMD), which is not easily captured on a population-widebasis [6]. Cohort studies involving primary data collectionhave been used to estimate osteoporosis prevalence [7, 8],

W. D. Leslie (*)Department of Medicine (C5121), University of Manitoba,409 Tache Avenue,Winnipeg, Manitoba R2H 2A6, Canadae-mail: bleslie@sbgh.mb.ca

L. M. LixSchool of Public Health, University of Saskatchewan,Saskatoon, Canada

M. S. YogendranManitoba Centre for Health Policy, University of Manitoba,Winnipeg, Canada

Osteoporos Int (2011) 22:37–46DOI 10.1007/s00198-010-1225-2

but such studies are expensive and time-consuming toconduct. Population-based administrative databases areincreasingly being explored for their potential to provideepidemiological information (i.e., incidence, prevalence,geographic, and socioeconomic disparities) about osteopo-rosis [9–14] and other chronic diseases [15–17]. Theadvantages of using administrative data include: (a) it isrelatively inexpensive to establish and maintain apopulation-based surveillance system using these databases,(b) longitudinal studies can be conducted using data linkagetechniques, and (c) disease cases and non-cases can becompared on co-morbid conditions.

Osteoporosis and fracture diagnosis codes have beenused in previous studies to ascertain disease cases inadministrative databases. Administrative data have beenused to construct case ascertainment algorithms for anumber of other chronic diseases including diabetes,asthma, and inflammatory bowel disease [15, 16, 18–23].However, osteoporosis is known to be under-diagnosed inadministrative databases when diagnosis codes are the solesource of case ascertainment [10]. This is not unique toosteoporosis; incomplete capture of diagnoses in adminis-trative data has been documented for other chronic diseases,such as hypertension [24]. One solution is to develop a caseascertainment algorithm that does not rely exclusively ondiagnosis codes. For example, osteoporosis drug treatmentshave been proposed for case ascertainment in pharmacydatabases [13, 25]. The objective of this study was to assessthe validity of various osteoporosis case definitions frompopulation-based hospital, physician, and pharmacy admin-istrative data. Validity was assessed using data from aregional BMD testing program in Manitoba, Canada.

Methods

Data sources

Population-based hospital, physician, and pharmacy admin-istrative data as well as BMD testing data were fromManitoba, a centrally located province in Canada with apopulation of 1.2 million [26] and a system of universalhealth care. Data were from the Manitoba Centre for HealthPolicy Repository [27]. Ethics approval was received fromthe University of Manitoba Health Research Ethics Boardand approval for data access was granted by the ManitobaHealth Information Privacy Committee.

A hospital abstract is completed when a patient isdischarged from an acute care facility. Each record includesup to 16 five-digit diagnosis codes from the InternationalClassification of Diseases, 9th Revision, Clinical Modifi-cation (ICD-9-CM). Physicians who are paid on a fee-for-service basis submit billing claims to the provincial

ministry of health; these claims capture almost all outpa-tient services, including those for hospital emergencydepartments and outpatient departments. Physician claimscontain a single three-digit ICD-9-CM code. While somephysicians are salaried (about 7% of family physicians[28]), it has been estimated that approximately 90% ofthese physicians submit parallel billing claims for admin-istrative purposes. Thus, the billing claims database isvirtually complete. Prescription drug records for all resi-dents of Manitoba, regardless of age, are captured in theDrug Programs Information Network (DPIN), a centralized,electronic, point-of-sale pharmacy database connecting allretail pharmacies in Manitoba. DPIN collects a variety ofinformation for each dispensation, including date, drugname, national drug identification number (DIN), dosageform, and quantity dispensed. DINs are linked to Anatom-ical Therapeutic Chemical codes developed by the WHOthat classify drugs into groups according to the organ orsystem on which they act and/or their therapeutic andchemical characteristics.

BMD tests were used as the gold standard to validatealgorithms constructed from administrative data. BMDtesting is performed under the Manitoba Bone DensityProgram [29]. The Program’s database includes all DXAtest results since the first instrument was installed in 1990[30]. The database is over 99% complete and accurate asjudged by chart audit [30]. BMD testing in Manitoba isconsidered a diagnostic test, not a screening test, andrequires physician referral. Criteria for testing are broadlyconsistent with most published guidelines, and emphasizethe importance of female sex, age 65 years or older,premature ovarian failure, prior fragility fractures or X-ray evidence of osteopenia, prolonged corticosteroid use,and other clinical risk factors. Access to testing is notrestricted to these indications, however, and most clinicaljustifications are accepted. Each record in the BMDdatabase contains information about the patient, the dateof testing and results for BMD tests on the lumbar spine,total hip, and an optional third site. Vertebral levelsaffected by artifact were excluded by experiencedphysicians using conventional criteria [31]. The BMDmeasurements are converted to T-scores. According toWHO guidelines, a diagnosis of osteoporosis is assigned ifan individual's T-score is 2.5 or more standard deviationsbelow the average BMD measurement for a young adultfemale [32].

Study population

This study adopted a retrospective cohort design in whichosteoporosis cases and non-cases were identified from theBMD database, their clinical records were linked toadministrative data, and then a series of data features from

38 Osteoporos Int (2011) 22:37–46

the administrative data were selected to distinguish betweenthe two groups.

There were 5,527 individuals with at least one BMD testbetween April 1, 2000 and March 31, 2001, the fiscal yearselected for validation. Administrative data to define the datafeatures were from the 2000/01 to 2003/04 fiscal years.Individuals that did not have continuous health insurancecoverage under the Manitoba Health Services Insurance Planfor the duration of this study period (N=327; 5.9%) wereexcluded. Additional exclusions were males, because ofsmall numbers and the potential for referral bias, andindividuals for who sex was not recorded. Existing clinicalguidelines do not recommend the use of BMD alone todiagnose osteoporosis in women under 50 years of age,therefore all individuals below this age were excluded [33].A total of 4,015 females 50 years of age and older wereretained in the cohort. Cases had a minimum T-score foreither the hip or spine 2.5 standard deviations or more belowthe reference mean [34, 35]. Non-cases had a minimumT-score greater than this, and included individuals with anormal T-score, less than one standard deviation below thereference value, as well as individuals with osteopenia,operationalized as a T-score between −2.5 and −1.0 standarddeviations below the reference mean. All BMD records inthe cohort were successfully linked to administrative data viaan anonymized personal health identification number.

Bone density measurements

BMD measurement performed between April 1, 2000 andMarch 31, 2001 used a single fan-beam scanner configuration(Prodigy, GE Healthcare). Three identical instruments werecross-calibrated using anthropomorphic phantoms and noclinically significant differences were identified (T-scoredifferences<0.1). Therefore, all analyses are based upon theunadjusted numerical results provided by the instrument.Scans were performed and analyzed in accordance withmanufacturer recommendations. Lumbar spine T-scores werecalculated using the manufacturer's USA White femalereference values. Hip T-scores were calculated from therevised NHANES III White female reference values (Prodigyversion 8.8) [36, 37]. Densitometers showed stable long-termperformance (coefficient of variation [CV]<0.5%) andsatisfactory in vivo precision (CV 1.7% for L1–4 and 1.1%for the total hip) [38].

Construction of algorithms

Fifteen algorithms were defined using single or multipleyears of data following the data of the index BMD test, andwere based on the number of osteoporosis diagnosesascertained from physician claims, osteoporosis diagnosesascertained from hospitalizations, and numbers of osteopo-

rosis prescriptions dispensed. For example, one algorithmthat has been examined in previous case ascertainmentresearch classifies individuals as disease cases if they haveat least one hospitalization or one physician claim with therelevant diagnosis (i.e., 1+ H or 1+ P) [20]. Anotheralgorithm classifies disease cases if they have at least onehospitalization or at least two physician claims with therelevant diagnosis (i.e., 1+ H or 2+ P).

The index date was the date of the first BMD test for acase or non-case in the 2000/01 fiscal year. Both single andmultiple years of data were used because previous researchhas shown improvements in algorithm sensitivity for otherchronic diseases when multiple years of data are used toidentify disease diagnoses [21, 22]. As diagnosis and drugintervention may be affected by non-BMD risk factors forosteoporosis such as prior fractures or prolonged systemiccorticosteroid therapy, analyses were also conducted in thecohort after exclusion of major fractures in the 3 years priorto the index date (diagnoses in hospital and physicianrecords for fractures of the hip [ICD-9-CM 820-821],vertebra [ICD-9-CM 805], humerus [ICD-9-CM 812, orwrist [ICD-9-CM 813-814]) as well as those with apharmacy dispensation for a systemic corticosteroid med-ication in the 3 years prior to the index date.

The number of osteoporosis diagnoses (ICD-9-CM733.01–733.09) in hospital separations and physician claimswere recorded. In the latter data source, diagnoses are onlycaptured to the third digit (i.e., ICD-9-CM 733), resulting in alack of specificity as this code also includes pathologicfracture (733.1), cyst of bone (733.2), hyperostosis of skull(733.3), aseptic necrosis of bone (733.4), osteitis condensans(733.7), malunion and nonunion of fracture (733.8), and otherunspecified disorders of bone and cartilage (733.9). Thenumber of prescriptions drug dispensations for osteoporosistreatments, including oral bisphosphonates, parenteral salmoncalcitonin, and raloxifene, were recorded from the pharmacydata [39]. Although the vast majority of these prescriptionsare for osteoporosis indications, bisphosphonates may alsobe prescribed for Paget’s disease or hypercalcemia ofmalignancy, while raloxifene may be used for reduction inrisk of invasive breast cancer. Specific anabolic therapies(teriparatide and strontium ranelate) and were not availableduring the years studied. Intravenous bisphosphonates wereexcluded as they were usually used for indications other thanosteoporosis, and systemic estrogen preparations wereexcluded since they are often used for treatment ofmenopausal symptoms.

Statistical methods

Performance indicators for classification of osteoporosiscases and normal or osteopenic non-cases were calculatedfor each algorithm [40]. Women with osteopenic BMD

Osteoporos Int (2011) 22:37–46 39

were included with the normal non-cases for the primaryperformance evaluation. However, secondary analyses wereperformed in which women with osteopenic BMD wereexcluded since there is no ICD-9-CM diagnosis code forosteopenia and since this category spans such a broad range(from −2.4 which is virtually identical in terms of manage-ment as someone at the WHO threshold for osteoporoticBMD, to −1.1 which is virtually identical in terms ofmanagement as someone at the WHO threshold for normalBMD). Sensitivity was defined as percentage of women withan osteoporosis diagnosis or prescription for osteoporosistreatment (depending on the algorithm) identified from amongall women with osteoporotic BMD. Conversely, specificitywas defined as the percentage of women without anosteoporosis diagnosis or prescription for osteoporosis treat-ment (depending on the algorithm) from among all non-cases.The positive predictive value (PPV) was the number ofwomen with osteoporotic BMD as a percentage of all womenidentified by the algorithm. Similarly, the negative predictivevalue (NPV) was the number of women with normal orosteopenic BMD as a percentage of all women not identifiedby the algorithm. Area under the receiver operating charac-teristic curve (AUC) was computed as a global measure ofalgorithm performance. A discriminating test would havesensitivity, specificity, PPVand NPV close to 100%, and AUCclose to 1.00. Since performance can be affected by diseaseprevalence and since the prevalence of osteoporosis varies bygender and across the age spectrum (from 13% at age 50–54 toover 50% after age 85 in women [41]), overall accuracy (i.e.,the fraction of all women correctly classified by thealgorithm) was estimated for different osteoporosis preva-

lence values (from 10% to 50%). These prevalence rangesinclude the average values for Canadian women 50+ years ofage based upon for more than 10,000 individuals studied innine sites across Canada (prevalence of osteoporosis usingspine and femoral neck BMD measurements 15.8%) [8].

Results

A total of 1,277 (31.8%) females 50+ years of age weredefined as osteoporosis cases in the cohort based on theminimum BMD T-score for the hip or spine. All of theseindividuals had a single BMD scan in 2000/01. Almost allwere new cases; only 5.3% had a test in the 5 years prior tocohort definition. Moreover, virtually all (99.4%) of thenon-cases remained as non-cases for a 3-year periodfollowing their index scan.

The demographic characteristics of cases and non-casesare described in Table 1. Descriptive data are presentedseparately for non-cases with a T-score indicating osteope-nia and a T-score indicating normal BMD. The mean agefor all cases was 68.7 years (standard deviation [SD]=8.9).For osteopenia non-cases, the mean age was 63.7 years(SD=8.8) and for normal BMD non-cases it was 61.0 years(SD=8.5). For comparison, demographic data wereobtained for the entire Manitoba female population 50+years of age as of December 2000. Overall, 44.7% of thestudy cohort was between 50 and 64 years of age comparedto 49.1% of the Manitoba population; 17.2% of the studycohort was 75+years of age compared to 27.3% of theManitoba population.

Table 1 Descriptive for the study cohort

Cases Non-cases

Osteoporotic BMD N=1,277 Osteopenic BMD N=1,643 Normal BMD N=1,095

f % f % f %

Age

50–64 years 426 33.4 915 55.7 752 68.7

65–74 years 483 37.8 497 30.2 251 22.9

75+ years 368 28.8 231 14.1 92 8.4

Physician claims for osteoporosis diagnosisa

1 or more–1 year 886 69.4 767 46.7 80 7.3

1 or more–2 year 945 74.0 850 51.7 110 10.0

1 or more–3 year 997 78.1 915 55.7 126 11.5

Prescription for osteoporosis treatment

1 or more–1 year 1002 78.5 694 42.2 108 9.9

1 or more–2 years 1050 82.2 761 46.3 126 11.5

1 or more–3 years 1089 85.3 824 50.2 143 13.1

a Too few hospital separations with an osteoporosis diagnosis to report

40 Osteoporos Int (2011) 22:37–46

Table 1 describes the characteristics of cases and non-casesin hospital, physician, and pharmacy administrative data.More than two-thirds of the cases had at least one physicianclaim with an osteoporosis diagnosis (i.e., ICD-9-CM 733) inthe 1-year period following the index BMD test. More thanthree-quarters had at least one physician claim with anosteoporosis diagnosis when up to 3 years of data followingthe index test were examined. However, almost one-third ofnon-cases also had an osteoporosis diagnosis in physicianclaims in the 1-year period following a negative BMD test,and most of these individuals were in the osteopenia category.There were too few cases and non-cases with an osteoporosisdiagnosis in hospital records to report these results.

More than three-quarters of cases had at least oneprescription for an osteoporosis pharmacologic treatmentin the 1-year period following the index BMD test. Thisfigure increased to 85.3% when up to 3 years of datafollowing the index data were examined. Slightly morethan 40% of osteopenia non-cases also received at leastone prescription in a 1-year period and this figureincreased to about 50% when 3 years of administrative datawere examined. The average and median number of prescrip-tions in a 1-year period was as follows: osteoporosis cases,mean=4.8 (SD=4.3; median=4); osteopenia non-cases,mean=2.1 (SD=3.3; median=1); normal BMD non-cases, mean=0.5 (SD=1.7; median=0).

Estimates of sensitivity, specificity, PPV, and NPV areprovided in Table 2 for different algorithms. Using only a

single year of data, sensitivity for osteoporosis caseidentification ranged from a minimum of 34.1% (algorithm3 [1+H or 2+P in 1 year]) to a maximum of 89.4%(algorithm 4 [1+H or 1+P or 1+Rx in 1 year). Specificityfor all non-cases ranged from 57.2% (algorithm 4) to 91.4%(algorithm 3). When osteopenia non-cases were excludedspecificity ranged from 91.5 % (algorithm 4) to 99.7%(algorithm 3). Inclusion of prescription data led to asignificant improvement in sensitivity (algorithm 4 [1+Hor 1+P or 1+Rx in 1 year], 89.4% versus algorithm 2 [1+Hor 1+P in 1 year], 69.4%; algorithm 5 [1+H or 2+P or 2+Rx in 1 year], 77.7% versus algorithm 3 [1+H or 2+P in1 year], 34.1%). There was a corresponding reduction inoverall specificity (algorithm 4, 57.2% versus algorithm 2,69.0%; algorithm 5, 72.0% versus algorithm 3, 91.4%).Based upon only a single year of data, the highest AUC(0.749) was seen with algorithm 4.

Similarly, inclusion of multiple years of data led to asmall increase in sensitivity for algorithms based uponequal numbers of codes, with a slight reduction inspecificity. The highest sensitivity was achieved with algorithm14 (1+H or 1+P or 1+Rx in 3 years) at 93.3%, with acorresponding overall specificity of 50.8% (86.8% if osteope-nia non-cases excluded). The highest AUC (0.754) was seenwith algorithm 10 (1+H or 2+P or 2+Rx over 2 years).

Table 3 indicates estimated overall accuracy measuresfor different prevalence rates of osteoporosis. At theosteoporosis prevalence observed in the study cohort

Table 2 Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and area under the receiver operatingcharacteristic curve (AUC)

Number ofyears of data

Algorithmnumber

Algorithm definitions Sensitivity (%) Specificity (%) PPV (%) NPV (%) AUCAll cases With/without

osteopenianon-cases

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

All cases

1 1 1+Rx 78.5 70.7/95.3 55.5/93.1 87.6/80.8 0.746

2 1+H or 1+P 69.4 69.0/94.5 51.1/91.8 82.9/79.5 0.692

3 1+H or 2+P 34.1 91.4/99.7 64.9/98.9 74.8/63.8 0.627

4 1+H or 1+P or 1+Rx 89.4 57.2/91.5 49.4/89.8 92.1/89.2 0.733

5 1+H or 2+P or 2+Rx 77.7 72.0/96.4 56.4/94.6 87.4/80.3 0.749

2 6 1+Rx 82.2 67.6/93.9 54.2/91.7 89.1/83.2 0.749

7 1+H or 1+P 74.0 64.9/92.2 49.6/89.5 84.3/82.1 0.695

8 1+H or 2+P 43.9 86.0/98.2 59.4/95.2 76.7/66.4 0.650

9 1+H or 1+P or 1+Rx 92.0 53.7/88.9 48.1/87.5 93.5/91.5 0.729

10 1+H or 2+P or 2+Rx 83.4 67.4/94.7 54.4/92.8 89.7/83.9 0.754

3 11 1+Rx 85.3 64.7/92.4 53.0/90.3 90.4/85.7 0.750

12 1+H or 1+P 78.1 61.9/91.0 48.9/88.4 85.8/84.0 0.700

13 1+H or 2+P 50.9 82.3/97.8 57.2/95.2 78.2/69.4 0.666

14 1+H or 1+P or 1+Rx 93.3 50.8/86.8 47.0/85.7 94.3/92.7 0.721

15 1+H or 2+P or 2+Rx 86.4 63.6/93.1 52.6/91.2 90.9/85.3 0.750

P physician claims, H hospitalizations, Rx osteoporosis prescriptions

Osteoporos Int (2011) 22:37–46 41

(31.8%), the highest overall accuracy was 73.8% achievedwith algorithm 5 (1+H or 2+P or 2+Rx in 1 year). Ifosteopenia non-cases were excluded, the highest accuracywas 91.1% with algorithm 10 (1+H or 2+P or 2+Rx in1 year). At a prevalence of 20% the maximum overallaccuracy was 79.9% with algorithm 3 (1+H or 2+P in1 year), and up to 92.7% for algorithm 5 if osteopenia non-cases were excluded. Exclusion of hospital data from thealgorithms had no discernible effect on the performancemeasures (data not shown).

Performance characteristics for the various algorithmswere recalculated after exclusion of women with higherosteoporotic fractures or chronic corticosteroid use duringthe 3 years prior to the index BMD test. Sensitivitymeasures were similar but there was some improvementin specificity for all algorithms evaluated (Table 4). Thehighest AUC (0.775) was seen with algorithm 11 (1+Rxover 3 years). At the osteoporosis prevalence observed inthe study cohort, the highest overall accuracy was 77.9%achieved with algorithm 5 (1+H or 2+P or 2+Rx in 1 year)and up to 90.1% for algorithm 4 (1+H or 1+P or 1+Rx in1 year) if osteopenia non-cases were excluded (Table 5). Ata prevalence of 20%, the highest overall accuracy was80.6% for algorithm 3 (1+H or 2+P in 1 year); ifosteopenia non-cases were excluded then seven algorithmsgave accuracy measures exceeding 90% with the highest91.6% for algorithm 5 (1+H or 2+P or 2+Rx in 1 year).

To evaluate robustness of the case definition over time,algorithm 2 (1+H or 1+P in 1 year) and algorithm 4 (1+Hor 1+P or 1+Rx in 1 year) were applied to the studypopulation based upon 1 year of codes during the first,second, and third years after the index BMD test (Fig. 1).Sensitivity for case identification with algorithm 2 (whichdid not include prescription information) showed a markeddeterioration after the first year (year one 69.4%, year two30.8%, year three 29.1%). In contrast, algorithm 4 (whichincluded prescription data) was much less affected (yearone 89.4%, year two 76.9%, year three 75.1%). Specificityfor all non-cases increased from 69.0% in year one to89.3% in year three for algorithm 2, and from 57.2% inyear one to 70.0% in year three for algorithm 4. Whenosteopenic non-cases were excluded from the specificityevaluation, there was a significant improvement in perfor-mance during year one for algorithm 2 (92.7%) andalgorithm 4 (86.1%). During years two and three specificityfor algorithm 2 improved to 97.2%, and for algorithm 4improved to 90.8%.

Discussion

This study constructed algorithms to ascertain osteoporosiscases in administrative data and validated these algorithmsusing test results from a regional bone density testing

Table 3 Estimated accuracy at different prevalences of osteoporosis

Numberof yearsof data

Algorithmnumber

Algorithmdefinitionsa

Accuracy (%)

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

Prevalence = 31.8% 10% 20% 30% 40% 50%

1 1 1+Rx 73.2/90.0 71.5/93.6 72.3/91.9 73.0/90.3 73.8/88.6 74.6/86.9

2 1+H or 1+P 69.1/86.5 69.0/92.0 69.1/89.5 69.1/87.0 69.2/84.5 69.2/82.0

3 1+H or 2+P 73.2/78.8 85.7/93.1 79.9/86.6 74.2/80.0 68.5/73.5 62.8/66.9

4 1+H or 1+P or 1+Rx 67.4/90.8 60.4/91.3 63.6/91.1 66.9/90.9 70.1/90.7 73.3/90.5

5 1+H or 2+P or 2+Rx 73.8/90.5 72.6/94.5 73.1/92.7 73.7/90.8 74.3/88.9 74.9/87.1

2 6 1+Rx 72.2/90.2 69.1/92.7 70.5/91.6 72.0/90.4 73.4/89.2 74.9/88.1

7 1+H or 1+P 67.8/86.4 65.8/90.4 66.7/88.6 67.6/86.7 68.5/84.9 69.5/83.1

8 1+H or 2+P 72.6/80.9 81.8/92.8 77.6/87.3 73.4/81.9 69.2/76.5 65.0/71.1

9 1+H or 1+P or 1+Rx 65.9/89.9 57.5/89.2 61.4/89.5 65.2/89.8 69.0/90.1 72.9/90.5

10 1+H or 2+P or 2+Rx 72.5/91.1 69.0/93.6 70.6/92.4 72.2/91.3 73.8/90.2 75.4/89.1

3 11 1+Rx 71.3/90.1 66.8/91.7 68.8/91.0 70.9/90.3 72.9/89.6 75.0/88.9

12 1+H or 1+P 67.1/86.9 63.5/89.7 65.1/88.4 66.8/87.1 68.4/85.8 70.0/84.6

13 1+H or 2+P 72.3/82.9 79.2/93.1 76.0/88.4 72.9/83.7 69.7/79.0 66.6/74.4

14 1+H or 1+P or 1+Rx 64.3/88.9 55.1/87.5 59.3/88.1 63.6/88.8 67.8/89.4 72.1/90.1

15 1+H or 2+P or 2+Rx 70.9/91.0 65.9/92.4 68.2/91.8 70.4/91.1 72.7/90.4 75.0/89.8

a Observed prevalence in the cohort

42 Osteoporos Int (2011) 22:37–46

program. The analysis looked at osteoporosis case identi-fication using a limited set of administrative data basedupon hospitalizations, physician claims, and prescriptiondispensations. The current analysis demonstrates that an

acceptable level of performance can be achieved using datasources that are available in many administrative healthsystems. However, there was poor sensitivity for identifyingwomen with osteoporosis when using data beyond a 1-year

Table 5 Estimated accuracy at different prevalences of osteoporosis with exclusions for prior osteoporotic fractures and chronic corticosteroid use

Numberof yearsof data

Algorithmnumber

Algorithm definitionsa Accuracy (%)

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

Prevalence = 31.8% 10% 20% 30% 40% 50%

1 1 1+Rx 77.5/88.4 78.7/93.1 78.1/91.0 77.6/88.8 77.0/86.7 76.5/84.5

2 1+H or 1+P 71.4/87.2 71.3/92.2 71.3/89.9 71.4/87.7 71.4/85.4 71.5/83.1

3 1+H or 2+P 73.8/78.9 86.4/93.2 80.6/86.6 74.9/80.1 69.1/73.6 63.3/67.1

4 1+H or 1+P or 1+Rx 71.6/90.1 66.6/91.1 68.9/90.6 71.1/90.2 73.4/89.7 75.7/89.3

5 1+H or 2+P or 2+Rx 77.9/88.8 79.6/94.0 78.8/91.6 78.1/89.3 77.3/86.9 76.5/84.5

2 6 1+Rx 76.7/88.8 76.3/92.3 76.5/90.7 76.6/89.1 76.8/87.5 77.0/85.9

7 1+H or 1+P 70.5/87.3 68.5/90.7 69.4/89.1 70.3/87.6 71.2/86.0 72.2/84.5

8 1+H or 2+P 73.6/80.5 83.6/92.6 79.0/87.0 74.5/81.5 69.9/75.9 65.3/70.3

9 1+H or 1+P or 1+Rx 70.2/89.4 63.8/89.1 66.7/89.2 69.7/89.4 72.6/89.5 75.6/89.7

10 1+H or 2+P or 2+Rx 76.6/89.7 75.9/93.1 76.3/91.5 76.6/90 76.9/88.4 77.3/86.8

3 11 1+Rx 75.9/89.1 73.9/91.4 74.8/90.3 75.7/89.3 76.6/88.2 77.5/87.2

12 1+H or 1+P 70.2/87.5 67.0/89.9 68.5/88.8 69.9/87.7 71.4/86.6 72.8/85.5

13 1+H or 2+P 73.9/82.6 81.6/93.0 78.1/88.2 74.5/83.4 71.0/78.6 67.5/73.9

14 1+H or 1+P or 1+Rx 68.7/88.5 61.2/87.3 64.6/87.8 68.1/88.4 71.5/88.9 74.9/89.4

15 1+H or 2+P or 2+Rx 75.2/89.8 72.7/92.1 73.8/91.0 75.0/90.0 76.1/89.0 77.2/88.0

a Observed prevalence in the cohort

Table 4 Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and area under the receiver operatingcharacteristic curve (AUC) with exclusions for prior osteoporotic fractures and chronic corticosteroid use

Number ofyears data

Algorithmnumber

Algorithmdefinitions

Sensitivity (%) Specificity (%) PPV (%) NPV (%) AUCAll cases With/without

osteopenianon-cases

With/withoutosteopenianon-cases

With/withoutosteopenianon-cases

All cases

1 1 1+Rx 78.5 79.2/95.3 56.8/93.1 89.1/80.8 0.765

2 1+H or 1+P 69.4 71.2/94.5 47.9/91.8 87.2/79.5 0.714

3 1+H or 2+P 34.1 92.2/99.7 62.1/98.9 79.2/63.8 0.633

4 1+H or 1+P or 1+Rx 89.4 64.3/91.5 47.5/89.8 93.1/89.2 0.757

5 1+H or 2+P or 2+Rx 77.7 80.4/96.4 57.8/94.6 88.8/80.3 0.765

2 6 1+Rx 82.2 76.1/93.9 54.6/91.7 90.3/83.2 0.770

7 1+H or 1+P 74.0 67.6/92.2 46.7/89.5 88.7/82.1 0.722

8 1+H or 2+P 43.9 88.2/98.2 57.0/95.2 80.5/66.4 0.653

9 1+H or 1+P or 1+Rx 92.0 60.8/88.9 46.0/87.5 94.5/91.5 0.756

10 1+H or 2+P or 2+Rx 83.4 75.6/94.7 54.4/92.8 90.6/83.9 0.772

3 11 1+Rx 85.3 73.0/92.4 52.9/90.3 91.7/85.7 0.775

12 1+H or 1+P 78.1 65.6/91.0 46.2/88.4 89.8/84.0 0.727

13 1+H or 2+P 50.9 85.1/97.8 55.3/95.2 82.1/69.4 0.675

14 1+H or 1+P or 1+Rx 93.3 57.8/86.8 44.7/85.7 95.1/92.7 0.749

15 1+H or 2+P or 2+Rx 86.4 71.6/93.1 51.9/91.2 91.8/85.3 0.772

P physician claims, H hospitalizations, Rx osteoporosis prescriptions

Osteoporos Int (2011) 22:37–46 43

time horizon unless pharmacy claims data were used in thealgorithm. Remarkably, even a single pharmacy dispensationfor an osteoporosis medication (1+Rx) gave reasonableresults (AUCs of 0.746, 0.749, and 0.750 for 1, 2, and 3 yearsof data). This has potential clinical and public healthimplications for implementing osteoporosis surveillancestrategies.

No single algorithm was found to be optimal for alllevels of disease prevalence. Choice would also beinfluenced by the relative importance of sensitivity versus

specificity, and completeness of the data sources. Wherepharmacy data are available then algorithm 5 (1+H or 2+Por 2+Rx in 1 year) showed good overall performance in thestudy cohort after exclusions for prior osteoporotic fracturesand chronic corticosteroid use (accuracy 77.9% and AUC0.765). Where pharmacy data are not available it ispreferable to combine multiple years of data, and algorithm12 (1+H or 1+P in 3 years) gave an overall accuracy of70.2% and AUC 0.727 after exclusions for prior osteopo-rotic fractures and chronic corticosteroid use.

An unavoidable challenge to using such algorithms is thefact that coding practices and comprehensiveness of thedata sources can differ between countries and among healthplans. Implementation is facilitated by access to well-organized, population-based databases such as those foundin Canada under universal health care. As a result, thefindings of the further validation and/or adaptation studiesshould be undertaken with the proposed algorithms in otherhealth care environments to assess their generalizability.

The advantage in including osteoporosis prescriptionmedication includes significant improvement in sensitivityand overall classification accuracy. Although there is somereduction in specificity, this partially relates to non-BMDindications for osteoporosis medication initiation such asprior osteoporotic fractures and chronic corticosteroid use.Although BMD was used as the gold standard in theosteoporosis case ascertainment here, individuals with priorosteoporotic fractures would also be considered to haveosteoporosis on clinical grounds. Similarly, the thresholdfor osteoporosis diagnosis and treatment initiation may belower in individuals receiving chronic corticosteroid ther-apy. Excluding these two sub groups of individuals didimprove overall specificity consistent with this supposition.Prescription data was particularly helpful in identifyingosteoporosis cases more than 1 year after the index BMDtest. Without prescription information, sensitivity withalgorithm 2 (1+H or 1+P in 1 year) after the first yearfell to 29.1–30.8%, while it was 75.1–76.9% with algorithm4 (1+H or 1+P or 1+Rx in 1 year).

The algorithms evaluated in this analysis showed gooddiscrimination between osteoporosis cases and normal non-cases, but did not discriminate well between osteoporosiscases and osteopenia non-cases. This is not surprising sincethere is no ICD-9-CM diagnosis code for osteopenia.Furthermore, the distinction between osteopenia and oste-oporosis based upon T-score is quite arbitrary and individ-uals with a T-score of −2.4 and a T-score of −2.6 arevirtually identical in terms of fracture risk and managementwhereas the latter would not be considered osteoporosis bythe WHO classification system. Categorization of thesenon-cases as osteoporosis cases may therefore not beinappropriate since therapeutic implications would actuallybe quite similar. Many guidelines recommend osteoporosis

0%

20%

40%

60%

80%

100%

Year 1 Year 2 Year 3

Year 1 Year 2 Year 3

Year 1 Year 2 Year 3

Sen

sitiv

ity (

oste

opor

osis

spi

ne o

r hi

p)

0%

20%

40%

60%

80%

100%

Spe

cific

ity (

excl

udin

g os

teop

enic

non

-cas

es)

0%

20%

40%

60%

80%

100%

Spe

cific

ity (

all n

on-c

ases

)

a

b

c

Fig. 1 Performance of algorithms 1 (1+H or 1+P) (gray) and 4 (1+H or1+P or 1+Rx) (black) in the first, second and third years after the indexBMD test. a Sensitivity (for osteoporosis cases), b specificity (for allnon-cases), c specificity (after excluding osteopenic non-cases)

44 Osteoporos Int (2011) 22:37–46

medication initiation in individuals with BMD in theosteopenia range, especially when there are coexistentclinical risk factors. Alternatively, osteopenia cases withT-scores close to the normal range are at relatively lowfracture risk in the absence of other clinical risk factors andtreatment would not be recommended. Therefore, ourfinding that a large fraction of osteopenia non-cases havediagnosis codes and osteoporosis medication dispensationsis not unexpected and may actually be desirable as ameasure of the burden of low bone mass identified in thepopulation. This is consistent with the observation thatosteoporosis treatment initiation follows a gradient responserather than discrete step increases across the T-score range.Cranney et al. [42] identified osteoporosis medicationinitiation in the year following BMD testing in 8,689women that had previously been untreated. Treatmentinitiation occurred in 8.2% with normal BMD, 41.0% withosteopenia and 78.5% with osteoporosis. In the osteopeniagroup, treatment initiation rates close to the normal rangewere less than 20% and increased to greater than 60% closeto the osteoporosis range.

Administrative data provide a rich source of epidemio-logical data to investigate health care use and costs, healthoutcomes, and disease risk factors [10]. In general, a moresensitive algorithm can be obtained by including osteopo-rosis drug treatment information, without loss of specificity.Some limitations of this study should be noted. Onlyadministrative data coded in ICD-9-CM were used in thestudy; in the 2004/05 fiscal year Manitoba’s hospitalrecords switched to ICD-10-CA coding. This change mightaffect the number of cases and non-cases identified withfracture and osteoporosis diagnoses; further research isneeded to investigate the impact that this change in codinghas on the validity of an algorithm based only onosteoporosis and fracture diagnoses. The classificationalgorithms were validated using the WHO’s criterion forosteoporosis. This criterion was initially developed forapplication to BMD measurements in postmenopausalwhite females, although there is now consensus that it canbe applied to other North American groups regardless ofrace or ethnicity [43]. Finally, there are no establishedguidelines for determining an acceptable level of sensitivity,specificity, and positive and negative predictive values ofcase ascertainment algorithms defined from administrativedata, making it difficult to arrive at a conclusive statementabout the validity of administrative data for ascertainingosteoporosis cases. More complex modeling methods andvariable sets have been evaluated, and may enhancecategorization even further. Lix et al. [44] applied artificialneural networks, classification and regression trees, andlogistic regression modeling to the same data set andconsidered additional parameters including fracture diagno-ses, comorbidity measures, age, region of residence and

income quintile. Overall discriminate performance basedupon the c-statistic ranged from 0.69 to 0.81. Thecomplexity of these classification algorithms and require-ments for additional data elements makes them less suitablefor implementation in other datasets, however.

In conclusion, a simple case definition for osteoporosiscase diagnosis is feasible based upon administrative healthdata with an acceptable level of sensitivity, specificity, andaccuracy. Performance is enhanced when the case definitionincludes osteoporosis medication use in the formulation. Useof one or more of these algorithms may facilitate implemen-tation of a population-based osteoporosis surveillance pro-gram, providing information that could help to inform andguide screening, prevention, and treatment resources.

Acknowledgments This research was supported by grants from theCanadian Institutes of Health Research, the Surveillance Division ofthe Centre for Chronic Disease Prevention and Control at the PublicHealth Agency of Canada, and by a Canadian Institutes of HealthResearch New Investigator Award. The authors are indebted toManitoba Health & Healthy Living for the provision of data (HIPCFile No. 2005/2006-32). The results and conclusions are those of theauthors, and no official endorsement by Manitoba Health & HealthLiving is intended or should be inferred. This article has beenreviewed and approved by the members of the Manitoba BoneDensity Program Committee.

Conflicts of interest William D. Leslie: honoraria for lectures: MerckFrosst Canada; research support from: Merck Frosst Canada; unrestrictededucational and research grants: The Alliance for Better Bone Health—Sanofi-Aventis and Procter & Gamble Pharmaceuticals Canada, Inc.,Novartis Pharmaceuticals Canada, Inc., Amgen Pharmaceuticals, Inc.,Genzyme Canada Ltd. Lisa M. Lix: unrestricted research grants: AmgenPharmaceuticals, Inc. Marina Yogendran: None.

References

1. Johnell O, Kanis JA, Oden A et al (2004) Mortality afterosteoporotic fractures. Osteoporos Int 15:38–42

2. Adachi JD, Ioannidis G, Berger C et al (2001) The influence ofosteoporotic fractures on health-related quality of life incommunity-dwelling men and women across Canada. OsteoporosInt 12:903–908

3. Lips P, van Schoor NM (2005) Quality of life in patients withosteoporosis. Osteoporos Int 16:447–455

4. Brown JP, Josse RG (2002) 2002 clinical practice guidelines forthe diagnosis and management of osteoporosis in Canada. CMAJ167:S1–S34

5. Papadimitropoulos EA, Coyte PC, Josse RG et al (1997) Currentand projected rates of hip fracture in Canada. CMAJ 157:1357–1363

6. Jaglal SB, McIsaac WJ, Hawker G et al (2000) Patterns of use ofthe bone mineral density test in Ontario, 1992–1998. CMAJ163:1139–1143

7. Lu Y, Genant HK, Shepherd J et al (2001) Classification ofosteoporosis based on bone mineral densities. J Bone Miner Res16:901–910

8. Tenenhouse A, Joseph L, Kreiger N et al (2000) Estimation of theprevalence of low bone density in Canadian women and menusing a population-specific DXA reference standard: the Canadian

Osteoporos Int (2011) 22:37–46 45

Multicentre Osteoporosis Study (CaMos). Osteoporos Int 11:897–904

9. Vestergaard P, Rejnmark L, Mosekilde L (2005) Osteoporosis ismarkedly underdiagnosed: a nationwide study from Denmark.Osteoporos Int 16:134–141

10. Yang NP, Deng CY, Chou YJ et al (2006) Estimated prevalence ofosteoporosis from a Nationwide Health Insurance database inTaiwan. Health Policy 75:329–337

11. Sanders KM, Seeman E, Ugoni AM et al (1999) Age- and gender-specific rate of fractures in Australia: a population-based study.Osteoporos Int 10:240–247

12. Burge RT, King AB, Balda E et al (2003) Methodology forestimating current and future burden of osteoporosis in statepopulations: application to Florida in 2000 through 2025. ValueHealth 6:574–583

13. Goettsch WG, de Jong RB, Kramarz P et al (2007) Developmentsof the incidence of osteoporosis in The Netherlands: a PHARMOstudy. Pharmacoepidemiol Drug Saf 16:166–172

14. Lippuner K, Golder M, Greiner R (2005) Epidemiology and directmedical costs of osteoporotic fractures in men and women inSwitzerland. Osteoporos Int 16(Suppl 2):S8–S17

15. Bernstein CN, Blanchard JF, Rawsthorne P et al (1999)Epidemiology of Crohn's disease and ulcerative colitis in a centralCanadian province: a population-based study. Am J Epidemiol149:916–924

16. Hux JE, Ivis F, Flintoft V et al (2002) Diabetes in Ontario:determination of prevalence and incidence using a validatedadministrative data algorithm. Diabetes Care 25:512–516

17. Svenson LW (1991) Regional disparities in the annual prevalencerates of Parkinson's disease in Canada. Neuroepidemiology10:205–210

18. Blanchard JF, Ludwig S, Wajda A et al (1996) Incidence andprevalence of diabetes in Manitoba, 1986–1991. Diabetes Care19:807–811

19. Fowles JB, Fowler EJ, Craft C (1998) Validation of claims diagnosesand self-reported conditions compared with medical records forselected chronic diseases. J Ambul Care Manage 21:24–34

20. Rector TS, Wickstrom SL, Shah M et al (2004) Specificity andsensitivity of claims-based algorithms for identifying members ofMedicare plus Choice health plans that have chronic medicalconditions. Health Serv Res 39:1839–+

21. Robinson JR, Young TK, Roos LL et al (1997) Estimating theburden of disease. Comparing administrative data and self-reports.Med Care 35:932–947

22. Saydah SH, Geiss LS, Tierney E et al (2004) Review of theperformance of methods to identify diabetes cases among vitalstatistics, administrative, and survey data. Ann Epidemiol 14:507–516

23. WilcheskyM, TamblynRM,HuangA (2004) Validation of diagnosticcodes within medical services claims. J Clin Epidemiol 57:131–141

24. Tu K, Campbell NRC, Chen Z-L et al (2007) Accuracy ofadministrative databases in identifying patients with hypertension.Open Medicine 1:E3–E5

25. Maio V, Yuen E, Rabinowitz C et al (2005) Using pharmacy datato identify those with chronic conditions in Emilia Romagna,Italy. J Health Serv Res Policy 10:232–238

26. Statistics Canada (2008) 2006 Statistics Canada census highlights.Statistics Canada Last accessed: Dec. 18, 2008. URL: http://www12.statcan.ca/census-recensement/2006/dp-pd/index-eng.cfm

27. Roos NP (1999) Establishing a population data-based policy unit.Med Care 37:JS15–JS26

28. Watson DE, Katz A, Reid RJ et al (2004) Family physicianworkloads and access to care in Winnipeg: 1991 to 2001. CMAJ171:339–342

29. Leslie WD, Metge C (2003) Establishing a regional bone densityprogram: lessons from the Manitoba experience. J Clin Densitom6:275–282

30. Leslie WD, Caetano PA, MacWilliam LR et al (2005) Construc-tion and validation of a population-based bone densitometrydatabase. J Clin Densitom 8:25–30

31. Hansen KE, Binkley N, Christian R et al (2005) Interobserverreproducibility of criteria for vertebral body exclusion. J BoneMiner Res 20:501–508

32. Kanis JA, Melton LJ III, Christiansen C et al (1994) The diagnosisof osteoporosis. J Bone Miner Res 9:1137–1141

33. Leslie WD, Adler RA, El-Hajj FG et al (2006) Application of the1994 WHO classification to populations other than postmeno-pausal Caucasian women: the 2005 ISCD official positions. J ClinDensitom 9:22–30

34. Lewiecki EM, Watts NB, McClung MR et al (2004) Officialpositions of the international society for clinical densitometry. JClin Endocrinol Metab 89:3651–3655

35. Siminoski K, Leslie WD, Frame H et al (2007) Recommendationsfor bone mineral density reporting in Canada: a shift to absolutefracture risk assessment. J Clin Densitom 10:120–123

36. Binkley N, Kiebzak GM, Lewiecki EM et al (2005) Recalculationof the NHANES database SD improves T-score agreement andreduces osteoporosis prevalence. J Bone Miner Res 20:195–201

37. Leslie WD, Caetano PA, Roe EB (2005) The impact of hipsubregion reference data on osteoporosis diagnosis. OsteoporosInt 16:1669–1674

38. Leslie WD (2006) The importance of spectrum bias on bonedensity monitoring in clinical practice. Bone 39:361–368

39. Brown JP, Josse RG (2002) 2002 clinical practice guidelines forthe diagnosis and management of osteoporosis in Canada. CMAJ167:S1–S34

40. Sackett DHR, Haynes RB, Guyatt G, Tugwell P (eds) (1991)Clinical epidemiology: a basic science for clinical medicine.Little, Brown and Company, Boston

41. Leslie WD, Siminoski K, Brown JP (2007) Comparative effects ofdensitometric and absolute fracture risk classification systems onprojected intervention rates in postmenopausal women. J ClinDensitom 10:124–131

42. Cranney A, Tsang JF, Leslie WD (2008) Factors predictingosteoporosis treatment initiation in a regionally based cohort.Osteoporos Int 20:1621–1625

43. Kanis JA, McCloskey EV, Johansson H et al (2008) A referencestandard for the description of osteoporosis. Bone 42:467–475

44. Lix LM, Yogendran MS, Leslie WD et al (2008) Using multiple datafeatures improved the validity of osteoporosis case ascertainmentfrom administrative databases. J Clin Epidemiol 61:1250–1260

46 Osteoporos Int (2011) 22:37–46

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.