Calculation of Standardised Hospital Mortality Ratios in England and the USA
(Text of lecture delivered by Professor Sir Brian Jarman to the Bristol Medico Chirurgical Society February 2003)
US Medicare data have been analysed to calculate hospital standardised mortality ratios (HSMRs). The analyses are based on work done initially in the UK. In the UK study, which was published in the BMJ on 5 June 1999 (which gives more details), we looked at four years of UK Hospital Episode Statistics (HES) data for England, from 1991-92 to 1994-95. For the more recent studies we used HES data from 1995-6 to 2001-2002. Mortality rates for the non-specialist large hospital Trusts were calculated (a Trust is a group of hospitals in a town or city being a main general hospital usually with a few nearby associated hospitals). We excluded community and specialty institutions and small hospitals (under 15,000 admissions during the preceding five years) leaving 167 hospital Trusts that were in place on 1 April 2001 (about 90% of hospital activity in England for those years). We excluded the data for a hospital for any year for which that hospital had poor quality data (usually because data were missing, or we were informed of poor data quality, or other quality checks indicated poor quality: a total of 3% of the data was excluded). Cases were included in the analysis if the primary diagnosis on admission was one of 80 primary ICD diagnoses that accounted for 80% of all hospital deaths in England.
Hospital standardised mortality ratios (HSMRs) were calculated as the ratio of actual number of deaths to the expected number of deaths multiplied by 100. We calculated death rates for the 5 years 1995-6 to 2001-2002 for all Hospitals in England stratified by age (using the HES age groups), sex, admission type, and length of stay (0-7, 8-14,15-28 and 29-365 days) for each of the 80 primary diagnoses on admission leading to 80% of all deaths (the % deaths covered by these 80 diagnoses varies slightly each year). These were used to calculate the expected deaths for each hospital by multiplying the number of hospital admissions in each stratum or cell by the England death rates for each stratum and adding across all strata to find the sum of the deaths that would be expected if a particular hospital had the national death rate for each stratum. For the BMJ paper we standardised only for age, sex and diagnosis but regression analyses showed that the method of admission (emergency/elective) was a powerful predictor of HSMRs (emergencies make up 60% of admissions but 93% of deaths). Also death rates will vary with length of hospital stay and some hospitals may discharge patients earlier than others eg to hostels etc (see the BMJ paper for discussion of this). Hence to allow for this later data were also standardised admission method and length of stay. Hospitals vary from twelve 95% confidence intervals below the England mean to six 95% CIs above. The effect of standardisation is to reduce the range of death rates (ratio of highest to lowest) between the Hospitals in England from 4.0 for crude mortality rates to 1.8 for standardised mortality ratios.
To allow for social factors we determined the electoral ward of the place of residence of all patients admitted to each hospital and calculated the average value of the electoral ward social factors (using three standard deprivation scores and each of their individual components and also the level limiting long-standing illness) for all admissions for each hospital. These social factors were not found to be significant explanatory variables once the hospital data had been standardised within each of the 80 diagnoses. These diagnoses are related to social factors - eg a hospital in a deprived area with above average rates of smoking, alcoholism and obesity will have a pattern of diagnoses such as stroke, myocardial infarction, carcinoma of the liver which reflect these social conditions. Many of the hospitals with the lowest HSMRs are in deprived inner city areas: for example, the Health Authority areas of the 10 Hospitals with the lowest HSMRs had average deprivation scores and unemployment levels which were markedly greater than the values for the 10 Hospitals with the highest HSMRs.
Several studies stress the importance of adjusting for severity of illness in hospital admissions when comparing quality of health care. We calculated several measures of comorbidity based on discharge diagnoses for each hospital including the number of bodily systems affected by disease, the percentage of patient admissions with one of the 15 most serious primary diagnoses (responsible for 50% of all deaths eg malignancy), and the percentage both of cases and of deaths with co-morbidities (that is, sub-diagnoses) in each of the 80 diagnoses that led to 80% of all deaths. We ranked sub-diagnoses by their univariable correlation with the hospital standardised mortality ratios and created a measure of comorbidity by combining the top two or three comorbidity diagnoses. We used each of these measures in our model as independent estimates of the severity of illness treated. However, we had already standardised within each of the diagnoses and when regression analyses were carried out using a wide range of possible explanatory variables we found that additional measures of casemix and co-morbidities were not among the most powerful explanatory variables (see BMJ paper for details).
We found that the number of hospital doctors per bed (total doctors per total beds) was the most powerful explanatory variable in explaining the variation of HSMRs for the large hospital Trusts in England for both the initial 4 years and later 6 years datasets (more doctors being associated with lower death rates). It is interesting to note that an OECD study of the variation of a number of measures of mortality across 21 OECD countries (Exploring the effect of health care on mortality across OECD countries. OECD 26-Sep-2000) found "that the doctors per capita are either the most important or the second most important determinant of mortality in all but one of the equations reported".
The US Agency for Healthcare Research and Quality (AHRQ) and Medicare analyses
AHRQ all-payer data were first analysed in a manner similar to that used in England but using in addition race and payer as standardisation variables. It was found that the all-payer AHRQ HSMRs correlated with the AHRQ Medicare-only HSMRs with a correlation coefficient of 0.96. Medicare patients account for 70% of all deaths in the AHRQ data, which is a representative 20% sample of US hospitals from 20+ states. The Medicare data set is a 100% sample of Medicare data: it is more up to date than AHRQ data and is in the public domain. For these reasons most of the analyses were carried out using Medicare data. For the Medicare analyses the variation of death rates with agegroup, sex, race, admission type, admission source, and length of stay were determined within each of the diagnostic groups which lead to 80% of all deaths in one year. For the US data the Clinical Classification System (a group of related ICD9 diagnoses) developed by AHRQ is used rather than ICD diagnoses and only 30 CCS groups are needed to cover the diagnoses leading to 80% of all deaths. Standardisation is done by each of the age, sex etc categories within each CCS diagnostic group. The standardised values of hospital mortality ratios (HSMRs) are then used in regression analyses as the dependent variable in order to find which of a number of independent variables best explain the variation of HSMRs. The independent variables used are social factors, Dartmouth Atlas variables for 306 Hospital Referral Regions or, where available, the 3436 Hospital Service Areas (HSAs) - data supplied by John E Wennberg and Elliott Fisher from Dartmouth variables. These many relevant factors that are linked using the HRR or HAS code of each hospital determined from the American Hospital Association (AHA) hospital database for the year 2000.
The most powerful factors that are found to be significant in explaining the variation of HSMRs are:
HSMRs are calculated by standardisation and the effect of allowing for the significant factors in the regression analysis are then calculated to give a regression adjusted HSMR (HSMR adjusted by standardisation and regression). Data are also calculated for the 3 years 1999-2001 and for these data there is a slightly different regression model.
Directly standardised charges, costs and reimbursements were also calculated for each hospital, adjusting for age and diagnosis within the diagnoses leading to 90% of all charges. The relationship (using Medicare data for 2000) between directly standardised reimbursements and regression adjusted HSMRs showed no significant correlation between them.
Prof. Sir Brian Jarman
February 12, 2003