Featured Research

from universities, journals, and other organizations

Mortality rates are an unreliable metric for assessing hospital quality, study finds

Date:
December 30, 2010
Source:
Harvard Medical School
Summary:
A comparative analysis found wide disparities in the results of four common measures of hospital-wide mortality rates, with competing methods yielding both higher- and lower-than-expected rates for the same Massachusetts hospitals during the same year.

Is quality in the eye of the beholder? Researchers at Harvard Medical School and Massachusetts General Hospital have found wide disparities among four common measures of hospital-wide mortality rates, with competing methods yielding both higher- and lower-than-expected rates for the same Massachusetts hospitals during the same year.

The findings, published Dec. 23 in a special article in the New England Journal of Medicine, stoke a simmering debate over the value of hospital-wide mortality rates as a yardstick for health care quality. The measure, which compares a hospital's actual patient death rate to statistical predictions, is reported publicly in countries including England, Canada and Denmark, but some hospitals and policy experts have questioned its value due to the complexity and variability of diagnoses.

"It's troubling that four different methods for calculating hospital mortality rates as a measure of quality should yield such different results," said lead author David M. Shahian, HMS professor of surgery at Massachusetts General Hospital. "Measurement theory -- not to mention plain common sense -- suggests there is a problem."

The potential of performance evaluation to improve both the quality and the cost of health care has fueled interest in provider "report cards," including mandates by state and federal law.

In 2008, the Massachusetts Division of Health Care Finance and Policy engaged researchers Shahian; Lisa I. Iezzoni, HMS professor of medicine at Mass General; and Sharon-Lise T. Normand, HMS professor of health care policy (biostatistics) and professor in the Department of Biostatistics at Harvard School of Public Health, to evaluate four vendor-created measures of hospital-wide mortality. The state was looking for a means to measure hospital quality for the public report cards mandated under its 2006 health care reform law.

The researchers compared four measures of hospital-wide mortality provided by commercial vendors. These vendors all believed that their hospital-wide mortality measures were an accurate reflection of hospital quality. Each vendor received identical data -- three years of patient discharge data from all 83 general acute care hospitals in Massachusetts, representing 2.5 million discharges -- and used the data to calculate each hospital's mortality rate. The researchers then compared results.

"The results of the horse race are that it's really not clear who won," said Iezzoni, who also is the director of the Mongan Institute for Health Policy at Mass General. "The problem is that we were measuring the success of each of these measures against a gold standard we simply do not have: an objective measure of hospital quality."

Without that gold standard, the researchers were left to compare vendor tools with one another. Their conclusion: Methods and results varied widely. For example, every tool excluded some discharges from its calculations based on the details of each. But where one tool excluded 5 percent of all discharges, another excluded 72 percent.

Even so, a high degree of convergence -- different methods yielding similar final results -- could have supported the validity of this approach to estimating hospital quality. But that's not what researchers found. For example: Of 28 hospitals designated by one method as having higher-than-expected hospital-wide mortality in 2006, 12 were simultaneously classified as having lower-than-expected mortality by at least one other method.

In August, the researchers told the state that they could not recommend any of the four vendor-created tools. "But the results should not be interpreted as an indictment of any particular technique," said Normand. "Rather, they call into question the concept of measurement of hospital-wide mortality, as four different methods yielded four different results. Thus, this may not be a good way to assess hospital quality."

The authors are longtime advocates of performance assessment and public reporting to provide transparency and accountability in health care.

Normand developed the statistical models used by Medicare and Medicaid for public reporting of heart attack, heart failure and pneumonia mortality rates. Shahian has been a leader in the public reporting initiative of the Society of Thoracic Surgeons. Together, they were leaders in developing and implementing public report cards for coronary artery bypass grafting (CABG) surgery and percutaneous coronary intervention (PCI) in Massachusetts.

But where mortality rates may reflect clearly the quality of care for some procedures, like coronary bypass, they may reveal less about care for other conditions, such as major trauma or advanced malignancy.

"An alternative to hospital-wide mortality rates would be to construct a more limited portfolio of mortality results for individual common conditions such as heart attack, stroke, pneumonia, CABG surgery and PCI," said Shahian. "These have the advantage of large sample sizes at most hospitals, a generally accepted association between mortality and quality of care, and credible risk models to adjust for patient severity."

Iezzoni, an internationally recognized expert in risk adjustment, has edited Risk Adjustment for Measuring Health Care Outcomes, now in its third edition.

"Underlying this finding is the more fundamental and as yet unanswered question about whether hospital-wide mortality rates provide meaningful insight into hospital quality," she said.

This work was funded by a contract with the Division of Health Care Finance & Policy, Executive Office of Health and Human Services (EOHHS), Commonwealth of Massachusetts, Boston, Mass.


Story Source:

The above story is based on materials provided by Harvard Medical School. The original article was written by R. Alan Leo. Note: Materials may be edited for content and length.


Journal Reference:

  1. david M. Shahian, Robert E. Wolf, Lisa I. Iezzoni, Leslie Kirle, and Sharon-Lise T. Normand. Variability in the Measurement of Hospital-wide Mortality Rates. N Engl J Med, 2010; 363:2530-2539; December 23, 2010 DOI: 10.1056/NEJMsa1006396

Cite This Page:

Harvard Medical School. "Mortality rates are an unreliable metric for assessing hospital quality, study finds." ScienceDaily. ScienceDaily, 30 December 2010. <www.sciencedaily.com/releases/2010/12/101222173035.htm>.
Harvard Medical School. (2010, December 30). Mortality rates are an unreliable metric for assessing hospital quality, study finds. ScienceDaily. Retrieved September 23, 2014 from www.sciencedaily.com/releases/2010/12/101222173035.htm
Harvard Medical School. "Mortality rates are an unreliable metric for assessing hospital quality, study finds." ScienceDaily. www.sciencedaily.com/releases/2010/12/101222173035.htm (accessed September 23, 2014).

Share This



More Health & Medicine News

Tuesday, September 23, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Ebola Costs Keep Mounting

Ebola Costs Keep Mounting

Reuters - Business Video Online (Sep. 23, 2014) The WHO has warned up to 20,000 people could be infected with Ebola over the next few weeks. As Sonia Legg reports, the implications for the West African countries suffering from the disease are huge. Video provided by Reuters
Powered by NewsLook.com
Ebola Cases Could Reach 1.4 Million Within 4 Months

Ebola Cases Could Reach 1.4 Million Within 4 Months

Newsy (Sep. 23, 2014) Health officials warn that without further intervention, the number of Ebola cases in Liberia and Sierra Leone could reach 1.4 million by January. Video provided by Newsy
Powered by NewsLook.com
WHO: Ebola Cases to Triple in Weeks Without Drastic Action

WHO: Ebola Cases to Triple in Weeks Without Drastic Action

AFP (Sep. 23, 2014) The number of Ebola infections will triple to 20,000 by November, soaring by thousands every week if efforts to stop the outbreak are not stepped up radically, the WHO warned in a study on Tuesday. Duration: 01:01 Video provided by AFP
Powered by NewsLook.com
5 Ways Men Can Prevent Most Heart Attacks

5 Ways Men Can Prevent Most Heart Attacks

Newsy (Sep. 23, 2014) No surprise here: A recent study says men can reduce their risk of heart attack by maintaining a healthy lifestyle, which includes daily exercise. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Health & Medicine

Mind & Brain

Living & Well

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins