Call for a new model to evaluate medical schools

"No previous studies have challenged the measurement strategy for evaluation of research-intensive medical schools, yet there is a critical need for better evaluation metrics, especially as the amount of funding for academic research has significantly decreased," explained Mitchell R. Lunn, MD, who conducted this research while he was a physician in the Department of Medicine at BWH. "USN&WR relies heavily on subjective and pre-medical student performance measures, such as Medical College Admission scores and undergraduate grade point average. Despite the fact that the USN&WR evaluation method has undergone changes, it remains subjective and limited," added Lunn, who is currently a clinical research fellow at the University of California, San Francisco.

Researchers developed a new model to evaluate medical school performance based on two fundamental principles: that objective, relevant and accessible criteria should be the basis for comparison, and that metrics should be based on outcomes that reflect the general mission, vision, and values of the nation's medical education enterprise.

Researchers collected data from more than 600,000 physicians and 127 medical schools though Doximity, the U.S. Department of Health and human Services, the American Medical Association's Physicians Masterfile, state medical boards, specialty boards, many award databases and others. In the model's scoring system, each physician was given a score that incorporated data from four categories: number of publications, grants, clinical trials and awards/honors. Clinical performance measures were intentionally omitted in this model, as there are no clear metrics for a physician's clinical quality publically available.

While many top institutions ranked highly in both this new model and the 2014 USN&WR rankings, many important differences exist. For example, graduates from the Albert Einstein College of Medicine of Yeshiva University excelled at obtaining awards and grants, which resulted in a rank of 13 in this new analysis, compared to a rank of 34 in USN&WR.

"Our model demonstrates the feasibility of an outcomes-based approach to evaluating medical schools' ability to produce academic physicians who go on to successful biomedical research careers," said Matthew J. Goldstein, MD, PhD, a resident physician in the Department of Medicine at BWH and lead author of the study. "It is important to note, however, that institutions without this priority, such as those that focus on creating primary care physicians, would likely be disadvantaged from our model."

"For a field like medicine, which is guided by the principles of evidence-based practice, we know very little about what educational processes produce the best physicians," said Lunn. "By fostering a national discussion about the most meaningful criteria we should be measuring and reporting, we hope to improve the quality of medical education across the nation."