Researchers are calling for a "time out" in the use of performance measures for physicians after finding that less than 40 percent of these metrics are valid.

The study, led by Catherine H. MacLean, MD, PhD, a specialist in health care quality and Chief Value Medical Officer for Hospital for Special Surgery in New York City, found that only 37 percent of quality measures the group assessed met a list of criteria for validity devised by the American College of Physicians. Of the rest, 35 percent were deemed invalid while for the remaining 28 percent, the validity was found to be uncertain.  

"The fact that only 37 percent of measures proposed for a national value-based purchasing program were found to be valid using a standardized method has implications for physician-level performance measurement," the authors wrote. "The use of flawed measures is not only frustrating to physicians but potentially harmful to patients. Moreover, such activities introduce inefficiencies and administrative costs into a health system widely regarded as too expensive already."

"What we should be paying for in a value-based system is higher-quality care," Dr. MacLean said. "It’s critically important that we have the right measures. A bad measure is a missed opportunity to inform clinical care."

The researchers reported the findings this week in The New England Journal of Medicine.

Physicians in the United States are now tracked on more than 2,500 performance measures, an explosion of which has occurred during the past 30 years as the Centers for Medicare & Medicaid Services (CMS; the government’s health insurance agency) and private payers have looked for ways to both improve the delivery of health care and drive down costs. Doctors have balked at the increasing burden, which cost roughly $15.4 billion, or about $40,000 per physician, annually to meet.

However, whether those measures – which come from dozens of organizations – are meaningful and robust remains unclear. The new study will reinforce the view among many physicians that the metrics are time and money misspent.

To evaluate the validity of performance measures, a committee for the American College of Physicians developed a five-item checklist: importance, appropriateness, strength of clinical evidence, feasibility of implementation and applicability.

The committee looked at 86 performance measures germane to general internal medicine. The measures were part of the Medicare’s Merit-based Incentive Payment System/Quality Payment Program, a sweeping initiative to link physician performance – and patient outcomes – to reimbursement under the program. CMS has declared that it wants to tie 90 percent of physician payments under Medicare’s fee-for-service system to performance metrics by the end of this year. However, the agency recently issued a request for proposals for new measures.

"We hypothesized that if most of the measures assessed were deemed valid using this process, physicians could have more confidence that adherence to the measures would result in improved patient outcomes," Dr. MacLean and her colleagues wrote.  

Thirty (35 percent) of the measures failed the test. Of those, 19 lacked sufficient clinical evidence to warrant implementation. For example, one measure calls for physicians to screen older patients for signs of elder abuse. But Dr. MacLean, who moderated the panel, and her colleagues note that although elder abuse is a concern, the U.S. Preventive Services Task Force does not recommend routine screening for the problem.

Similarly, the study found that another recommendation, for managing people with high blood pressure using a strict threshold (140/90 mmHg) also failed the validity test because it might do more harm than good in older, sicker patients and those with certain health conditions.

The study also revealed "troubling inconsistencies" in which measures major organizations regard as valid. The authors suggest this may be due to different methods used by different organizations and call for a standard method to grade the validity of these measures.  

"Quality measures only should be based on practices about which we are certain there is a meaningful health benefit. There should be no controversy in quality measures," Dr. MacLean said.

Dr. MacLean and her colleagues do not dismiss the need for performance measures for physicians. But they call for changes to the way the assessments are created. One step, they said, is to avoid an overreliance on administrative data – such as billing claims – which, while easy to obtain, are not particularly informative or nuanced. Another is to try to move away from a system in which physicians are rated long after they deliver care to one in which they receive feedback on their performance in real time.

"The point of the paper isn’t that we don’t support quality measures. We’ve had careers in developing them," Dr. MacLean said. "The problem is that bad quality measures can be harmful. Additionally, they are a waste of time; they’re frustrating, and they’re a waste of money. It’s gotten to the point where it’s almost measures for measurement’s sake."