Doctors vs Data

There is a storm brewing in Healthcare. Doctors have been in charge of healthcare for a long time, and have become comfortable, sometimes even arrogant, with their authority and power. But dumb data beats smart doctors every time. Forward thinking doctors are embracing data, with surprising grace and humility. Others are having much more trouble adjusting.

Doctors, historically, have been the “end of the discussion” on clinical matters. Doctors make the diagnosis, they make the calls in the surgery suite, they get to decide if someone is suffering enough to justify pain medications, they frequently decide whether someone is mentally incompetent or merely eccentric. Our society places a lot of trust in doctors, because they have the training needed to make really hard choices.

Doctors, as a group, have been in charge of how healthcare operates for centuries. In times past, the only way to determine whether a doctor was doing a good job was to become a doctor yourself, and then perform case reviews. Even in court, if you wanted to refute a doctor, you needed another doctor.

But that time has changed. Now we have data on doctors. We have data on how they work together, on how they prescribe. We can tell whether they wash their hands well enough, and whether they use the right procedures in the right situation. In many ways the culmination of the EHR adoption is really the dawn of doctor science. We can now tell, with a fair amount of objectivity, how good doctors are at their jobs.

But doctors are very resistant to being evaluated in this way, as shown by recent polls done by  Many doctors find the notion of subjective patient ratings on various online websites distasteful.  This is interesting, since there is some evidence that online rating scores are at least aligned with more traditional, and expensive, patient surveys. Another poll from the same organization showed that doctors were split on whether CMS should release more medical claims data. Apparently, a sizeable contingent of doctors are opposed to being rated and evaluated, no matter whether the data is subjective or objective.

There is a valid reason for doctors to have a dim view on both patient ratings and claims data. Both data sources are biased in different ways.

Claims data are particularly problematic. Some doctors feel that they should have privacy with regard to how much they are paid by the Federal Government (via Medicare and Medicaid). Which is reasonable, except for one hiccup. Using claims data as a source, it is not possible to describe precisely what a doctor is doing without also describing how much they have been paid with reasonable accuracy. Without describing precisely what a doctor is doing, there is no way to even begin determining whether a doctor is doing a good job. It may still be impossible to tell whether a doctor is doing a good job, but at least the data science community would have some place to start. You know.. that whole necessary but not necessarily sufficient line.

Claims data is severely limited–it is not, by any stretch, good data. Doctors know this and are reluctant to endorse it as a data set.

But rejecting bad data is a valid response only when you are proposing some better data source. Claims data is much better than patient anecdotes in terms of data richness, whether you favor Yelp or HCAHPS.

I am put in the position, almost daily now, to advise people on which doctors to work with based on data that I have. I can happily say that my data set (DocGraph) is likely the best available data for making these kinds of decisions. But “best” does not mean “good”. What I can do with bad data now is impressive to healthcare administrators, because the alternative is trusting in blind luck, which is how healthcare has been coordinated before now. This is a standard that I am comfortable beating.

If you are uncomfortable with the notion that someone like me is advising on the hiring and firing of doctors with less than perfect data (I certainly am) then you should consider letting CMS know that they should release as much “better” data as possible. They have a request for public comment that will be open for a few more days. Now is your chance.

As we shift from using no data, to bad data, to better data, and ultimately to good data, we need to be fair to doctors regarding where we are. But we already have enough data to know when a doctor performs badly. Given what we already know, simply resisting evaluation is not an attitude that is going to pay off for doctors. This is not the kind of movement that you can opt out of. Medical science used to be so complex that it was impossible to evaluate, that is changing. Medicine has not gotten any simpler, but our tools for managing complexity have improved manifold. Medical technology has accelerated, but medical culture has yet to catch up. The scrutiny phobic medical culture is changing. Slowly, painfully, but it is changing.

Strata Rx Heath Data Conference
— Strata Rx brings together the diverse communities driving innovations in big data analytics for health care. Learn about the transformation of health care through big data and how to position your company to benefit from these trends. Learn more.
tags: , , , ,
  • Panha Chheng

    great commentary and activism fred. keep up the great work, and fight the good fight. your docgraph is the first of many straws on the proverbial camel’s back. – panha

    • Lora Kratchounova



    Doctors have been in charge of healthcare for a long time, and have
    become comfortable, sometimes even arrogant, with their authority and

  • disqus_zFY8P50XHF

    Fred Trotter, you could not be more wrong about claims. I agree that big data – in the hands of doctors and patients and instead of just insurance companies – will improve the situation. However, doctors have been essentially out of the loop on claims for decades, with the exception of those few doctors who work in the medical insurance industry. How much a doctor can charge for a procedure is completely dictated by the insurance and MHO companies, not by the doctor. What procedures a doctor can order and what drugs they can prescribe are also, largely, determined by the insurance companies and the government.

    Medicine has basically been taken over by accountants and clerks (the insurance companies) and by lobbyists and politicians (the lawyers). Doctors have been disempowered, in essentially all ways, to the point where top students no longer go into medicine in this country, and haven’t for a long time. And now there’s a dearth in the USA, being filled by physicians and health-care professionals from other countries.

    This is the real crisis in skilled labor we have in the USA – as opposed to the faked-up STEM crisis. (there is actually a surplus of STEM professionals, which is one of the reasons so many are forming startups, an/or jumping on the “big data” bandwagon).

    I applaud your use of statistics and and real data to improve peoples’ ability to make sound medical choices. Now, I suggest that you turn your DocGraph tools into HMO-Graph and Insuror-Graph, and datamine up some real causality.

    To summarize – doctors do not choose procedures, prescriptions, or prices anymore and haven’t for a long time. Even if some doctors you know say otherwise, most don’t. At best, they pick from a menu that the insurers, HMO managers, and the FDA create, complete with prices.

    • georgemargelis

      I agree, the claim that dumb data beats smart doctors is in itself very dumb. Whilst doctor bashing seems to be in vogue in the IT world their failure to provide any credible evidence for their claims is deplorable.

    • Lora Kratchounova

      disqus_zFY8P50XHF , Fred is not wrong – docs have not been out of the loop on claims, well, at least not most of them. Docs know how much they can charge for procedures, hence why we have such variance – because docs, and their admin, have been maximizing the payouts – this is what coders do for them. Doctors haven ot been disempowered, they have not adapted to the new transparent world order. But I agree, wish we had an HMO-Graph… although we do – look no further than the rankings of insurance companies. The point is – docs need to embrace data and earn their reputation, like the rest of us

  • Medicalquack

    Good show Fred on the flawed data as you are correct. Data selling epidemic adds to the problem as well. You and I wan good clean clinical data which is what we should have but we don’t get it all the time or someone takes data that has value and queries it with junk or what I call non credible data and bingo! Truckloads of new analytisc software appear on the horizon on a guess with really trying to stroke out a non linear relationship with that “big data” out there:)

    I’m with you on the government as they can’t or won’t model and that’s the problem we have with HHS, the SEC, FTC and others trying to use lawyers to govern agencies where hi tech prevails and it runs everything today, decision making. Then you add in some useless politicians to cloud it even more and then what you read in the news is a mess.

    Reality is that government (maybe except the NSA of course) is no match for the hundreds of quants employed by insurers and other folks in the money side as they don’t get it. I made this post about a PLOS One study and sure there’s some satire in here but I’m really beginning to think there’s something to this study that the doctors did with MRI scans of the brain and more to come to the conclusion that the fear of math gives people real physical pain.

    You know that pain too as I think all developers have it when they see a huge complex code project to mess to work with…lo ( we could call it a headache:)l…but the baby step fear at the consumer level and politicians too I think is really prevalent out there.