Are your data lost in translation?

This question may seem a bit odd because health care executives don’t tend to think about their data after submitting numbers to third-parties like CMS, professional associations, researchers, and industry analysts.  We assume that outside users of our data will utilize exactly the information we provided when they conduct their studies and prepare their secondary reports. 

Not necessarily!  As a data geek of long standing, I have helped students and clients uncover errors—many with negative consequences—in third parties’ displays and transformations of numbers submitted to them.  For example, facility-specific mortality rates officially reported by HCFA (now CMS) in the 1990s often varied substantially from death counts submitted by the hospitals.  More recently, providers find that their cost and services data are not always the numbers that get published on consumer-oriented Web sites.  An east-coast hospital CEO told me just last week about a significant discrepancy between the actual number of physicians on the medical staff and the count reported in an industry data base.  The publicly available report even showed no physicians practicing in a clinical area where the hospital had four full-time boarded specialists.      

We understand the “garbage in-garbage out” phenomenon, but how do good data get turned into garbage?  (I am giving providers the benefit of the doubt here.  Their numbers can be inaccurate, too, creating a serious problem that must be solved as they start using analytics and other performance improvement tools to achieve efficiency and effectiveness.)  Many errors are introduced as information is “keyboarded” to a third-party’s data base from provider-prepared forms—a problem that should diminish as more data are transferred electronically through all-digital data exchanges.  External reports can also fail to match the time period when the data were collected with the date of the report, giving the impression that historical information is current.  Discrepancies of two to three years are common.  Finally, some of the most damaging errors are created when third-parties transform data with general models or statistical techniques that do not fit the specific circumstances of an organization providing the information.

Growing pressure for transparency and accountability will increase the potential for adverse consequences resulting from data distortion by Web sites and other external entities.  Consequently, health care executives are advised to double-check their numbers, comparing the data they submitted with the numbers published in external reports prepared by third-parties.  In my experience, errors in data translation can be costly in many ways.  What are your experiences with outsiders’ use of your numbers?