In a world where you can access nearly all human knowledge from a device in your pocket, it’s absurd that you still can’t find out how much having a baby will cost you. But it’s not surprising.

“Health data” is such a hot topic these days that you’d think it was a recent invention, and in many cases you’d be right. Apple’s HealthKit and other trackers passively gather exabytes of our health data as we move around. Saliva samples let us explore genetic conditions. A handful of companies can discover the bugs living inside our guts.

(AP Photo/Don Petersen, File)

Looking at these new forms of data is exciting, but decades worth of existing health data could provide value—if only it weren’t such a disappointing mess. A prime example is health insurance claims, which tell a rich story of diagnosis patterns, doctors’ practices and billing negotiations—in short, a goldmine that could help millions of Americans make more informed healthcare decisions about where to get care and how much they should expect to pay. Yet these claims remained locked within a black box, accessible to doctors and payers but not us, the patients.

Digital health innovators have overlooked health insurance claims. Why? It’s largely due to how claims are created and processed, and the lack of incentives for changing the status quo. Examining these challenges can help us identify what’s blocking our path—and how we can change course.

A brief history

When we visit a doctor or pick up a prescription, we leave behind a digital paper trail. Twenty years before electronic health records became the norm, healthcare events were digitized through insurance claims and medical bills because it meant getting paid faster. Doctors’ notes were translated by administrative staff into coded electronic messages that were sent to insurance companies for payment millions of times per day.

For years these logs were considered to be the useless exhaust of the engine connecting doctors and insurance companies. They were optimized for doctors to get faster and higher payment, and for insurance companies to avoid overpaying. As the electronic exchange of insurance claims and electronic health data grew during the 1990s, it became clear that privacy and security rights and requirements needed more definition, leading to the Health Insurance Portability and Accountability Act of 1996 (HIPAA).

Fast forward to today. Having spent two years at Amino analyzing insurance claims representing more than 188 million Americans, my diagnosis is that healthcare claims and medical bills (1) remain too complex for most researchers to generate useful insights without a lot of sophisticated clean-up, and (2) are difficult to collect because of outdated and misaligned incentives, leading to prohibitively high prices to obtain and store the data.

The dirty world of insurance claims

The most immediate challenge is that health insurance claims are notoriously dirty. As you likely know from personal experience, a hospital visit is described by a series of obscure codes on your medical bill. If you undergo a procedure, you might see a different code for each action—the administration of anesthesia, the surgeon’s fee for the procedure, and so on.

Think of it this way: you order a salad at a restaurant and get charged for each leaf of lettuce and the exact number of cucumber slices, tomatoes and spoonfuls of dressing—yet you don’t see the word “salad” anywhere on your receipt. You might not even see which restaurant location billed you for the meal! This is what we’re facing when deciphering health claims data. Finding meaningful events is hard to do without building complex models to group these overly specific codes, and attributing claims to specific doctors or hospitals can be inconsistent without sophisticated data science detective work.

Plus, there are 68,000 diagnosis codes and countless ways for human errors (or fraud) to result in medical billing inaccuracies. A typo can result in a $4,000,000 charge for a colonoscopy, or a diagnosis of Parkinson’s instead of gout. Checking the wrong box can mean a man is now pregnant, or a child suddenly has a history of arthritis. Then there are hospitals and doctors’ offices that upcharge for services or use billing shortcuts, making the picture of healthcare all but transparent.

But when you compile billions of insurance claims with these diagnosis codes, you can start to hear the signal amidst the noise—and meaningful patterns and insights emerge.

Putting the data to use

Right now, there’s no incentive for companies that have health insurance claims data to share it publicly. Some cite HIPAA, arguing that it’s expensive or restrictive. But a strength of HIPAA—not a limitation—is that it clearly outlines how our healthcare data must be made available to us, the patients described in these databases. Unfortunately, most investors and professionals in healthcare don’t understand HIPAA, and they seize up, thinking the safest route is to prevent data from being shared at all.

Information is power in every industry, and when information is unlocked in a new way, it reshapes everyone’s role in the marketplace for the best. We used to rely solely on travel agents to find our airline tickets, or stock brokers to tell us how our portfolios could improve. When it comes to healthcare, data holds clear value for people receiving care (although some big industry playerscertainly disagree). Data can tell you if a specific cancer treatment has associated side effects; it can tell you which co-morbidities to expect if you get diagnosed with a chronic illness; it shows you how frequently an OB/GYN performs C-sections. Yet there’s tremendous fear in a market where information sharing has been made to seem scary, and the leading businesses have grown without serving consumers’ thirst for transparent behavior.

Even more challenging is that the healthcare industry sets the bar extremely high for those of us working on its most difficult problems. The culture of rapid innovation celebrated in tech—releasing new features, assessing their impact, and tweaking them based on performance—runs counter to established systems in healthcare. A tolerance for only unanimously approved measures can have a chilling effect on efforts to develop new technology. Since launching Amino and witnessing how price estimates or quality measures are debated, I’ve found that innovative new products with the promise of continuous improvement are met with too little encouragement.

To create a useful consumer experience built on all of our health data, we must encourage innovation in healthcare data standards, securely open up databases so that we can see how others receive care and make it easier for us to access our own medical history. We must also support a series of steady improvements in new research and products aimed at complete transparency, and not let perfect be the enemy of good.

With healthcare spending reaching almost a fifth of our country’s GDP, the federal government and several states have pushed forward important data liberation efforts. Recent developments to keep an eye on include:

      • Florida and Vermont recently passed bills to make healthcare bills more predictable and transparent, and Texas recently proposed bipartisan legislation to increase healthcare cost transparency.
      • The U.S. Department of Health and Human Services announced a national contest challenging innovators to design a medical bill that people can actually understand.
      • A growing number of states (at least 18 to date) have enacted all-payer claims databases (APCDs), requiring insurance carriers to contribute their insurance claim logs into a single data warehouse for the benefit of researchers and consumers. (This is a best practice already common in other countries.)
      • The Medicare Qualified Entity (QE) program, a high-impact but wildly under-appreciated component of the Affordable Care Act, continues to add new members, giving them unique access to all Medicare claims data in order to profile the quality of care at physicians’ offices nationwide. (My company, Amino, was recently included among them.)

Collaborating with the federal and state government is key, but there’s a lot more work to be done. Certainly we must bring together healthcare innovators, providers and patients. But more importantly, we must push past cynics and skeptics who continue to support backward thinking, and instead take a leap on behalf of consumers—the people who need to find care every day.

We shouldn’t lose sight of the fact that we are all healthcare consumers in this country. Fighting for them means fighting for our own families and ourselves.