Torture the Data Until it Confesses | THCB

Do you recall reading about companies implementing a wellness program or other health benefit changes and reporting tremendous savings? In many cases (most) those savings are a fantasy because the numbers have been manipulated or simply because those running the program don’t know what they are talking about. 

Or, they are trying to substantiate their job and make themselves look good… my experience talking here😏

Following is an excerpt from an interesting blog post explaining this gap between hype and reality … remember all this when you hear about the Obamacare reality gap🤑 

When a company claims they implemented something that caused their health plan costs to drop 15% or so, ask a few questions:

The big question is did the company adjust for plan design changes, such as raising deductibles and copays, that merely shifted costs to employees?

Did the changes really save claim dollars?

Did they factor in stop loss premiums?

How many life years of data did they observe?

Did the company exclude large or “shock” claims? (This is not uncommon, especially among wellness vendors.)

Did it experience any big changes in demographics, such as through implementation of an early retirement program or layoffs that impacted older workers the worst?

When I’ve asked those kinds of questions, I’ve almost never seen a big claim of cost reductions by a small company hold up under scrutiny, and same for some big companies too.

Today torturing the data to get the desired results is all too common. That’s no surprise. They keep catching academics and big pharma doing the same thing. Skepticism is in a good thing.

Source: Torture the Data Until it Confesses | THCB

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s