By AL LEWIS
Last month Aon, the major benefits consulting firm, released a “study” claiming:
A significant opportunity to reduce healthcare costs for employers and enhance overall workforce health through a comprehensive obesity management program that includes GLP-1 medications.
This, of course, is the opposite of what most researchers have shown. And in the immortal words of the great philosophers Dire Straits: “Two men say they’re Jesus, one of them must be wrong.” We’ll shortly see who’s wrong (um, meaning about weight loss drugs) when we dive into the study in a minute. But first, let’s review Aon’s previous analyses.
A brief history of Aon
Aon claimed that Accolade saved 8%, but it looks like they must coincidentally have been absent both on the day that the biostatistics professor explained how control groups work, and also on the day the fifth-grade math teacher explained how averages work.
Then, they claimed that Lyra – which is a mental health company – achieved the following non-mental improvements in the set of patients who had at least one mental health encounter with one of their “220,000 high-quality providers”:
§ A 30% reduction in non-mental health-related ER visits
§ A 30% reduction in generic drug spending
§ A 20% reduction in specialty drug spending
Thanks in part to starting the y-axis at $4000 to improve the optics, Aon also revealed that Lyra achieved a very high “efficiency ratio”:
I can’t object to that finding because – despite three decades in this field, about 100 articles/interviews/quotes/citations including the Wall Street Journal, two trade-bestselling books and one Harvard Business School case study – I still don’t know what an “efficiency ratio” is, other than that has nothing to do with comparing participants to non-participants in a mental health study. Apparently an “efficiency ratio” in healthcare measures how quickly a hospital turns over its inventory. So Aon’s use of the term recalls the immortal words of the great philosopher Bob Uecker: “Juuussst a bit outside.”
When publicly and privately asked to explain any of these things, Aon clammed up. That was likely wise on their part.
Nor will they respond here, because they understand the Streisand Effect. (Barbara sued a photographer for photographing her Malibu mansion from the air as a routine part of documenting erosion along the California coastline at the behest of the state. Six people had downloaded that image before she sued. After she sued, one million people downloaded the image. Also, she lost and had to pay attorney fees as well.)
Most recently, it appears that they may have had their hand in the PBM cookie jar as well.
Aon’s Weight Loss Drug Study
While admitting that costs jump in the first year, Aon found a 7% “bend” in the cost curve in the second year, by participants as compared to a “precisely matched control group.” Matched controls, no matter how “precise,” are invalid, period. That is why the FDA doesn’t let pharma companies use them. Most famously, some Very Stable Geniuses in the wellness industry inadvertently proved this when they published this graph. They thought they were showing that participants in wellness programs saved money vs. matched non-participants. Unfortunately for them, a cursory look at the x-axis reveals the “total savings” from the aptly named “treatment” started two years before the treatment started, simply because voluntary participants are motivated.
The related issue is that over a two-year period – the same duration that Aon studied – most weight loss drug users have dropped out. Yet, there is no accounting for – or mentioning of – dropouts in this study.
Only people still on the drugs are counted. The others would be “lost to follow-up.” Counting only the ones still in the program at the end is called “survivor bias,” or the “last man standing” fallacy. It’s why any weight loss program shows great results – most people quit most programs because they aren’t succeeding. Ironically, the greater the dropout rate, generally the better the results among the few survivors.
It is also quite literally impossible for costs to “bend” 7% overall by reducing the rate of heart attacks and strokes by 44%. That’s because there simply aren’t enough of these events to do that. The rate of both is about 1 per 1000 in the <65 insured population. And Aon didn’t even claim a 44% reduction in those events. They claimed to reduce the “risk” of these events by 44%. A cynic might observe that obviously if they did reduce actual events by that amount, they would have said so.
No need to take our word for this conclusion. We have made our Weight Loss Drug Economics Calculator free. Enter your own assumptions and decide for yourself.
How they can determine someone’s risk from their claims is anyone’s guess. Suppose twins have parents who died early of heart disease. The first is very concerned about this. He takes statins, metformin, maybe sees a cardiologist, gets a stent etc. The second does nothing to mitigate his genetic risk. The second is at much higher risk than the first, but the “risk score” will say the opposite. Many people don’t even know they are at risk for coronary artery disease until they have an event. So how can Aon know?
What is Aon up to?
Surely an actuarial consulting firm whose reputation is based on, well, being an actuarial consulting firm wouldn’t risk that reputation by writing articles like these, right?
Well, certainly not for free.
They got paid by Lyra, got paid by Accolade, and (allegedly) got paid by Express Scripts. In this case – for anyone who doesn’t feel like opening the free Weight Loss Drug Calculator above to figure out themselves – Aon will “work with employers in modeling the long-term business impact of GLP-1 adoption.”
And since their model is wrong, “working with” Aon is – once again in the immortal words of Dire Straits – “money for nothing.”
Al Lewis is CEO of Quizzify, Chairman of the Validation Institute and bete noir of the wellness industry. He blogs occasionally at They Said What?
