In some cases, Engates observes, even the contractors lacked visibility into the infrastructure layer because that was under the control of the hosting provider, Verizon Terremark. "What I think they've got now is a monitoring system that looks down into the application layer and rolls up some of the performance information with the hardware. They have a much better picture with this system," he says.
Now that HealthCare.gov has been running for a while, says Engates, hosting provider Verizon Terremark has been able to throw more powerful hardware at bottlenecks that normally would have surfaced in load testing cycles prior to launch. Engates adds that because the site is for domestic rather than global use, Terremark has the luxury of being able to bring portions of the system down in the middle of the night for upgrades and maintenance.
Lightening the load
Most people have heard about the fatal decision to register people before they could see their plan options. As Engates observes, this turned the conventional e-commerce funnel on its head. Instead of going from general browsing to entering the specific info necessary for a transaction, applicants entered that information first, which incurred a string of activities and dependencies that ultimately brought the system to its knees.
A host of other databases needed to be pinged, including those maintained by the Social Security Administration, Health and Human Services, Homeland Security, and the IRS for such tasks as determining citizenship or whether an applicant qualified for supplemental assistance. As Engates explains, "before, when you compared plans, not only did it have to look up who you were, then it started to look up the plan information from all these data sources. Every time somebody would load a particular page, it was doing multiple queries against multiple databases. Just optimizing the way the system works, presenting data that's pre-computed instead of computed on the fly, I think that can have a tremendous benefit -- and I think they've done quite a bit of that."
Moreover, the option to "see plans before I apply" is now prominently displayed on the Healthcare.gov home page. And of course, third parties can provide Affordable Care Act health plan comparisons as well. Three young San Francisco developers made a name for themselves a few weeks ago by building a site in a matter of days called HealthSherpa, which enables applicants to find which policies are available in their area, whether they may be entitled to subsidies based on income, and how to obtain coverage that meets their needs.
Fixing the basic problem
The HealthCare.gov story is sadly familiar. Back in 2005 I wrote a story about the FBI's Trilogy project, whose $170 million Virtual Case File system was never implemented at all. In that instance, you could blame ridiculous requirements bloat as the main culprit. But in a fundamental sense the same dysfunctional dynamic that whacked the Trilogy project also hammered HealthCare.gov.
As I learned back then, government contractors have huge incentive to agree to unrealistic requirements and deadlines thanks to "cost plus" contracts, which were given both to CGI Federal for HealthCare.gov and to SAIC for the Trilogy project. These types of contracts estimate the real cost of a project and add a profit margin that is awarded annually to the contractor -- in full, in part, or not at all, depending on the government's rating of the contractor's performance for that year.
For the Trilogy article, Gartner Fellow John Pescatore explained the effects of cost-plus contracts in colorful terms: "Here's what happens. In the beginning, you never want to say no, because you'll get a bad rating. It essentially incents the contractor to be much more accepting of out-of-scope changes. It's kind of like a mass-suicide pact, except you're hoping a miracle is going to occur later on."
It's stunning to me that cost-plus contracts, which have been implicated in outrageous Defense Department waste for decades, are allowed to persist across the federal government. Despite failed projects, many of the same contractors pop up again and again. As Engates notes, winning a government contract appears to depend more on the contractor's skill at vaulting bureaucratic and political hurdles than on the ability to build, deploy, and maintain viable systems.
That's how it appears to have gone down with HealthCare.gov. Engates puts it this way: "There are a lot of guys that could have probably done a great job on this. But they aren't necessarily competing on these contracts. And even if they were, they don't have all the right characteristics to win one. In the way these contracts are awarded today, it's very hard to get the right folks on the job."
This article, "The inside scoop on how HealthCare.gov is getting fixed," originally appeared at InfoWorld.com. Read more of Eric Knorr's Modernizing IT blog. And for the latest business technology news, follow InfoWorld on Twitter.