The Bugs Yet To Be Fixed

David Auerbach assesses the current state of Healthcare.gov:

The report says little about what work is ahead of the engineering team, but at least one graph suggests there’s some ways to go:

software fixes.

Aside from confirming that it took the administration until early November just to start making significant progress on functionality fixes, the graph also indicates that engineers are still frantically fixing the site. If the site were anywhere near stability, you would expect the number of fixes to start to level off close to the administration’s self-imposed Nov. 30, signaling that they were ramping down crisis mode and making fewer fixes per day. But it doesn’t, indicating that they are still in the hump of this bug-fixing graph:

Bug fixes.

Suderman worries that fixing these bugs will prevent the rest of Healthcare.gov from being built properly:

With 30 to 40 percent of the site, including critical insurer payment systems, yet to be completed and tested, you can bet these problems will continue. Not only because it clearly takes longer than expected to excise flaws from the system, but because lingering problems with the portion that’s already been built will take time and energy away from constructing and testing the portions of the system that have yet to be put in place. The administration delayed the Spanish language version of site from its initial planned opening, and just last week announced that the federally run small business exchange that was supposed to open last month would be postponed by a year. The team working on Obamacare is already reported to be working around the clock on repairs; even if they don’t burn out from the weeks of long hours, it’s inevitable that building additional functionality and putting it through the paces will fall by the wayside if the existing troubles aren’t fixed.

Update from a reader:

In this post, the data regards bug fixes per day. To understand the stability of the site, however, you need to know how many new bugs are found per day (by end-users and QA staff). If their bug backlog was at 600 on November 1, and no other bugs have been filed, then the fix rate will drop off drastically in a week or two. If their bug backlog is still at 600, however, then the bug-fix trend line could continue to rise for months to come.  To get a true understanding of the stability of the web site, we need to see open bugs over time and/or bugs filed over time.