Our tests show that Windows Vista and Office 2007 not only smash Redmond's previous records for weight gain, but given the same hardware diet, run at less than half the speed of generation XP
The Bronze Age: Windows XP/Office XP
The introduction of Windows XP in 2001 marked the first mainstream (not just for business users) version of Windows to incorporate the Windows NT kernel. In addition to better plug-and-play support and other improvements, XP sported a revamped user interface with true-color icons and lots of shiny, beveled effects. Not wanting to look out of style, and smelling another sell-up opportunity, the Office group rushed out Microsoft Office XP (aka Office 10), which was nothing more than a slightly tweaked version of Office 2000 with some UI updates.
Hardware had evolved a bit in the two years since the Windows 2000 launch. For starters, Intel had all but abandoned its ill-fated partnership with Rambus. New Intel designs featured the more widely supported DDR-SDRAM, while CPU frequencies were edging above 2GHz. Intel also upped the L2 cache size of the Pentium 4 core from 256KB to 512KB (the Northwood redesign) in an attempt to fill the chip's stall-prone 20-stage integer pipeline. Default RAM configurations were now routinely in the 256MB range, while disk drives sported ATA-100 interfaces.
Windows XP, especially in the pre-SP2 timeframe, wasn't all that more resource intensive than Windows 2000. It wasn't until later, as Microsoft piled on the security fixes and users started running anti-virus and anti-spyware tools by default, that XP began to put on significant weight. Also, the relatively modest nature of the changes from Office 2000 to Office XP translated into only a minimal increase in system requirements. For example, overall working set size for the entire Office XP suite during OfficeBench testing under VMware was only 10MB, just 1MB higher than Office 2000, while CPU utilization actually fell 1 percent across the three applications (Word, Excel, and PowerPoint). This did not, however, translate into equivalent performance. As I noted before, Office XP on Windows XP took 17 percent longer than Office 2000 on Windows 2000 to complete the same OfficeBench test script. View the overall test results. View more detailed test results at xpnet.com.
I was fortunate enough to be able to dig up a representative system of that era: A 2GHz Pentium 4 system with 256MB of RAM and integrated Intel Extreme graphics. Running the combination of Windows XP (SP1) and Office XP on bare iron allowed me to evaluate additional metrics, including the overall stress level being placed on the CPU.
By sampling the Processor Queue Length (I ran the DMS Clarity Tracker Agent in parallel with Clarity Studio and OfficeBench), I was able to determine that this legacy box was only moderately stressed by the workload. With an average Queue Length of three ready threads, the CPU was busy but still not buried under the computing load. In other words, given the workload at hand, the hardware seemed capable of executing it while remaining responsive to the end-user (a trend I saw more of as testing progressed).
The Industrial Revolution: Windows XP/Office 2003
Office 2003 arrived during a time of real upheaval at Microsoft. The company's next major Windows release, code-named Longhorn, was behind schedule and the development team was sidetracked by a string of security breaches in the Windows XP code base. The resulting fix, Windows XP Service Pack 2, was more of a relaunch than a mere update. Whole sections of the OS core were either replaced or rewritten, and new technologies – such as Windows Defender and a revamped firewall – added layers of code to a rapidly bloating platform.
Into this mess walked Office 2003, which, among other things, tried to bridge the gap between Windows and the Web through support for XML and the ability to store documents as HTML files. Unlike Office XP, Office 2003 was not a minor upgrade but a major overhaul of the suite. And the result was, not surprisingly, more bloating of the Windows/Office footprint. Office suite memory consumption went up modestly to 13MB during OfficeBench testing, while CPU utilization remained constant versus previous builds, despite the fact that the suite was spinning an extra four execution threads (the overall thread count was up by 15).
Where the bloat took its toll, however, was in raw application throughput. Completion times under VMware increased another 8 percent vs. Office XP, putting the Windows XP (SP2) and Office 2003 combination a full 25 percent off the pace of the original Windows 2000/Office 2000 numbers from three years earlier. In other words, with all else being equal – hardware, environment, configuration – Microsoft's desktop computing stack was losing in excess of 8 percent throughput per year due to increased code path complexity and other delays. View the overall test results. View more detailed test results at xpnet.com.
Of course, all else was not equal. Windows XP (SP2) and Office 2003 were born into a world of 3GHz CPUs, 1GB of RAM, SATA disks, and symmetrical multithreading (that is, Intel Hyper-Threading). This added hardware muscle served to offset the growing complexity of Windows and Office, allowing a newer system to achieve OfficeBench times slightly better (about 5 percent) than a legacy Pentium 4 system, despite the latter having a less demanding code path (TGMLC in action once again).
Welcome to the 21st century: Windows Vista/Office 2007
Given the extended delay of Windows Vista and its accompanying Office release, Microsoft Office 2007, I was understandably concerned about the level of bloat that might have slipped into the code base. After all, Microsoft was promising the world with Vista, and early betas of Office showed a radically updated interface (the Office "ribbon") as well as a new, open file format and other nods to the anti-establishment types. Little did I know that Microsoft would eventually trump even my worst predictions. Not only is Vista and Office 2007 the most bloated desktop software stack ever to emerge from Redmond, its system requirements are so out of proportion with recent hardware trends that only the latest and greatest from Intel or AMD can support its epically porcine girth.
Let's start with the memory footprint. The average combined working set for Word, Excel, and PowerPoint 2007 when running the OfficeBench test script is 109MB. By contrast, Office 2000 consumed a paltry 9MB, which translates into a 12-fold increase in memory consumption (170 percent per year since 2000!). To be fair, previous builds of Office benefited from a peculiar behavior common to all pre-Office 12 versions: When minimized to the task bar, each Office application would release much of its noncritical working set memory. This resulted in a much smaller memory footprint, as measured by the Windows performance counters (which are employed by the DMS Clarity Tracker Agent used in my tests).
Microsoft has discontinued this practice with Office 2007, resulting in much higher average working set results. However, even factoring in this behavioral change, the working set for Office 2007 is truly massive. Combined with an average boot-time image of more than 500MB for even the minimal Windows Vista code base, it seems clear that any system configuration that specifies less than 1GB of RAM is a nonstarter with this version. And none of the above explains the significantly higher CPU demands of Office 2007, which are nearly double that of Office 2003 (peak utilization of 73 percent versus 39 percent). Likewise, the number of execution threads spawned by Office 2007 (32) is up, as is the total thread count for the entire software stack, which is nearly double the previous version (615 versus 370). View the overall test results. View more detailed test results at xpnet.com. Compare memory consumption.
Clearly, this latest generation of the Windows/Office desktop stack was designed with the next generation of hardware in mind. And in keeping with the TGMLC pattern, today's latest and greatest hardware is indeed up to the challenge. Dual cores, combined with 4MB or more of L2 cache, have helped to sop up the nearly doubled thread count, while 2GB standard RAM configurations are mitigating the nearly 1GB memory footprint of Vista and Office 2007.
The net result is that, surprise, Vista and Office 2007 on today’s state-of-the-art hardware delivers throughput that's still only 22 percent slower than Windows XP and Office 2003 on the previous generation of state-of-the-art hardware. In other words, the hardware gets faster, the code base gets fatter, and the user experience, as measured in terms of application response times and overall execution throughput, remains relatively intact. The Great Moore's Law Compensator is vindicated.
Give and take
The conventional wisdom regarding PC evolution, that Microsoft devours every Intel advance, continues to hold true right up through the current generation of Windows Vista and Office 2007. What's shocking, however, is the way that the IT community as a whole has grown to accept the status quo. There is a sense of inevitability attached to the concept of the Wintel duopoly, a feeling that the upgrade treadmill has become a part of the industry's DNA. Forces that challenge the status quo – such as Linux, Google, and Apple – are seen as working against the very fabric of the computing landscape.
But as Microsoft is learning, you can only push your customers so far before they push back. In the case of Windows Vista, the combination of heavy hardware requirements and few tangible benefits to IT has resulted in a mass rejection of Microsoft’s latest and greatest. Companies are finally saying enough is enough and stepping off the treadmill, at least for a while. Microsoft’s challenge will be to woo these customers back, and they can start by taking a hard look at their OS and application development practices. Instead of targeting the next generation of hardware, Microsoft engineers should try making sure that their new features and functions work well on the hardware of today, thus guaranteeing that they won’t overshoot their target and disrupt the fragile TGMLC balance. Maybe then customers will start looking forward to upgrading for a change.
You may still be better off sticking with Win7 or Win8.1, given the wide range of ongoing Win10...
With myriad problems now evident, it may be best to skip the Anniversary Update for now
An unlikely combination of two Windows updates can reduce scan times from hours to minutes
These 13 tools and techniques prove that, when it comes to coding, laziness is a virtue
GitLab and Atlassian have GitHub in the cross-hairs among organizations seeking enterprise-grade...
Concurrency and runtime improvements make the JVM language attractive for IoT development
When a core team member bows out, a crucial process hits an insurmountable obstacle -- until IT figures...