The OfficeBench benchmark measures the time it takes for a system to complete a series of tasks in Microsoft Word, Excel, PowerPoint, and Internet Explorer. The OfficeBench times for the Acer Aspire One AOD150 (117 seconds), Asus N10Jc (115 seconds with Nvidia graphics processor, 123 seconds with Intel GMA 950), HP Mini 2140 (113 seconds), and MSI Wind U123 (118 seconds) show that these Atom-based netbooks are only half as powerful as a full-fledged laptop. (See how these four netbooks stack up in our review.)
A Dell XPS M1710 notebook with a Core 2 Duo (2GHz) processor, 2GB of DDR-2 RAM, a 7,200-rpm hard drive, and an Nvidia GeForce Go 7900GS graphics processor completed my OfficeBench script in just 62 seconds. In addition to their 1.6GHz or 1.66GHz Atom processors, the netbooks also had less RAM (1GB of DDR-2) and slower hard drives (5,400 rpm) than the Dell XPS M1710.
[ Can a netbook serve as your primary PC? See "A week in a netbook, and I'm not missing my regular laptop" and "When does a netbook stop being a netbook?" ]
I created the OfficeBench test script back in 1999/2000 while my company, Competitive Systems Analysis (CSA), was under contract to Intel's Desktop Architecture Labs (DAL). CSA was responsible for a great deal of internal benchmarking and whitepaper development surrounding the Pentium III and Pentium 4 CPU launches.
OfficeBench was designed from the beginning to be a "run anywhere" benchmark. By "run anywhere," I mean that the script will execute reliably under almost any Windows runtime environment. At the time it was being developed, this meant Windows 2000 and Terminal Server. As Windows evolved, so did OfficeBench. Today it supports every version of Windows since 2000, including XP, Vista, Server 2003, Server 2008, all flavors of Terminal Server, and all known application and desktop virtualization environments.
OfficeBench is also version independent. That is, it's designed to work with any version of Microsoft Office. When it was originally conceived, the state of the art was Office 2000. Since then, Microsoft has rolled out three additional versions: XP, 2003, and, most recently, 2007. OfficeBench runs unmodified across all four versions. Combined with the support for the various Windows platform releases, OfficeBench is the only test script of its kind that allows you to compare performance across multiple generations of Windows and Office.
OfficeBench uses OLE automation to drive the applications. This is different from most test scripts, which use window messages or keystroke and mouse click simulation. Using OLE automation has numerous benefits, including allowing test scripts to run unmodified across the four Office versions. It also factors out any input-related anomalies while eliminating the chance that a UI change or third party modification will somehow break the script.
Key OfficeBench tasks include the following:
- Reformat all section headers and subheads in Word.
- Generate multiple chart objects in Excel.
- Generate complete multislide presentation in PowerPoint.
- Multipage scroll with copy/paste of chart objects into Word.
- Slide sort/apply multiple templates in PowerPoint.
- Multipage scroll/print preview/print-to-file in Word.
- Multichart print preview/print-to-file in Excel
- Global search/replace in Word (multiple).
- Multislide preview/print-to-file in PowerPoint.
- Navigate simulated research Web site in IE (multiple).
The above are just some highlights. There's a lot more going on than meets the eye, and the key is that it's the exact same set of tasks executing across all versions of Office.
OfficeBench doesn't exist in a vacuum. It's delivered as part of a sophisticated, extensible, multiprocess testing framework we call DMS Clarity Studio. DMS Clarity Studio provides a variety of scalable workload objects for testing everything from client/server database connections to MAPI-based message store access to streaming multimedia. OfficeBench has been engineered to run in parallel with these workloads, providing for a rich variety of targeted test scenarios spanning the range of Windows client and server platforms. It's all coordinated through the DMS Clarity Studio framework and also seamlessly integrated with the exo.performance.network's Clarity Analysis Portal.
Note: DMS Clarity Studio is offered for free as part of the exo.performance.network. It's also part of the larger DMS Clarity Suite framework in use across thousands of trading workstations and other mission critical systems in the financial services sector. Some of the largest trading firms in the world trust DMS Clarity Suite to tell them when their systems are underperforming.
In summary, OfficeBench is part of a proven testing ecosystem that spans the range of Windows platforms and runtime scenarios. It is a sophisticated, version-independent benchmarking tool that executes reliably under almost any Windows runtime environment, allowing IT organizations to accurately assess multigenerational performance across all versions of Windows and Office.