I created the OfficeBench test script back in 1999/2000 while my company, Competitive Systems Analysis (CSA), was under contract to Intel's Desktop Architecture Labs (DAL). CSA was responsible for a great deal of internal benchmarking and white paper development surrounding the Pentium III and Pentium 4 CPU launches.
OfficeBench was designed from the beginning to be a "run anywhere" benchmark. By "run anywhere" I mean that the script will execute reliably under almost any Windows runtime environment. At the time it was being developed, this meant Windows 2000 and Terminal Server. As Windows evolved, so did OfficeBench. Today it supports every version of Windows since 2000, including XP, Vista, Server 2003, Server 2008, all flavors of Terminal Server, and all known application and desktop virtualization environments.
OfficeBench is also version independent. That is, it's designed to work with any version of Microsoft Office. When it was originally conceived, the state of the art was Office 2000. Since then, Microsoft has rolled out three additional versions: XP, 2003, and, most recently, 2007. OfficeBench runs unmodified across all four versions. Combined with the support for the various Windows platform releases, OfficeBench is the only test script of its kind that allows you to compare performance across multiple generations of Windows and Office.
OfficeBench uses OLE automation to drive the applications. This is different from most test scripts, which use window messages or keystroke and mouse click simulation. Using OLE automation has numerous benefits, including allowing test scripts to run unmodified across the four Office versions. It also factors out any input-related anomalies while eliminating the chance that a UI change or third party modification will somehow break the script.
Key OfficeBench tasks include the following:
- Reformat all section headers and subheads in Word.
- Generate multiple chart objects in Excel.
- Generate complete multi-slide presentation in PowerPoint.
- Multi-page scroll w/copy paste of chart objects into Word.
- Slide sort/apply multiple templates in PowerPoint.
- Multi-page scroll/print preview/print-to-file in Word.
- Multi-chart print preview/print-to-file in Excel
- Global search/replace in word (multiple).
- Multi-slide preview/print-to-file in PowerPoint.
- Navigate simulated research Web site in IE (multiple).
The above are just some highlights. There's a lot more going on than meets the eye, and the key is that it's the exact same set of tasks executing across all versions of Office.
OfficeBench doesn't exist in a vacuum. It's delivered as part of a sophisticated, extensible, multi-process testing framework we call DMS Clarity Studio. DMS Clarity Studio provides a variety of scalable workload objects for testing everything from client/server database connections to MAPI-based message store access to streaming multimedia. OfficeBench has been engineered to run in parallel with these workloads, providing for a rich variety of targeted test scenarios spanning the range of Windows client and server platforms. It's all coordinated through the DMS Clarity Studio framework and also seamlessly integrated with the exo.performance.network's Clarity Analysis Portal.
Note: DMS Clarity Studio is offered for free as part of the exo.performance.network. It's also part of the larger DMS Clarity Suite framework in use across thousands of trading workstations and other mission critical systems in the financial services sector. Some of the largest trading firms in the world trust DMS Clarity Suite to tell them when their systems are under-performing.
In summary, OfficeBench is part of a proven testing ecosystem that spans the range of Windows platforms and runtime scenarios. It is a sophisticated, version-independent benchmarking tool that executes reliably under almost any Windows runtime environment, allowing IT organizations to accurately assess multi-generational performance across all versions of Windows and Office.