| About InfoWorld : Advertise : Subscribe : Contact Us : Awards : Events : Store |
 |
||||
|
||||
 | ||||
 |
|
|||
 |
||||
|
|
|
||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 TEST CENTER It's not a bug, it's a feature By Randall C. Kennedy November 21, 2001 IT'S AN OLD STORY: Developer writes code. Code taps vendor API. Vendor changes API without properly documenting the effects of the change. Developer gets burned.
This change may seem innocuous, but only until you consider how a sudden switch of network libraries can affect application behavior. The TCP/IP library for SQL Server consumes more CPU cycles than does the Named Pipes library, especially in transactional scenarios where requests are batched together to improve performance. More CPU time for the libraries means less CPU time for the application. It's not hard to see how the introduction of an unexpected external load (more demanding libraries) can adversely affect a heavily tasked system. Looking for answers We discovered this phenomenon quite by accident while testing client/server database workloads on a Windows XP PC. Although throughput to and from the server was excellent, the processing overhead involved in maintaining the SQL Server connection began to overwhelm the PC. Multitasking test scenarios that required only seconds to complete on Windows 2000 were taking several minutes on Windows XP. It wasn't until after several conversations with Microsoft that we learned of the change in default MDAC behavior. And, as it turned out, we were indeed executing a test script -- an OLE (Object Linking and Embedding) database connection string -- that relied on the original behavior, namely, defaulting to Named Pipes. The story might have ended there had we not decided to dig deeper. For starters, we wanted to know exactly when the change became part of a shipping MDAC release. Was Windows XP the first product to feature the revised libraries? Were there other products or service packs that might feature the update? We also wanted to know where and how the change was documented. Changing a default behavior and then documenting it is risky enough; changing a default behavior and leaving the change undocumented (or poorly documented) is unforgivable. It's the kind of irresponsible indifference that ultimately alienates developers. We eventually learned from Microsoft that the change first appeared in MDAC 2.6, the version that shipped with SQL Server 2000 (Windows 2000 ships with MDAC 2.5) and that the change in default behavior was made, ostensibly, at the request of several of Microsoft's larger corporate customers. As for documentation, a thorough review of the release notes for MDAC 2.6 and MDAC 2.7 revealed no mention of the change. Neither did an exhaustive search of Microsoft's Knowledge Base. In more than 200 articles on MDAC we found not a single mention of a change to the default network library. In fact, Microsoft was hard-pressed to find a reference itself. In the end the company referred us to a single Knowledge Base article, one focusing on an otherwise unrelated topic (ODBC), where the reference was made mostly in passing. So there we were. We had identified the source of the performance anomaly: A poorly documented, one might argue undocumented, change in the default behavior of a key MDAC component. We almost let the issue die right there, but something Microsoft said kept gnawing at us: The change to the default network library first appeared in MDAC 2.6. This version of MDAC, we learned, was released specifically to support new features in Microsoft SQL Server 2000. All of this evidence begged the question, if Microsoft is frequently updating MDAC to support its server products, how would the introduction of a change such as that in the default network library affect a more servercentric solution, such as a database-driven Web site? Back to the test lab Naturally, the only way to find out was to build an actual Web site and test it; and that's what we did. Using our soon-to-be released ASP (Active Server Pages) workload simulator, ASP Stress, we conducted a series of tests to gauge transactional database query performance using a dual-processor Pentium III Web server running Windows 2000 with MDAC 2.7. By monitoring the server's CPU utilization counters via the Windows Performance Monitor application, as well as our own ASP Stress transaction timer, we were able to measure the impact of a switch from Named Pipes (the Windows 2000/MDAC 2.5 default) to TCP/IP (the Windows XP/MDAC 2.6 and MDAC 2.7 default). And impact there was. Although overall server CPU utilization rose by a modest 8 percent using TCP/IP, context switches per second -- a measure of the operating system's ability to juggle multiple tasks -- dropped by more than 150 percent for a 10-user workload. In real-world terms, this meant a 10 percent to 15 percent loss in overall SQL Server transaction performance as we scaled the server from 10 users to 25 users and finally to 50 users. Needless to say, this is the sort of unexpected behavioral change that can cause a particularly busy Web application to start dropping connections or, in extreme cases, to crash. And it happens because Microsoft switched defaults to appease a few of its preferred customers. To be fair, we can't blame Microsoft for trying to keep its top IT partners happy. That's just good business. But it certainly should have taken the time to clue the rest of us in about the changes. The moral of the story is this: Never put trust in a default behavior, especially when a sudden change to that behavior could have an adverse effect on application performance. Of course it doesn't help that most examples of Microsoft SQL Server connect string code omit the Network Library parameter, thereby encouraging developers to rely on the default value. Although it's true that many IT shops will catch such changes during qualification testing (assuming scalability is part of the test matrix), we can't help but be concerned when Microsoft starts to tinker with such an integral part of its key server products such as SQL Server, and developer tools such as Visual Studio, both of which often incorporate updates to MDAC.  A 17-year veteran of the IT industry, Randall C. Kennedy is director of research and co-founder of Competitive Systems Analysis. Contact him via the CSA Research Web site at www.csaresearch.com .
 RELATED SUBJECTS  Web TechnologiesSPONSORED WHITE PAPERS 
SPONSORED LINKS
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||
|
||||||||||
