Watching the Java stack

Most enterprise programmers know that the French playwright Jean-Paul Sartre was onto something when he wrote, “Hell is other people.” But they would tell you that hell is really other systems. A set of Web services, databases, and Web applications that work well on their own can slow to a crawl or even deadlock when they’re stitched together. It’s not enough to do a good job on the individual parts because the sum of the parts is something completely different.

Free IT resource Open Source Business Conference (OSBC) May 22-23, 2007 Sponsored by OSBC Free IT resource TechNet: More ways to know it, share it, and keep it running. Sponsored by Microsoft Wily Introscope 5.3.1 Wily Technology, wilytech.com Very Good  8.3 criteria score weight Management 9 20% Monitoring 8 20% Reporting 8 20% Scalability 9 20% Setup 7 10% Value 8 10% Cost: $6,250 per CPU; additional features called PowerPacks may be required Platforms: Any Java JVM; comes equipped with more sophisticated tools that are customized for BEA WebLogic, IBM WebSphere, JBoss, Oracle Application Server 10g, Sun Java System Application Server 7, and others Bottom Line: Wily’s Introscope helps you find the bottlenecks in your application by providing agents that wire up all Java classes to report information to a central application. It provides a great collection of agents for most common Web applications, and you can build custom dashboards to graph this data in a way you can use to understand what goes wrong. About our Reviews and Scoring Methodology

Wily Technology’s Introscope system is meant to be a cure for this confusion or, to be more specific, a tool that might provide the understanding that leads to a cure. Introscope provides a collection of agents that gather data about all the actions in your application stack. If problems strike, you probe this data to find the troublemaking interactions. Wily calls this “Blame technology,” a pitch-perfect gag that illustrates how well the company understands the social landscape of the cubicle farms where teams of programmers toil.

Reporting for duty

The architecture is straightforward. The Enterprise Manager sits at the center collecting logging information from a collection of agents. Anyone who wants to monitor the applications can use the Workstation Explorer to make some sense of the logging data. It’s easy, for instance, to watch the time it takes to answer every SQL query. If one sticks out as slow, you drill down to examine the query itself.

The Explorer is a compact Swing application for all the information it offers. The raw information from the agents can be parsed, graphed, and watched for significant events using a collection of plug-in modules. The information can be graphed or connected to “stoplights” that display green or red to signal good or bad conditions. If some users are being frozen out of information, you drill down and try to isolate the chain of events.

If you want, you can create custom dashboards that allow you to arrange graphs, stoplights, and information for monitoring your applications. The graphical quality of these tools is perfectly adequate, but it might be nice to have some fancier graphics for impressing people who wear suits.

The Explorer relies on a collection of agents to gather the raw data. Many of the most common applications, such as BEA WebLogic, IBM WebSphere, and Oracle, come with standard agents that automatically gather information about the applications. They report high-level details about when requests arrive and when responses are sent back, as well as lower-level information about the memory usage, thread concurrency, and CPU load.

This information helps you locate the bottlenecks in applications. By watching thread concurrency, for instance, it’s possible to look for places where the application is stalling.

There are also more generic agents you can include with your applications. A JDBC agent, for instance, reports on the number of connections available, the response time, and the response size. There are generic tools for tracking most common Java enterprise tools such as servlets, JSPs, and entity beans and some, such as Struts, that are tuned to standard frameworks.

The collection of agents is well designed for the enterprise programmer, although the utility is tempered by the cost of recording information. The basic Struts tool, for instance, tracks the “average method invocation time” for the different Struts Actions. But if you want more details about the different methods, you’ve got to do a bit more work and enable additional gathering.

Any instrumentation such as this must cut corners in some places to keep the logging from swamping the server. Although the Wily agents are generally somewhat lightweight, it’s easy to imagine how they could collect too much information. The standard agents seem well-considered. You drill down to pick up individual SQL queries, but the more memory-intensive responses are thrown away.

The deep end

If the standard agents aren’t enough, you can also quickly instrument any Java class file and pick up the kind of standard method invocation timing often provided by profilers. Entries are automatically created when every method starts and finishes. This deep instrumentation is much more sophisticated than many other monitoring tools, but the method-level information pays off in tricky situations.

The Wily engineers also have an eye for the clever hacks that impress other programmers. One agent I like replaces the base Collections class and then automatically begins gathering information on all collections in an application. This makes it easy to spot the data structures that grow with time because someone forgot to delete information. Wily calls this its LeakHunter.

The data from these agents is recorded in a compact internal format. If you need to look back in time, you flick on the “history” button and all the graphs and charts will magically begin displaying the data gathered during a time frame you specify. This makes it easier to go back in time and diagnose intermittent problems reported long after the system has recovered.

The ability to drill down into the system and track performance over time makes it much easier to spot the kind of deadlocks and bottlenecks that emerge when systems grow unevenly. Sometimes the best works of the best programmers don’t work well together, and instrumentation such as this may be the only way to identify the weakest part of the system. Wily’s technology gives enterprise programmers an ideal tool for wiring up an entire application stack and playing the blame game.