The VoIP management challenge

Enterprise network managers are implementing VoIP, but how are they managing these installations? Here are four tools to simplify the task

The slowly maturing VoIP platform has enterprises taking a long look at this technology, and not just for cost savings. Many of these potential users are opting for in-house rather than service-provider solutions because of the additional savings, future flexibility, and a simple lack of consistent vision by the large telco providers.

Those adventurous souls who implement VoIP themselves face the daunting task of managing a voice-oriented data network. For network managers accustomed to data networks, this job requires a specialized toolkit.

To determine the best option for the enterprise market, we gathered four VoIP testing contenders and turned them loose on the VoIP installations at our Advanced Network Computing Laboratory (ANCL) testing facility at the University of Hawaii.

The reviews below aren't intended as direct comparisons. These testing tools are different enough in their approaches to the VoIP management problem that an apples-to-apples comparison is simply unworkable. For this reason, we instead examined each tool on its own merits and rated them from the perspective of a typical network manager.

Acterna DA-3400 Data Network Analyzer

We reviewed the network testing capabilities of Acterna's product line in our March roundup. This time around, Acterna brought a DA-3400 equipped with its latest VoIP testing software technology. Heavily aimed at service providers and carrier-style implementations, the Acterna solution is feature-rich but probably overkill for all but the largest enterprise VoIP implementations.

As before, the DA-3400 comprises an innocuous, 1U rack-mountable box, which we attached to our network via a Net Optics tap. That's important, because unlike the Brix product, the DA-3400 is an entirely passive device.

To access the DA-3400's PVA-1000 VoIP analyzing software, you log into the box using a Web browser. We plugged the DA-3400 into our lab network and decided to expand its testing scope to reflect its carrier orientation. We skipped managing local VoIP traffic and instead aimed the device at an Asterisk SIP server used by Priority Networks.

We initiated several conversations using Priority's network and the same people in each conversation to preserve audio frequency ranges. We measured call quality during a series of two-minute conversations and captured both sides of the conversation so that we could play it back later.

To measure QoS and Diffserv, the DA-3400 displays the appropriate priority bits that tell the receiver (or router) what priority the traffic has. The big differentiator in Diffserv is that the priority labels must be correctly configured in order for the switch to "differentiate" the data streams.

Although Diffserv is mostly standardized by now, the standards continue to be implemented differently enough across platforms that having the ability to dig down to the bit level with the DA-3400 can be critical to fast problem resolution.

When measuring call quality, the DA-3400 tracked the RTP (Real-Time Transport Protocol), RTCP (RTP control protocol), and packet values along with call delay, codec identification, lost packets, and jitter. Additionally, it is capable of following the TIPHON (Telecommunications and Internet Protocol Harmonization Over Networks) perceptual call quality assessment protocol for calls in progress.

Surprisingly, our call quality across the Las Vegas-based Asterisk server was just as good as a local POTS call, however we were running across Internet2 with a cross-connect onto Qwest's backbone for the commodity Internet.

Our perceived test scores were excellent, but the DA-3400 was unable to display the actual MOS (mean opinion score), because the user interface displays MOS results as a visual band and not an actual number. This is a recurring concern: Acterna occasionally oversimplifies the results of its rather powerful tool set. The visual-only display of the MOS scores is one example; another is the rather light implementation of SNMP traps.

Acterna says it will address many of these complaints in the next version of its software, due this month. In addition to detailed MOS scores and improved SNMP, you'll also be able to integrate the Acterna console into larger management packages, such as Hewlett-Packard OpenView and IBM Tivoli.

We were surprised by this, given Acterna's leanings toward the carrier side of VoIP implementations. Clearly, Acterna smells opportunity in the enterprise market. Larger enterprises can make good use of the DA-3400, although Brix's offering is more specifically geared for such an installation.

The Brix System

In case you're wondering,  Brix's etymology comes not from masonry slang, but from a term related to the proper processing of grapes into wine. After unearthing this data nugget, we dug into Brix Networks' offerings to find a pleasant surprise: This relatively new company has a well-thought-out, comprehensive suite of VoIP monitoring and testing products and services that's fully capable of competing with more established vendors' offerings.

The Brix product line is a software/hardware combination. The hardware appliances, called Verifiers, come in three flavors: Brix 100, Brix 1000, and Brix 2500. These boxes run agent software, essentially the long arm of the Brix monitoring system. Customers may install the agent software on workstations as long as they purchase the proper number of Brix Verifier Agent software licenses.

Command of the Brix system falls to either a BrixMon enterprise-class centralized management console or to a BrixWorx console, which is designed to run within a service provider's network monitoring system and control Brix Verifier Agents across multiple customer sites.

Because of our enterprise focus, we tested the BrixMon enterprise solution. Brix arrived at our ANCL testing facility with a preconfigured BrixMon workstation, two Brix 100 Verifiers, and several copies of Verifier Agent software.

Deploying the Brix system was easy, because the BrixMon software was preinstalled on Brix's Windows-based workstation, and because discovery and configuration of the Verifiers was largely automatic. After the system knew where the Brix 100s were, we could configure them for long-term monitoring or short-term testing tasks. Most of these features are specifically designed to enhance performance management and provide service-level assurances, which suggests that BrixMon still has ties to its BrixWorx sibling's service-provider orientation.

For those with large VoIP installations, however, the BrixMon system is an excellent central-site management solution. It provides not only secure management communications but also all the tools an enterprise network manager requires.

One key feature is Brix's ability to automatically launch network tests based on preset performance thresholds. If the monitoring system reports a problem, for example, the same agent machine can initiate network tests that will provide the service technician with detailed diagnostic data as soon as he starts work on the problem.

BrixMon's interface is Web-based, but that doesn't impede the feature set or system performance. In fact, we were pleasantly surprised by its detailed reporting capabilities. The reports run in real time, often doing double duty as performance analyzers.

Brix thoughtfully includes support for extended historical reporting data, allowing the Brix System to act as a data mine for long-term network performance and service-level questions. For the technician on the go, the Brix System contains the usual suite of executive-oriented reports, plus a Quick Report suite that extracts down-and-dirty technical data for diagnostics and management. 

Because the Brix System can act as a general network management tool, the features necessary for high-end VoIP management are part of the optional BrixMon Advanced VoIP Test Suite. It provides the tools to conduct ongoing VoIP management and allows predeployment testing -- a feature near and dear to the hearts of those deploying VoIP in a place where the suits are extra sensitive to call quality.

The system is designed to run its test suite against a variety of VoIP protocols, including SIP, H.323, and Cisco SCCP (Skinny Client Control Protocol). Within these protocols, Brix tests call setup performance, call quality, and transmission speed.

With support for H.323, it's no surprise that Brix makes available an optional Video Test Suite for testing video conferencing. The suite includes a wealth of network statistical verifiers, from voice and video quality to packet loss, jitter, and bandwidth utilization. Acerna and Fluke Networks can do this using different software or different features of their software, but the functionality is aimed at general network management metrics, not specifically at VoIP.

Because of its youth, the Brix System isn't as fully utilized as the architecture will support. Therefore Brix's strategy to implement new testing capabilities as software that can be installed on the BrixMon station and then pushed out to Verifiers is an excellent way for users to protect their investment -- they'll gain significant new features via software updates without resorting to new hardware purchases.

Although the Brix system is overkill for smaller or local VoIP installations, it's one of the best we've seen for managing larger enterprise VoIP deployments, especially those with many branch offices.

Fluke Networks OptiView Protocol Expert Plus

For large and midsize companies interested in managing a VoIP implementation in-house, this analyzer does a fine job of augmenting Fluke Networks' OptiView software line. Fluke Networks already has significant market penetration with small-IT-budget companies, and these customers can now access robust VoIP monitoring and testing tools without ponying up for an all-new tool suite.

OptiView is largely software-based, but Fluke allowed us to test the software using a dedicated hardware appliance, so installation was smoother. Our box was a 1U rack-mountable unit, though we kept it outside the rack during testing. We tested the new OptiView Protocol Expert Plus.

For those with their own workstations, OptiView runs on any NT, Windows 2000, or Windows XP Pro workstation. Network managers familiar with the OptiView interface and features will appreciate how well Fluke Networks integrated its VoIP tools with the existing OptiView software.

The Expert Plus version contains the same expert packet- and decode-analysis found in Protocol Expert, but also remotely monitors and even controls other Protocol Expert boxes and handheld OptiView Link Analyzers. Whereas this architecture isn't dedicated to distributed management capabilities (as in the Brix design), it does manage to mirror much of that functionality without a huge investment in new software and hardware.

It still offers all the usual goodies, including line rate traffic monitoring and traffic capture, packet decoding, Gigabit Ethernet support, and the usual series of network management metrics such as top talkers, VLAN analysis, utilization, and error rate.

Fluke Networks also adds its aptly named VoIP Option. This plug-in software for existing OptiView installations gives network managers many enterprise-oriented VoIP management tools.

With OptiView's history as a protocol analyzer, we expected support for a wide variety of VoIP protocols -- and we weren't disappointed. VoIP Option supports not only SIP, H.323, and Cisco SCCP, but also ASN.1, MGCP (Media Gateway Control Protocol), and SGCP (Simple Gateway Control Protocol). That's more than any other tool we reviewed.

The tool is heavy on QoS validation and measurement, too. It measures real-time QoS metrics for completed calls, initiated calls across most of its supported protocols, and even active calls.

It extends the QoS philosophy to other measurements as well, allowing users to set customer-defined "Quality Grades" for jitter, packet loss, r-factor, and setup time. These measurements can be served up numerically or graphically, presenting calls that measure up to the preset thresholds and those that do not.

In testing, we ran the OptiView a differently, as OptiView's engineer decided to plug the device into the switch port on the back of our Cisco 7960 VoIP phone instead of using a copper tap as we did with Acterna's device.

Under normal circumstances, the OptiView unit would be inserted into a network rack to monitor a SPAN's (switched port analyzer's) port. Indeed, Fluke Newtorks has a copper tap that looks suspiciously like a Net Optics unit, but our engineer didn't use one.

Unfortunately, because the port on the back of the 7960 is a switch not a hub, the OptiView couldn't see both sides of a conversation during monitoring, although it could play back the full conversation later.

After installation, however, the box gave almost exactly the same statistics as the Acterna DA-3400. The Fluke Networks unit did give us an actual MOS score, and the MOS scores were nearly identical to those measured by the

DA-3400, covering both RTP and RTCP to determine call quality by delay and jitter values of the packets themselves.

With the lowest price tag in the review, and the capability to drop its feature set onto a network management solution that large numbers of network managers already own and use, Fluke Networks' OptiView has a big leg up on the enterprise VoIP management market.

Although its features and tools are very similar to those of Acterna's

DA-3400, it manages them in a slightly easier and certainly more accessible user interface. The Brix System is more flexible, but the OptiView has had more time to win the hearts and minds of the same network managers who are now implementing VoIP, especially those from midsize companies looking to maximize their IT budgets.

Spirent Abacus 5000

Unlike the other tools in this roundup, Spirent's Abacus 5000 isn't intended to measure VoIP network quality. Instead, it is intended for manufacturers of VoIP equipment to measure just how well their products meet industry standards. The type of VoIP gear it's capable of testing ranges from small VoIP appliances all the way up to large, telco-oriented VoIP PBX and switching products.

Like most of Spirent's products, the Abacus 5000 comes as a stand-alone box equipped with an expandable port chassis to accommodate multiple test configurations. All hardware is controlled by a dedicated software suite running on an attached workstation using Windows 2000 or Windows XP Pro.

Our only gripe with the Abacus' software is that each phase of a test setup opened a new window, resulting in so many simultaneous windows that a multiheaded video setup would have been useful. Having additional screen real estate can go a long way toward making sure you don't miss a screen squashed at the bottom of the mosh pit.

Because the most important measurement is the user's experience, the Abacus approach to measuring call quality is more real-world than the simple MOS scores used by many other systems. Instead of just measuring through a simulated phone, Spirent has an adapter that replaces the phone's audio handset (the earpiece) with an electrical interface so that it can send and receive real WAV files.

Spirent goes a bit further and uses various WAV file recordings of different people speaking to measure voice quality over a wide range of users, instead of relying on the so-called "95th percentile human" theoretical measurement used by other test suites.

Straight standard-compliance and product testing is hardly the device's limit. A wide variety of accessories turn Abacus into an all-purpose VoIP tool -- as long as you're willing to spend the additional retrofitting time and, of course, the bucks. You can attach POTS interfaces to be configured as FXO (Foreign Exchange Office), FXS (Foreign Exchange Subscriber), T1/E1 Ethernet, or a multitude of other telco-standard interfaces.

Calls are simulated either through the telco interfaces or by creating simulated VoIP handsets (SIP, MGCP, H.323) with variable ramp up times. This is valuable data considering that every time a VoIP handset is turned on, it must obtain its configuration from the proxy server. If call accounting is turned on, then the handset must also create entries in a database.

Thus, being able to vary the ramp-up rate allows the Abacus to more realistically simulate the real world. It's impossible to simultaneously power up 200 or more phones, which is why this ability to ramp up the call rate exists. You can also control a call's start, end, and duration in addition to the type of voice or sound played back over the link.

Another interesting Abacus feature is trunking. Trunking will aggregate a large number of calls to the PBX as if they were coming in from a gateway or T1/E1 channelized telco circuit, or as if a number of calls were coming in from another PBX over the Ethernet. That means the Abacus can test a large number of phones over a variety of links, and it can test them in a variety of implementations.

Abacus' only catch is that it assumes that any products it tests adhere strictly to published standards. But as we discovered during InfoWorld's recent VoIP PBX test, the VoIP industry rarely does this.

We used the Abacus as the central test device in that review of four VoIP PBX products. Three of those four PBXs had tweaked their implementation of the SIP or H.323 protocol enough to flummox the Abacus and require dedicated fix upgrades from Spirent's technical staff. The device that stuck strictly to the SIP standard easily completed all the Abacus' tests.

This standards-tweaking forced Spirent to spend an amazing amount of engineer time working with each VoIP vendor in our PBX review so that each would handle exceptions. Although this sounds like a real problem, keep in mind that the Abacus seems to have been designed by telco engineers, and the telco world is built around the strict implementation of standards.

Still, standards-tweaking and extra engineering fixes can't continue to be the norm. Extending simple protocols such as SIP and H.323 may provide custom features, but the VoIP marketplace can't allow noninterchangeable, nonstandard phone systems for much longer. The effort required to integrate systems is simply too big a financial burden for service providers or large enterprises to bear.

Even with this caveat, the Abacus isn't designed for enterprise VoIP network management. Although it's certainly flexible enough to manage such tasks, its price point and engineering focus just don't extend in that direction. This is a VoIP device testing tool for telcos and at that job, it truly excels.

InfoWorld Scorecard
Features (25.0%)
Performance (25.0%)
Value (10.0%)
Setup (10.0%)
Standards (15.0%)
Scalability (15.0%)
Overall Score (100%)
Acterna DA-3400 Data Network Analyzer 8.0 8.0 7.0 8.0 6.0 8.0 7.6
Brix System 8.0 8.0 7.0 7.0 7.0 9.0 7.8
Fluke Networks OptiView Protocol Expert Plus 8.0 7.0 8.0 8.0 9.0 7.0 7.8
Spirent Abacus 5000 8.0 8.0 7.0 7.0 5.0 8.0 7.4
From CIO: 8 Free Online Courses to Grow Your Tech Skills
Join the discussion
Be the first to comment on this article. Our Commenting Policies