- Video performance. Video was comparable on all three carriers, Metrico said. AT&T and Verizon both scored above 98 percent average success rates for delivery of a Metrico video clip at 300 Kbps and 20 frames a second, while Sprint scored 96.42 percent on that measure. Video delivery dropped somewhat with YouTube video clips on all three carriers.
- Voice quality. While Sprint fared the best in live network voice quality on the uplink (when speaking) , AT&T took the lead on live network voice quality on the downlink (when the user is listening). Metrico used an algorithm called Perceptual Objective Listening Quality Analysis to make the voice quality comparisons, ranking each carrier from 1 to 5, with 5 as the highest.
On the uplink for voice quality, Sprint scored a 3.95, AT&T was 3.85 and Verizon was 3.5. On the downlink for voice quality, AT&T scored 3.82, Sprint was 3.72 and Verizon was 3.55.
Across the three carriers, Metrico also compared the iPhone 4S with previous tests performed on other 3G smartphones in the carriers' portfolios. It found the iPhone 4S to be among the top performers in low noise, average in speech quality when using Bluetooth, and below average in speakerphone speech quality.
- Call failure rate. Verizon performed slightly better than AT&T and Sprint on the iPhone 4S's ability to make and hold a call successfully. Verizon had a 2.1 percent call failure rate; AT&T had 2.8 percent and Sprint had a 3.7 percent .
Even though Verizon performed the best on call success rates, Metrico said Verizon's iPhone 4S is "well below average" when compared on that measure to Android, BlackBerry, and other phones tested earlier in Verizon's portfolio.
Metrico sells its studies and findings in complete detail to carriers and other companies in the telecommunications space, and only released a six-page summary of the iPhone 4S performance results to reporters. The summary did not provide many data points, including Sprint's Web browsing average performance time, which was revealed by Malhotra in an interview.
Malhotra said the summary and the full 54-page report show that "each carrier has strengths in different areas." He said Metrico was "not out to find a winner, necessarily, but to provide objective data on how each carrier performs across important performance categories."
While some customers will use an iPhone 4S more for data and Web browsing than for voice calls, others will want the opposite, he said. "We won't steer people to one phone or the other," he added. "We're not really trying to highlight the negatives."
One intention of the large nationwide study is to show how performance problems in one city reported by the news media and bloggers shortly after a phone is launched can be less relevant over the entire network over time, Malhotra added. "These [Metrico results] aren't assessments made with a lot of quick tests; there is a lot of variability nationwide in the networks," he added.
Matt Hamblen covers mobile and wireless, smartphones and other handhelds, and wireless networking for Computerworld. Follow Matt on Twitter at @matthamblen or subscribe to Matt's RSS feed. His email address is firstname.lastname@example.org.
Read more about wireless networking in Computerworld's Wireless Networking Topic Center.