An acquisition of Avaya by Nortel might help the buyer expand its enterprise telephony business and pare down a crowded industry, but the deal could be tough for customers.

Avaya is in talks with possible buyers and recently discussed a possible deal with Nortel, the Wall Street Journal reported Tuesday. Together, the companies would become the world's biggest vendor of telephony gear with as much as 35 percent of the market in its home base of North America, according to analyst Inder Singh of Prudential Equity Group.

"Anywhere you look, they would become the dominant vendor," Singh said.

Nortel, still emerging from years of accounting scandals and reorganizations, could use Avaya to boost its enterprise business, which delivers higher margins than its carrier network side, according to Singh. Because the companies have many similar products, the deal might also help Nortel cut costs by eliminating redundancies, he added.

But for customers, the combination might not be such a good deal. Although Nortel is a bigger company, Avaya is the stronger of the two in IP telephony, which is where enterprise phone systems are going. Integration with Nortel and its unified communications partner, Microsoft, would cause a lot of confusion for customers, said Yankee Group analyst Zeus Kerravala. Avaya has made great technical strides recently, especially in software, he said.

"Darwin should take over here and let the strongest survive," Kerravala said.

One possible benefit to customers would be a one-stop shop with a strong telephony lineup as well as enterprise data networking gear, such as switches, which Avaya lacks, said Dell'Oro Group analyst Alan Weckel.

Even if Nortel doesn't buy Avaya, analysts see acquisitions on the horizon in the industry. Enterprise telephony is a fragmented business: In 2006, in terms of numbers of lines installed, Nortel had 13.4 percent of the market, Avaya 11.6 percent and Cisco Systems Inc. 8.5 percent, according to Weckel.

It takes a big vendor to keep up with the new features enterprises will seek as traditional phone systems give way to IP telephony and unified communications with its combination of voice, text, videoconferencing, and mobility, Weckel said. There also won't be room for as many companies in the next several years because even as IP telephony spending grows at 40 percent per year, revenue from traditional phone systems is falling as much as 15 percent, creating lackluster growth overall, Prudential's Singh said. Meanwhile, Cisco, which isn't saddled with a legacy equipment business, keeps growing.

Though some observers have downplayed Nortel's unified communications partnership with Microsoft, which also has interoperability relationships with Cisco and other vendors, Singh believes that could be the factor that makes such a deal significant. Nortel and Microsoft are sharing software and doing joint development through a deal announced last year.

"It would be industry-game-changing if you had these three companies lined up on one side with Cisco on the other side," Singh said.