Another question concerns how Ballmer's successor handles the company's Dynamics ERP (enterprise resource planning) and CRM (customer relationship management) software business, which has been built up through a number of acquisitions.
Ballmer carved out a special place for Dynamics in his reorganization plan, saying Microsoft would "keep Dynamics separate as it continues to need special focus and represents significant opportunity." He also kept Kirill Tatarinov in place as head of Dynamics.
While Microsoft lags behind the likes of Oracle and SAP in enterprise applications, analysts see Dynamics as a conduit of sorts for selling a wide array of other Microsoft services into enterprises. Given the strategic nature of ERP and CRM systems to a business, it also gives sales representatives an opportunity to speak with CEOs, chief marketing officers and chief financial officers, as opposed to IT managers.
"Dynamics is key to Microsoft's enterprise cred," said analyst Ray Wang, CEO of Constellation Research. "I think that it was a visionary move to bring Dynamics in when he did."
However, Dynamics didn't start becoming well-integrated and aligned with other Microsoft products such as SharePoint, SQL Server and Azure until more recently, he said. "If it had happened earlier, they'd be further ahead."
There has often been talk of Microsoft selling off the Dynamics business. "I think that's up in the air," Wang said. "Dynamics set up as a stand-alone unit could be possible given how the company is currently structured. The CRM side would go to Office and the ERP stuff would be spun out."
At any rate, Ballmer's exit doesn't mix well with the just-announced, massive reorganization and new strategy shift toward devices and services, especially when the expectation had been for him to be on board for several years at least executing on the plan, Cearley said. "This creates additional uncertainty in the market."
Even if Ballmer's replacement sticks with the current plan, it's to be expected that this person will put his or her own imprint on the company's operations and strategies, so there will be some changes, even if they're not radical, he said.
For example, it's unlikely that the board would pick someone to come in and do a 180-degree turn, such as breaking up Microsoft into more autonomous businesses. After all, Ballmer will remain with the company and participate in the selection of his replacement. Plus, the board of directors backs the current direction of the company, Cearley said. "There is no repudiation of the current Microsoft strategy nor of the things Steve was trying to do," he said.
However, all bets are off looking ahead three or four years from now, so CIOs and enterprise IT decision makers need to keep an eye on how things develop once Ballmer steps down. "We see no big change in the near term, but longer term there are more questions about the strategy," Cearley said.
Ballmer's departure will be a historic turning point for Microsoft. Ballmer, who is 57 and joined Microsoft in 1980, has been CEO since 2000. In a statement, he said the decision to step down wasn't easy, but that he believes it is the right one.
In particular, he's convinced Microsoft should have a CEO who is on board for the entire "transformation" process set off by the reorganization announced last month.
"Our new senior leadership team is amazing. The strategy we have generated is first class. Our new organization, which is centered on functions and engineering areas, is right for the opportunities and challenges ahead," Ballmer wrote.
Wall Street's response has been enthusiastic, with the share price up almost 7 percent in mid-afternoon trading.
Now the attention turns to his successor, Cearley said. "Steve Ballmer leaving Microsoft is a major event, because he has been such a strong force in the company for many years, but an even bigger watershed event will be the naming of a successor," he said. "That'll signal more clearly what future direction Microsoft will take."
(Chris Kanaracus in Boston contributed to this report.)