Retirement of the rambunctious Ballmer leaves the world's largest software company in a succession crisis amid the worst slump in history for the personal-computer industry it helped create.
The boss of Nokia has been tipped as one of the favourites to run Microsoft due to his knowledge of the devices and services market.
Stephen Elop, who managed Microsoft Office before becoming chief executive of Nokia, is a strong candidate to succeed Ballmer, who revealed his retirement on Friday. During his troubled 13-year reign the company's share price has fallen 40%.
Microsoft's dominant position as the supplier of software for PCs has been steadily undermined by the rise of Apple and Google smartphones and tablets.
Click to enlarge
"There is never a perfect time for this type of transition, but now is the right time," said Ballmer, who has been widely criticised for his failure to catch the rising mobile computing tide. "My original thoughts on timing would have had my retirement happen in the middle of our company's transformation to a devices and services company.
"We need a CEO who will be here longer term for this new direction."
Investors cheered the news of Ballmer's departure, sending Microsoft shares up seven percent to $34.75 on the Nasdaq stockmarket.
Ballmer, now 57, took over as CEO in 2000 from co-founder Bill Gates, a classmate and friend from their brief days at Harvard University in the 1970s.
He took over when Microsoft was the undisputed tech sector leader and the world's largest company in market value. But in recent years Microsoft has struggled as consumers move from PCs to mobile devices.
Microsoft's attempt to crack the growing market for tablet computers has failed to catch the public imagination. The Surface project has led to a $900 million writedown of inventory as consumers continued to buy rival products like Apple's iPad or Samsung's Galaxy tabs.
Surface shipped just 900,000 units in the March quarter, according to IDC. That contrasted with the 19.5 million iPads sold by Apple in the same quarter, and Samsung's sale of 8.8 million Android tablets.
Meanwhile, Microsoft's flagship business has been struggling as the market for PCs falters.
Whoever takes over Microsoft leadership will have a big job ahead in forcing the company to swallow a business strategy that wraps itself around devices and services.
"Microsoft has clearly been on the wrong track for a long time," said Roger Kay, an analyst at Endpoint Technologies Associates. "Microsoft missed the shift to high mobility e 2ntirely."
Who will succeed Big Steve? One strong runner could be Tony Bates, currently VP of business development, involving corporate strategy, customers and developer support.
He joined Microsoft only in 2011 after Ballmer bought Skype, where he was then CEO. Observers say he was very successful at turning Skype into a profitable enterprise and has been doing well in his key role at Microsoft. At 49, he appears to fill the requirement for "a CEO who will be here longer term for this new direction".
Ballmer won't be heading out with a big retirement package as Microsoft doesn't offer "special retirement programs" or severance pay for its executives, but he will leave an extremely rich man. He owns more than 333 million share