"Until they have a material loss or one of their peers has an accidental information disclosure, it won't hit home," Gordon said.
The level of concern about leaks due to government spying also hinges on the type, size and industry of a company. "I'm not aware of any instances of this happening to a mid-size wholesale company like us," said Hal Greene, vice president of IS at Composites One, a distributor of plastic and glass products in North America that uses Google Apps.
But Paul Grewal, CEO of Sage Human Capital in San Bruno, California, an executive search and recruitment firm, worries about a nightmare scenario in which government snooping on his company's data could result in a leak. "We are definitely concerned. It creates a liability," he said.
A leak could be extremely harmful to the candidates seeking jobs, their current employers and the companies that are hiring. "Our data is extremely confidential," he said.
The company would find itself potentially liable for breaching confidentiality agreements with clients, and it would also see a major trust breakdown.
Sage Human Capital deployed a business intelligence tool from Jaspersoft on the Amazon EC2 cloud service about six months ago to give clients a granular analytics view of how a search is going. "The reason we went to the cloud was ease of implementation and deployment," Grewal said, adding he doesn't plan on rolling back that decision.
He's confident Amazon will provide top-notch encryption and security, but he's also aware that "NSA has a heavy hand and can make offers people can't refuse."
Analysts say CIOs need to weigh risks and rewards and adhere to best practices, whether the government is snooping on their systems or not.
"The answer to whether the risks outweigh the benefits will be different for different companies and CIOs," said Scott Strawn, an IDC analyst.
"Our advice to organizations is to recognize the sensitivity of their data, and if it's highly sensitive, they should take very careful precautions about where they put it, and place heroic levels of protection around it," Gartner's Heiser said.
For starters, companies need to decide which applications and data can be put in a public cloud service, which can go in a private cloud service and which should remain behind the on premises firewall.
"You must be observant and think about data integrity before putting sensitive, mission-critical information in the cloud," said Lars-Göran Eklöf, CIO at construction company Lindab in Sweden.
"We only use cloud services on a limited basis, and the information stored in the cloud, including sales statistics, doesn't have a very high security classification," Eklöf said.
Criteria that CIOs can use to calculate appropriate levels of security include how critical data is, and what the applicable laws and regulations for privacy and data security in their country and for their industry are.
IRB Services, an Ontario, Canada-based company which conducts independent reviews of clinical research involving humans, choose a software-as-a-service product from Intralinks for secure collaboration on review files because Intralinks can house the data outside of the U.S.
IRB Services customers in Europe have for some time not wanted their data stored in the U.S., according to Simon Corman, the company's director of business operations. Before the NSA scandal, "we were just getting that question from compliance groups. Now we're getting it more from an operational level," he said.
IRB customers have always been concerned about the privacy of their data but the NSA controversy has "absolutely amplified the issue," Corman said.