Previously, only large retailers such as Amazon and Wal-Mart or portal providers such as Google and Yahoo have had access to such large data sets. So far these companies have held their cards close to their vests, and for good reason. Their customer data is a key component of their competitive advantage. Now Twitter and others -- particularly, social networks such as Facebook and MySpace -- are banking that smaller vendors are willing to pay to level the playing field, and they're developing data platforms to make that happen.
Selling out users' privacy
Selling access to raw user data is not without peril, however. Privacy concerns are growing among users of online services. Earlier this week, regulators from 10 countries sent a letter to Google CEO Eric Schmidt urging his company to collect only a minimum of personally identifying information and to ensure that the data was protected by adequate security measures. Failure to do so, the letter implied, could threaten Google's ability to legally conduct business in the co-signers' countries.
Data platforms will likely make enforcing security more difficult. Twitter believes its data store contains information valuable to marketers, such as how much buzz a new movie premiere is getting or how successful a viral ad campaign has been. But Twitter users who assume nobody is listening except a few friends and family might also unwittingly tweet information of a more sensitive nature -- their children's names and where they go to school, for example. Recently, an activist site called PleaseRobMe demonstrated how easy it was to scan for tweets announcing that Twitter users had left their homes. Developers will need to tread carefully to avoid potential liability for similar searches.
Equally worrisome to many consumers is the ability to cross-reference multiple databases to produce a much more thorough profile of an individual than any one source could provide. Even if a data platform's API makes only a limited amount of information available, there's little to prevent developers from querying multiple sources and matching corresponding fields. If these and similar privacy concerns cause users to quit online services en masse, plans to launch new data platforms will have backfired.
Where do developers fit in?
Data platforms present other concerns for developers, however. Even more than mobile platforms such as Apple's iPhone, a data platform like Twitter's is a walled garden. If Twitter cuts off a developer's access to its data sources for any reason, that developer's business is sunk. Rather than risking public exposure, then, the safer bet for independent code shops may be to develop "cloud middleware" around data platforms, including tools that facilitate unique kinds of analysis not foreseen by the original data vendors.
One problem with this approach, however, is that it inevitably pits developers in competition with their data platform vendors. For example, Twitter's "analytical products" could overlap those built by its developer community, and Twitter may be in a better position to reach potential customers.
But this risk might not be as great as it sounds. Like many companies that have only recently begun reaching out to developers, Twitter's relationship with its developer community is evolving. For example, rather than create its own iPhone client, earlier this month Twitter acquired Tweetie, a leading independent iPhone app for the platform. Just as Microsoft has purchased many startups over the years, being acquired may prove to be a viable strategy for developers offering novel products built on data platforms. The field of data mining social networks is yet very new; it's a good bet there's money to be made helping companies sort it all out.
This article, "Twitter and the rise of data platforms," originally appeared at InfoWorld.com. Read more of Neil McAllister's Fatal Exception blog and follow the latest news in software development at InfoWorld.com.