MySpace announces plans to open source Qizmt

The open source Qizmt project and the CodePlex Foundation could benefit by working together

MySpace announced that it's open sourcing Qizmt, a MapReduce framework used by the MySpace Data Mining team. Unlike other leading MapReduce frameworks that are typically implemented in C++ or Java, Qizmt was developed using C#.Net. MySpace's chief operating officer Mike Jones writes:

This extends the rapid development nature of the .NET environment to the world of large scale data crunching and enables .NET developers to easily leverage their skill set to write MapReduce functions. Not only is Qizmt easy to use, but based on our internal benchmarks, we have shown its processing speeds to be competitive with the leading MapReduce open source projects on a lesser number of cores.

Count me surprised by the claims that Qizmt can perform comparably with open source MapReduce projects, even while using fewer processing cores. I'd love to hear more about the performance benchmarks. But that's another story.

[ Stay up to speed with the open source community via InfoWorld's Technology: Open Source newsletter. ]

This statement from Jones is why the story caught my attention:

Many companies leverage Microsoft technologies in their BI platforms and Qizmt is a natural extension to these platforms. As companies deal with continued data growth and deeper analytics needs, Qizmt becomes a more integral part of BI both from a data processing and a data mining perspective.

I couldn't agree more. With the number of companies and ISVs that rely on .Net, Qizmt could become an important technology for .Net ISVs and customers. This is where CodePlex.org steps in. By helping Microsoft ISVs and customers get comfortable with contributing their IP into Qizmt, CodePlex.org could help Qizmt mature a lot faster than is likely with MySpace simply hosting the project on Google Code, as is the case today.

For appearance's sake, CodePlex.org may not want Qizmt as the first project it shepherds. Qizmt's strong .Net and Microsoft linkage will not go unnoticed by those of us watching how the CodePlex Foundation will shift from vision to execution. But here's an important fact: We watchers don't have skin in the CodePlex Foundation game and likely won't for some time, if ever. The CodePlex Foundation should start with an audience that could have skin in the game, namely .Net users. As the Foundation demonstrates its independence and value to the community, the Microsoft/.Net linkage will dissipate. But to get there, the CodePlex Foundation needs to show value to developers and to projects soon.

What do you think?

Follow me on Twitter: SavioRodrigues.

p.s.: I should state: "The postings on this site are my own and don't necessarily represent IBM's positions, strategies, or opinions."

Related:

Copyright © 2009 IDG Communications, Inc.

How to choose a low-code development platform