Video codecs: The ugly business behind pretty pictures

Google's settlement with MPEG-LA is a fresh development in a decades-old story of software patents. Will it finally open video codec technology to open source developers?

When Google announced last week it had finally made peace over its VP8 codec with the patent pool MPEG-LA, some declared the company had sold out and was joining the software patent circus. But it's not.

The truth is far more colorful and could presage a big shift in the struggle to control our online viewing and listening habits. As a consequence, powerful interests have quickly stepped in to try to spike VP8's guns before they can be fired.

[ Simon Phipps tells it like it is: Why software patents are evil. | Stay up to date and subscribe to InfoWorld's Open Source newsletter. | Read Bill Snyder's Tech's Bottom Line blog for what the key business trends mean to you. ]

Codec context

The realm of video codecs is complex and infested with acronyms and political intrigue stretching back decades. Even those closest to the situation disagree about both the reality and history of the situation. Here's a condensed overview:

When you view or stream video, you may think it comes as QuickTime, Flash, or even Ogg, but those are just delivery mechanisms. Raw video is an enormous amount of data, and delivering it to you requires data compression. The actual video is encoded in a format created by video compression software, then displayed on your screen after that format is decoded by decompression software.

The codec is the software that performs this processing. The theoretical work behind codecs is exceptionally complex, and there's always a trade-off between maximum compression, processing time, and optimal quality. As a result, a wide range of codec designs exists, and the know-how in implementing them is a valuable commodity.

As early as 1993, it became obvious that format standardization for the data handled by codecs would be needed. International standards bodies ISO and IEC formed an expert working group called the Motion Picture Expert Group -- MPEG -- that has since devised a series of standards for various fields.

The field is heavy with patented techniques. Codec standardization is rooted in the thinking of the telecommunications industry, where it's common for patented techniques to be permitted in standards and then used to derive royalty payments from implementations. To make collection of payments easy, a company called MPEG-LA, LLC (that is, confusingly, entirely unafilliated with MPEG) formed to manage a patent pool acting on the behalf of many -- but not all -- the patent holders contributing to MPEG standards. All use of standards, such as H.264, is likely to necessitate a license from MPEG-LA -- even free uses.

That arrangement worked well in the old, control-point-based world where software was created by corporations. But the new meshed society and the techniques it necessitates, such as open source, don't blend well with a world where each new use demands permission first. Elements that need permission in advance -- control points -- are anathema to the Internet. They get treated as damage, and clever minds seek ways to work around them.

The rise of open codecs

Once it was clear that the open Web demanded open codecs to process open media formats, clever minds started working around the problems. The science of codecs is well documented, but using any of the well-known techniques was highly likely to result in breaching some software patent associated with MPEG-LA. It wasn't so simple as just modifying an MPEG-derived standard to route around patents. Those standards created such a patent thicket that any new work using the same math was almost certain to encroach on a patent portfolio somewhere.

Creating open codecs demanded new thinking. Fortunately, there were commercial concerns working on alternative codec ideas. In particular, a company called On2 created a family of codecs that used ideas that were outside of the MPEG patent thicket and filed its own patents to prevent encroachment. In 2001, it released codec technology called VP3 as open source, protected by its own set of patents. That technology formed the base of what would become Theora. On2 continued its work to produce a series of niche codec technologies until it was acquired by Google in 2010.

1 2 Page 1
From CIO: 8 Free Online Courses to Grow Your Tech Skills
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.