Showdown at the VM corral

The CPAN (Comprehensive Perl Archive Network) is the backbone of the Perl community. It includes hundreds of modules that extend Perl in every imaginable direction: networking, directory services, XML processing, statistics, you name it. Before writing a substantial piece of new code, a seasoned Perl programmer will check CPAN to see whether an existing module can meet the need. The same holds true for users of PHP, Python, Ruby, and other dynamic languages.

As these language communities have evolved in parallel, each has accumulated its own library of supporting modules. Sometimes a module exists in order to provide an easy-to-use interface to a low-level system service -- network sockets, for example. In that case, all the languages talk to a common piece of the operating system's C or C++ code, but each language has to create its own bindings to that code. Alternatively, a module may directly implement a higher-level service such as the HTTP protocol. Here too, though, each dynamic language has to create its own version of the module.

As you can imagine, this process entails a lot of duplicate effort. Perl needs XML, LDAP, database, and other modules. Versions of the same modules are also needed for Python, PHP, Ruby, and so on. The ongoing reinvention of all these wheels always struck me as wasteful. Energy spent wrapping existing services is energy diverted from writing applications and creating new language-independent services.

Despite this inefficiency, things have worked well enough so far. Each of the dynamic languages I've mentioned occupies an important niche in the IT ecosystem. Each language appeals to a particular audience of developers and makes those developers highly productive when solving a variety of problems.

Two factors largely account for that productivity. The first is cultural. All of these languages evolved on the Internet and are shepherded by pragmatic open source developers who know how to collaborate to get things done. The second factor is technical. At the core of each of these languages there is a VM that manages memory automatically, just as the JVM does for Java programmers and the CLR (Common Language Runtime) does for .Net programmers. Managed code isn't a panacea, but it's the dominant reason why programming is getting easier and safer.

Core operating-system services aren't yet built with managed code, to be sure, and they probably won't be for a few years. Yet we already rely on managed interfaces to those services. Some developers acquire those interfaces by way of dynamic languages, others get them courtesy of Java and .Net. These two camps have historically regarded one another with suspicion, but the inexorable logic of managed code creates a common bond.

As operating systems consolidate around managed interfaces, they'll choose the Java and .Net VMs, not the Perl, Python, or PHP VMs. But the agility of the dynamic languages and the collaborative energy of their open source communities are two of the pistons that crank the engine of progress. These worlds need to converge -- and at the O'Reilly Open Source Conference there was new evidence that they will. Jim Hugunin, father of Jython (Python for JVM), made a pair of dramatic announcements. He has released the first version of IronPython (Python for CLR/Mono). And by the time you read this, he'll have started his new job at Microsoft. Good hire!