Why we need even more programming languages

Upgrading existing, popular languages to support new features is a lot harder than you might think

Whenever a new programming language is announced, a certain segment of the developer population always rolls its eyes and groans that we have quite enough to choose from already, thank you very much.

As a case in point, take Dart, the language Google hopes will replace JavaScript. Was it really necessary to create a whole new client-side language for the Web? Superficially, Dart doesn't seem like much more than JavaScript with a few Java-like features mixed in. It even compiles to JavaScript (and not very efficiently). Wouldn't it have been better to work to improve the JavaScript we already have?

HTML5 Deep Dive
[ Neil McAllister reveals the ugly truth behind programmer hiring quizzes . | Speaking of quizzes, see if you can pass InfoWorld's programming IQ test, round 1 , and programming IQ test, round 2 . | Get software development news and insights from InfoWorld's Developer World newsletter . ]

To its credit, Google says it's doing just that, in tandem with its Dart effort. But once a language reaches a certain tipping point of popularity, overhauling it to include support for new features, paradigms, and patterns is easier said than done. In fact, judging by the past ten years or so, it can be very, very difficult.

The PHP 6 debacle

Take PHP, for example. The next version of the popular Web applications language shipped its second release candidate this week, and the final build is expected to arrive early next year. This release won't be the long-awaited PHP 6, however. Instead, it will be a far less ambitious revision designated PHP 5.4.

Doubtless that's a disappointment to developers who have been anticipating PHP 6 since the project launched in October 2005. But at this point, even if a release dubbed PHP 6 does eventually appear, it will bear little resemblance to the version that's borne the designation so far. PHP creator Rasmus Lerdorf officially shelved the PHP 6 project in March 2010 after almost five years of fruitless labor, in favor of refocusing on a formal 5.4 release.

Some of the reasons for the PHP 6 effort's failure were technical. The primary focus of the project was retooling PHP to include native support for Unicode. That wouldn't be limited to strings, either; in PHP 6, developers would be able to specify variable names, function names, and other identifiers using any Unicode script, including multibyte encodings such as Chinese and Hindi. As the years rolled on and hidden gotchas began to surface, however, it became clear that the PHP developers had bitten off more than they could chew.

It didn't help that as an open source project, PHP development is largely a volunteer effort. According to PHP contributor Andrei Zmievski, relatively few developers really understood the Unicode push and were committed to making it happen. It was hard to get excited about rewriting lots of working code to support Unicode, and enthusiasm for the project waned. By the time PHP 6 was abandoned in 2010, Lerdorf observed that PHP development "hasn't been fun for quite a while."

Languages move forward, but slowly

Personally, I'm no fan of PHP. I've long held it's a poster child for bad language design, so it doesn't surprise me to learn that evolving it into something better is a Sisyphean task. But I shouldn't single out PHP here. In fact, many of the more popular languages have struggled to move forward with major new versions in recent years.

The best example is surely Perl, whose community has been working on version 6 continuously since 2000. In 2008, the Perl community chastised me for suggesting that Perl 6 was "vaporware." It exists, they insisted, and you can use it today. But while that may be technically true, whether Perl 6 will ever be mature enough for production use remains an open question. Although several implementations are available, and some are even fairly stable, so far none of them supports all of the features of the Perl 6 specification.

The Python community has had better luck implementing its language. The most recent version, Python 3, was released in 2008, after about three years of development. Three years later, however, adoption remains slow. Python 3 took the radical step of breaking backward compatibility with earlier versions, and a number of popular Python libraries and frameworks (including Django) have yet to catch up. As a result, many Python developers still cling to version 2.x, particularly for Web work, and widespread migration to Python 3 is expected to take several more years.

These kinds of difficulties aren't limited to scripting languages, either. The Java community has long clamored for significant language updates. Before Java SE 7 shipped earlier this year, it had been five years since the last major release. But while Java SE 7 was originally expected to include much-requested capabilities such as lambda expressions and a module system, Oracle has since delayed those features until Java SE 8 at the earliest.

Darting ahead of JavaScript

All of this brings us back to Dart and why Google chose to invent its own language rather than working to improve JavaScript. JavaScript is based on the ECMAScript standard. Before the publication of the ECMAScript 5 specification in 2009, ECMAScript had remained unchanged for a full decade -- it made the Perl community look like a hotbed of activity!

Not that the ECMAScript working group had been idle throughout those years. Work on ECMAScript 4 began in 1999, but soon foundered and went on hiatus. The committee reconvened in 2003, but still the various stakeholders couldn't agree. Around and around they went for another five years, until -- much like PHP 6 -- the ECMAScript 4 effort was officially abandoned in 2008, in favor of a less-contentious specification that became ECMAScript 3.1.

The lesson from all of these examples is clear: Programming languages move slowly, and the more popular a language is, the slower it moves. It is far, far easier to create a new language from whole cloth than it is to convince the existing user base of a popular language to accept radical changes.

So don't knock Google for the work it's done on Dart. If the Web development community responds well to Dart, perhaps it will eventually replace JavaScript. On the other hand, it's possible that by creating a working prototype of Dart, Google will encourage the ECMAScript working group to evolve JavaScript more quickly. Either way, efforts like Dart are an important engine of progress for the software development community and a hedge against stagnation. Remember that the next time you ask whether we really need another new programming language.

This article, "Why we need even more programming languages," originally appeared at InfoWorld.com. Read more of Neil McAllister's Fatal Exception blog and follow the latest news in programming at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.

Copyright © 2011 IDG Communications, Inc.