Has Apache lost its way?

Since its inception, the Apache Software Foundation has had a profound impact in shaping the open source movement and the tech industry at large.

Founded by the developers of the Apache HTTP server and incorporated as a nonprofit in 1999, the ASF has served as an incubator and support structure to dozens of projects that range from the modest to the massive. Subversion, OpenOffice, Tomcat, newcomers Cassandra, Lucene, Hadoop -- all have come of age under the aegis of the ASF and its core principles, informally known as "the Apache Way."

But tensions within the ASF and grumbling throughout the open source community have called into question whether the Apache Way is well suited to sponsoring the development of open source projects in today's software world. Changing attitudes toward open source licensing, conflicts with the GPL, concerns about technical innovation under the Way, fallout from the foundation's handling of specific projects in recent years -- the ASF may soon find itself passed over by the kinds of projects that have helped make it such a central fixture in open source, thanks in some measure to the way the new wave of bootstrapped, decentralized projects on GitHub don't require a foundationlike atmosphere to keep them vibrant or relevant.

The Apache Way: The ASF difference maker

Ask most anyone involved with the ASF, "What sets the foundation apart?" and a common answer will likely dominate: "The Apache Way."

The six tenets of the Way form the core philosophy of the Apache Software Foundation. In the foundation's own words, these are: "collaborative software development; commercial-friendly standard license; consistently high-quality software; respectful, honest, technical-based interaction; faithful implementation of standards; security as a mandatory feature."

Ashish Thusoo, co-creator of Hive at Facebook and currently serving on the Project Management Committee (PMC) for Hive at Apache, expresses the Way as "consensus-oriented communities whose aim is to create high-quality software that leads its field."

Thusoo explains that the ASF's approach is closer to mentorship than actual project management. This consists of "coaching new members on how to develop consensus, the various types of mechanisms to use for voting and project governance, making sure that new members of a project come from a breadth of companies in the industry," Thusoo says.

Arun Murthy, chair of the Apache Hadoop PMC, describes the ASF's "mantra" as "community over code, i.e., people are the lifeblood of the ASF."

The Way also emphasizes the practical over the theoretical. For instance, when incubating a project under the ASF, there is a strong emphasis on working code, not just an idea -- and on donating the resulting code and IP to the foundation "without fearing lock-in for itself or for its users," as Apache itself puts it.

This last detail, the fact that Apache as a whole becomes the custodian of the code, is part and parcel of an approach that also includes the ASF's trademark policy, which is designed to prevent Apache-sponsored projects from having their branding diluted. Being protective of patents may seem counterintuitive for open source projects, but Apache and others have argued for the use of trademarks as protection against predatory behavior.

This doesn't mean just defending against packaging copies of OpenOffice with malware. Apache OpenOffice contributor Rob Weir has noted how Tightrope Interactive attempted to file for ownership of the OpenOffice trademark immediately after Oracle announced it was no longer developing the project.

What's still worth asking, though, is whether a project needs to be assigned to a foundation to be kept both alive and out of the hands of corporate meddlers -- or, for that matter, whether an existing foundation is even needed for such a thing. (Monty Widenius created his own foundation to oversee MariaDB, his fork of MySQL.)

No one-way ticket to success

In practice, the Apache Way is not a one-size-fits-all solution for incubating or supporting open source projects. Much of this is due to the ASF's highly laissez-faire approach to the projects that come under its stewardship.

As Thusoo explains, the ASF may step in "if it feels that the project is blatantly violating the Apache Way," but by and large it provides "infrastructure, the legal guidance, and above all coaching and membership. Seldom does it come to micromanagement."

This approach is a two-edged sword. On the one hand, projects are largely left on their own on a technical level. On the other hand, the Apache Way can come off as "a very regimented, planned form of governance," as Brian Proffitt, adjunct instructor of management at University of Notre Dame, puts it.

"This can be a very good thing, since some projects are in need of organization," Proffitt says. "But it can also cause tension, since the rules and regulations of the ASF may rankle those who see it as a bureaucracy."

Joe Brockmeier, Apache CloudStack PMC member, notes that the ASF is not "magic dust you sprinkle on a project for instant success. If the people doing day-to-day development [on the project] aren't good at building community, or if the project just doesn't appeal to a large enough audience, Apache isn't going to make it magically successful."

Here is where the first real test of Apache's future comes to the fore: Can today's climate of bootstrapped, intensely collaborative open source projects stand to benefit from the Apache Way if the added bureaucracy provides no sure path to greater adoption?

The ASF's open source niche

The answer, of course, is that it depends on the project.

"ASF and open source in general are best suited for wide-ranging platform-type technologies," says Hadoop PMC chair Murthy. "These are the foundational elements of a development and infrastructure community. Some of the most successful Apache projects have been foundations or infrastructure."

Rob Davies, currently of Red Hat and a member of the PMC for Apache Camel, Apache ActiveMQ, and Apache ServiceMix projects, had the latter two projects moved under the wings of the ASF in 2005 "because we wanted to build a larger community, and at the time, the ASF was the only main open source community for middleware."

Davies explains the attraction of putting a project under Apache's care: "[The project members] know that a project will not die if a key developer gets run over by a bus or the company that developer works for gets taken over. [But] in reality this means it's hard to start new projects at Apache and grow a diverse community, as open source projects don't typically work that way."

An open source project typically starts because of the efforts of one or two people, Davies asserts, and attracts contributors only after it shows it has legs. To that end, Davies adds, "the ASF is best suited to established projects that want to benefit from wider exposure and attract a greater diversity," as was the case with Hadoop, which was donated by Yahoo. "If you want to start a completely new project, the ASF might not be the first place to start."

Facebook's Thusoo feels the ASF is best for projects "that are interested in developing a broader community that has representation from a number of companies in the industry. For open source projects that focus really on having the controls with a single entity and that perhaps do not encourage a lot of wider industry participation in terms of authoring, ASF is not really a correct vehicle."

Part of the inflexibility hinted at here may arise from the way Apache's licensing conflicts with the GPL, still among the most broadly used licenses for open source software. The conflicts revolve around "patent termination and indemnification provisions," according to the ASF -- in short, some of the very elements that make the Apache license and community what it is.

While the GPL may in fact be falling out of favor, replaced largely by the Apache License, more permissive licenses like the GPL-compatible MIT License are becoming increasingly popular. GitHub CEO Tom Preston-Werner used his recent OSCON keynote to endorse the MIT License for its brevity and permissiveness -- central virtues for many projects on GitHub's rapidly evolving open source ecosystem.

Competition is your responsibility

Mention of GitHub brings to mind another criticism of the Apache Way: It doesn't automatically spur projects to remain technologically competitive.

Consider Apache HTTP Server, the ASF's original 1995 project, and still one of the foundation's flagship projects. Once responsible for powering the overwhelming majority of websites, it's now experiencing increasing competition from the Nginx server. Created in 2008, Nginx already powers around 15 percent of websites (Apache stands at 53 percent, down from over 60 percent since June 2011), in large part because it uses a different architecture that is said to handle high loads better, is far easier to configure, and is offered under the simpler and more liberal BSD license.

The ASF's version-control system Subversion has also seen challenges on the technical side, having taken a major backseat with developers to Git and especially GitHub, at least in some measure because the distributed nature of Git is a better complement to the work habits of modern developers.

While the ASF helped keep these projects alive and well, the need to keep them technologically competitive falls outside its bailiwick. But many argue this is a feature of the ASF, not a bug.

Apache CloudStack PMC member Brockmeier believes the ASF does want its projects to be "technologically competitive and widely used." But "it's not within the scope of the foundation to take responsibility for that. And I doubt Apache would be as popular as it is, were the ASF leadership to try to dictate to projects exactly how they can do that."

Ben Cherian, Chief Strategy Officer at Midokura, network virtualization company that contributes to Apache CloudStack, agrees. "I don't believe it's the Foundation's responsibility to deal with the changing market and evolution of the projects," he says. "It's the responsibility of each project's community to adapt to the changing winds of technology and market pressure. As with every software project, sometimes there are lifespans associated with these projects, and death and rebirth are part of the normal lifecycle of software. There will be some projects that are wild successes and some that flounder. This isn't a failure of the Apache project. It's just a reality of how software is accepted and adopted by the market."

To that end, any project under Apache's sponsorship must realize on its own how to remain competitive and not let the sponsorship of the ASF suffice for that.

Project politics and its discontents

The ASF has taken heat of late based on how specific projects have been handled. Case in point: OpenOffice.org, which was donated to the ASF by Oracle in June 2011.

Four months after OpenOffice.org changed hands, the ASF published a statement to quell fears about the future of the project and to forestall some criticism already thrown its way. The statement claimed "destructive statements have been published by both members of the greater FOSS community and former contributors to the original OpenOffice.org product, suggesting that the project has failed during the 18 weeks since its acceptance into the Apache Incubator. We understand that stakeholders of a project with a 10+-year history -- be they former product managers or casual users -- may be unfamiliar with the Apache Way and question its methods." Apache went on to cite the Subversion and SpamAssassin projects as "proof that the Apache Way works."

Some criticism might have been due to the way the change in ownership of OpenOffice.org apparently delayed the software's release schedule. A beta of version 3.4 had been offered in April 2011, but the full-blown 3.4 release wouldn't come out until May of the following year. (Version 4.0, with major code contributions from IBM, was released in July 2013.)

Part of the delay was due to relicensing the suite under the Apache license, a time-consuming process. But some of it was more directly attributable to OpenOffice.org, regardless of who sponsors it, having a release-when-ready approach rather than dropping new versions on any fixed schedule. By contrast, the OpenOffice.org sister project, LibreOffice, drops new releases every six months under the LGPL.

Another recent issue involved the retirement of one of the Foundation's smaller software projects, the Apache C++ Standard Library project. Active since 2005, this project had seen its last revision in mid-2008. The chair for the project, Jim Jagielski, stepped down at the end of May 2013; in July, the ASF board voted to retire the project to the Apache Attic, a space "to provide process and solutions to make it clear when an Apache project has reached its end of life."