Interview: Torvalds gets down to the kernel

Linux creator talks up Version 2.6

Life seems to be getting even better for Linus Torvalds. Having just taken a leave from his "day job'' at chip designer Transmeta, Torvalds is about to begin a new one at the Open Source Development Lab (OSDL) where he will be able to focus even more on completing the upcoming 2.6 version of the Linux kernel. This version is seen by most observers as critical for Linux to succeed in the enterprise because it is being refined to work more capably on large, multiprocessor systems, especially those using nonuniform memory access, and given significant improvements in scalability and more reliable performance.

At this week's CA World 2003 show in Las Vegas, Torvalds sat down with InfoWorld Editor at Large Ed Scannell and Staff Writer Brian Fonseca to talk about the completion this past weekend of Version 2.6's first test release; what will keep him up at night worrying about that test release; his thoughts about the impending SCO-IBM suit and possible impact on Linux development; and something of a wish list of technologies to be included in upcoming versions of the open source operating system.

InfoWorld: How difficult a development effort was Version 2.6 compared with previous efforts? Are they getting increasingly difficult as you include more complex features?

Torvalds: This was fairly comparable to Version 2.4. It is hard to tell, really. I actually put out the first test release on Sunday evening [July 13], and that is the last beta of the program. With 2.4, that testing took about six or seven months to complete. But I think we are actually in better shape this time; we are aiming for three months but we will see what happens. It has gotten slightly more complicated, mainly because there are now more people involved. And what I mean is that doing a release always means synchronizing. And when you have more people to synchronize it takes longer because you get more issues that come up. But on the whole it was not that different from 2.4.

InfoWorld: When you say more people were involved, what people were involved this time who have not typically been so before?

Torvalds: In previous efforts, and I mean looking back at [Version] 2.0 or the precommercial days, most of the people involved tended to be literally engineers along with some involvement of the early Linux companies. But these days you still have all the engineers, but now you also have a number of larger companies. They interact with the kernel through their own engineers but they have their own issues that they want to make sure are sorted out and things like that. So that is really where the added people came from.

InfoWorld: In joining OSDL do you think you will have more technical resources at your disposal, thereby making the development efforts easier and the delivery of future kernels faster?

Torvalds: Well, long before I joined they had their own test projects. They had their own server farms where they developed not just their own hardware tests but literally had database loads to test Linux on their own load so could verify both performance and how they were operating. That has been all independent of me. What OSDL gives me is basically a little space where I can do Linux full time. And I don’t rely on OSDL itself. I consider that very independent of what I do. It is important, but it is not like my moving there changes much.

InfoWorld: How many test versions do you anticipate putting out before the code goes to GA (general availability)?

Torvalds: It will depend. Last time around we had 12 test versions and then one final one. We wanted to avoid the embarrassment of having to deal with last-minute changes. And so like every other week there was a new version. But you don't want to make them too often because you want to have people actually test the test versions and really beat on them to figure out all the problems before you make the next one. Developers obviously get daily snapshots they work on, but it depends a lot on what the feedback is. It also depends on the [Linux] companies rolling out these test versions, because one thing we have found is that it is important for companies like SuSE and Red Hat to roll them out to people who are not afraid to get involved in testing pre-releases and who are normally involved in a development process. But also it is surprising how much new stuff users find that developers never do. You put a copy in front of a normal user and they find all these bugs that you would think developers would find. The real users and developers are completely different species as far as I am concerned.

InfoWorld: What areas of the kernel are you expecting developers and users to really pound on? Any one area you have concerns about?

Torvalds: I am actually less concerned this time. That is one of the reasons I think 2.6 looks better than 2.4 did at this point. We had a much more stable [VM] Virtual Machine system. We had a much more stable file system infrastructure. We did a lot of changes between 2.2 and 2.4, and there are lots of changes between 2.4 and 2.6 as well but they tend to be not as quite as central as some other 2.4 changes. What I end up being most nervous about are device drivers. Drivers are nasty in the sense that you can't think about them that well. They depend too much on actual hardware behavior. It is hard to analyze their behavior, so that is where you want to have a lot of users just pounding on the system. So device drivers end up being my main worry, but that is always true, it is not a new issue.

InfoWorld: Has the specter of the IBM-SCO  suit altered the way you approach your work on the kernel?

Torvalds: From a technical angle not at all. I mean, there has been some worry about the IP [intellectual property] issues, especially early on. SCO was running around talking about IP, so people ended up looking through code for anything that looked like it was SCO's. That all kind of died down when SCO started telling us about more of the details and it became clearer that the IP issue wasn't really at the center of it. So for the kernel we didn’t need to worry that much.

InfoWorld: What about legal concerns for the larger Linux community?

Torvalds: Well you would be crazy not to have concerns about lawsuits going forward because, first, it is a lawsuit and it could drag on forever. So I am concerned but at the same time I am personally convinced that the Linux development model itself, which is all about openness, means you have very good visibility into exactly what is happening. We don't have a paper trail, but we have an incredibly detailed electronic trail. But if something turns out to have been improper, which I think will be fairly unlikely, we already have a better system in place than any proprietary system has ever had in terms of being able to figure out what happened in a certain case. I think we already have a very good development model for handling IP issues. Besides, if you read the SCO-IBM lawsuit it isn't abut IP issues at all. It's very clearly about contracts. All the IP noises were just that. They had nothing to do with the lawsuit.

InfoWorld: With 2.6 you focused on both new server capabilities as well as embedded devices. What about things for the desktop?

Torvalds: Well we spent a lot of time trying to make the VM system in particular very predictable. Even under heavy workloads people expect [desktop] systems to be very smooth. It is OK if it slows down, people kind of expect it to slow down when you have a big compile going on in the background or something like that. But you do not want to have multisecond pauses because a compile took away all the disk I/O or throughput. The biggest desktop feature, apart from the drivers, has been the VM system, so you get the behavior that people expect. That sounds like a simple thing, but it actually ended up being a pretty hard problem to make sure you always have the throughput you need to load applications smoothly even when something else is going on in the background. With a lot of server applications, you don't see that because they tend to have fairly well-behaved access patterns so they don't run only one thing. But on the desktop where you have people who might be also running it as a server and they start up something like Mozilla, which is a huge behemoth, they expect to get good interaction -- and at the same time there is a lot of disk I/O going on and a lot of other processes may be bogging down the CPU. So yes that is something we spent a lot of time on.

InfoWorld: I raise the issue of desktop capabilities because Microsoft is moving steadily forward in terms of more tightly tying Windows Server 2003 and their Office System 2003together with Sharepoint Portal Serverliving in the middle. Will we see a more compelling story from Linux distributors and their partners in this area?

Torvalds: I don't see it so much as tying them more together. Maybe some companies want to sell a solution that includes both servers and clients, and I am sure that is what [a company] like Computer Associates, for instance, is looking for. Linux has always been both server and desktop. I have always used Linux as a desktop. I mean I use Linux as a server too, but when I do I don't even think about it because usually I have an IS department set up that server for me. My day-to-day work since day one has always been about the desktop. [Microsoft] has had a completely separate user base for the desktop and another one for the server. Linux has never had that because it has been unified from day one. So we have never had to deal with the issues you are bringing up. Besides it goes beyond server and desktop use. You still see that same mentality from Microsoft when it comes to embedded devices. They still do not have that one OS to rule-them-all sort of thing. If you look at Linux, when you get a Linux PDA it is the same basic OS as when you get one of the supercomputers at the top of the supercomputer list. It is literally the same code running on anything from something you could hold in your hand like a cell phone to a system with hundreds or even thousands of CPUs.

InfoWorld: What are the three or four most interesting technologies, either existing or not yet existing, you would like to see included in Linux?

Torvalds: There are a number of areas where it all works, but it may not be as integrated as it should be for us. In the server space there’s distributed file system infrastructure. On the desktop space, it’s, I think, 3-D acceleration. It works on a lot of the common graphics cards, and the ones it doesn’t work on tend to be because the 3-D companies do not give enough specs. More importantly, it’s recently that we have to have several generations of technology. I know that over the next four or five years we’ll have to re-architect something. That’s going to happen. It’s always been daunting to put in new technology, but it’s more interesting than scary. We’ll see about the integration of wireless. Not wireless in the 802.11 sense, but the seamless changing from say something like DSM [direct-sequence modulation] integrated into 802.11 into wired. You can make it feel that way, but again it’s the same thing as the 3-D. What you need is healthy generations of loyal users.

Copyright © 2003 IDG Communications, Inc.

How to choose a low-code development platform