Programming Java threads in the real world, Part 1

A Java programmer's guide to threading architectures

All Java programs other than simple console-based applications are multithreaded, whether you like it or not. The problem is that the Abstract Windowing Toolkit (AWT) processes operating system (OS) events on its own thread, so your listener methods actually run on the AWT thread. These same listener methods typically access objects that are also accessed from the main thread. It may be tempting, at this point, to bury your head in the sand and pretend you don't have to worry about threading issues, but you can't usually get away with it. And, unfortunately, virtually none of the books on Java addresses threading issues in sufficient depth. (For a list of helpful books on the topic, see Resources.)

This article is the first in a series that will present real-world solutions to the problems of programming Java in a multithreaded environment. It's geared to Java programmers who understand the language-level stuff (the synchronized keyword and the various facilities of the Thread class), but want to learn how to use these language features effectively.

Platform dependence

Unfortunately, Java's promise of platform independence falls flat on its face in the threads arena. Though it's possible to write a platform-independent multithreaded Java program, you have to do it with your eyes open. This isn't really Java's fault; it's almost impossible to write a truly platform-independent threading system. (Doug Schmidt's ACE [Adaptive Communication Environment] framework is a good, though complex, attempt. See Resources for a link to his program.) So, before I can talk about hard-core Java-programming issues in subsequent installments, I have to discuss the difficulties introduced by the platforms on which the Java virtual machine (JVM) might run.

Atomic energy

The first OS-level concept that's important to understand is atomicity. An atomic operation cannot be interrupted by another thread. Java does define at least a few atomic operations. In particular, assignment to variables of any type except long or double is atomic. You don't have to worry about a thread preempting a method in the middle of the assignment. In practice, this means that you never have to synchronize a method that does nothing but return the value of (or assign a value to) a boolean or int instance variable. Similarly, a method that did a lot of computation using only local variables and arguments, and which assigned the results of that computation to an instance variable as the last thing it did, would not have to be synchronized. For example:

class some_class
    int some_field;
    void f( some_class arg ) // deliberately not synchronized
        // Do lots of stuff here that uses local variables
        // and method arguments, but does not access
        // any fields of the class (or call any methods
        // that access any fields of the class).
        // ...
        some_field = new_value;     // do this last.

On the other hand, when executing x=++y or x+=y, you could be preempted after the increment but before the assignment. To get atomicity in this situation, you'll need to use the keyword synchronized.

All this is important because the overhead of synchronization can be nontrivial, and can vary from OS to OS. The following program demonstrates the problem. Each loop repetitively calls a method that performs the same operations, but one of the methods (locking()) is synchronized and the other (not_locking()) isn't. Using the JDK "performance-pack" VM running under Windows NT 4, the program reports a 1.2-second difference in runtime between the two loops, or about 1.2 microseconds per call. This difference may not seem like much, but it represent a 7.25-percent increase in calling time. Of course, the percentage increase falls off as the method does more work, but a significant number of methods -- in my programs, at least -- are only a few lines of code.

import java.util.*;
class synch
   synchronized int locking     (int a, int b){return a + b;}
    int              not_locking (int a, int b){return a + b;}

    private static final int ITERATIONS = 1000000;
    static public void main(String[] args)
        synch tester = new synch();
        double start = new Date().getTime();
      for(long i = ITERATIONS; --i >= 0 ;)
        double end = new Date().getTime();
        double locking_time = end - start;
        start = new Date().getTime();
      for(long i = ITERATIONS; --i >= 0 ;)
        end = new Date().getTime();
        double not_locking_time = end - start;
        double time_in_synchronization = locking_time - not_locking_time;
        System.out.println( "Time lost to synchronization (millis.): "
                        + time_in_synchronization );
        System.out.println( "Locking overhead per call: "
                        + (time_in_synchronization / ITERATIONS) );
            not_locking_time/locking_time * 100.0 + "% increase" );

Though the HotSpot VM is supposed to address the synchronization-overhead problem, HotSpot isn't a freebee -- you have to buy it. Unless you license and ship HotSpot with your app, there's no telling what VM will be on the target platform, and of course you want as little as possible of the execution speed of your program to be dependent on the VM that's executing it. Even if deadlock problems (which I'll discuss in the next installment of this series) didn't exist, the notion that you should "synchronize everything" is just plain wrong-headed.

Concurrency versus parallelism

The next OS-related issue (and the main problem when it comes to writing platform-independent Java) has to do with the notions of concurrency and parallelism. Concurrent multithreading systems give the appearance of several tasks executing at once, but these tasks are actually split up into chunks that share the processor with chunks from other tasks. The following figure illustrates the issues. In parallel systems, two tasks are actually performed simultaneously. Parallelism requires a multiple-CPU system.

Unless you're spending a lot of time blocked, waiting for I/O operations to complete, a program that uses multiple concurrent threads will often run slower than an equivalent single-threaded program, although it will often be better organized than the equivalent single-thread version. A program that uses multiple threads running in parallel on multiple processors will run much faster.

Though Java permits threading to be implemented entirely in the VM, at least in theory, this approach would preclude any parallelism in your application. If no operating-system-level threads were used, the OS would look at the VM instance as a single-threaded application, which would most likely be scheduled to a single processor. The net result would be that no two Java threads running under the same VM instance would ever run in parallel, even if you had multiple CPUs and your VM was the only active process. Two instances of the VM running separate applications could run in parallel, of course, but I want to do better than that. To get parallelism, the VM must map Java threads through to OS threads; so, you can't afford to ignore the differences between the various threading models if platform independence is important.

Get your priorities straight

I'll demonstrate the ways the issues I just discussed can impact your programs by comparing two operating systems: Solaris and Windows NT.

Java, in theory at least, provides ten priority levels for threads. (If two or more threads are both waiting to run, the one with the highest priority level will execute.) In Solaris, which supports 231 priority levels, this is no problem (though Solaris priorities can be tricky to use -- more on this in a moment). NT, on the other hand, has seven priority levels available, and these have to be mapped into Java's ten. This mapping is undefined, so lots of possibilities present themselves. (For example, Java priority levels 1 and 2 might both map to NT priority level 1, and Java priority levels 8, 9, and 10 might all map to NT level 7.)

NT's paucity of priority levels is a problem if you want to use priority to control scheduling. Things are made even more complicated by the fact that priority levels aren't fixed. NT provides a mechanism called priority boosting, which you can turn off with a C system call, but not from Java. When priority boosting is enabled, NT boosts a thread's priority by an indeterminate amount for an indeterminate amount of time every time it executes certain I/O-related system calls. In practice, this means that a thread's priority level could be higher than you think because that thread happened to perform an I/O operation at an awkward time.

The point of the priority boosting is to prevent threads that are doing background processing from impacting the apparent responsiveness of UI-heavy tasks. Other operating systems have more-sophisticated algorithms that typically lower the priority of background processes. The downside of this scheme, particularly when implemented on a per-thread rather than a per-process level, is that it's very difficult to use priority to determine when a particular thread will run.

It gets worse.

In Solaris, as is the case in all Unix systems, processes have priority as well as threads. The threads of high-priority processes can't be interrupted by the threads of low-priority processes. Moreover, the priority level of a given process can be limited by a system administrator so that a user process won't interrupt critical OS processes. NT supports none of this. An NT process is just an address space. It has no priority per se, and is not scheduled. The system schedules threads; then, if a given thread is running under a process that isn't in memory, the process is swapped in. NT thread priorities fall into various "priority classes," that are distributed across a continuum of actual priorities. The system looks like this:

Windows NT's priority architecture

The columns are actual priority levels, only 22 of which must be shared by all applications. (The others are used by NT itself.) The rows are priority classes. The threads running in a process pegged at the idle priority class are running at levels 1 through 6 and 15, depending on their assigned logical priority level. The threads of a process pegged as normal priority class will run at levels 1, 6 through 10, or 15 if the process doesn't have the input focus. If it does have the input focus, the threads run at levels 1, 7 through 11, or 15. This means that a high-priority thread of an idle priority class process can preempt a low-priority thread of a normal priority class process, but only if that process is running in the background. Notice that a process running in the "high" priority class only has six priority levels available to it. The other classes have seven.

NT provides no way to limit the priority class of a process. Any thread on any process on the machine can take over control of the box at any time by boosting its own priority class; there is no defense against this.

The technical term I use to describe NT's priority is unholy mess. In practice, priority is virtually worthless under NT.

So what's a programmer to do? Between NT's limited number of priority levels and it's uncontrollable priority boosting, there's no absolutely safe way for a Java program to use priority levels for scheduling. One workable compromise is to restrict yourself to Thread.MAX_PRIORITY, Thread.MIN_PRIORITY, and Thread.NORM_PRIORITY when you call setPriority(). This restriction at least avoids the 10-levels-mapped-to-7-levels problem. I suppose you could use the system property to detect NT, and then call a native method to turn off priority boosting, but that won't work if your app is running under Internet Explorer unless you also use Sun's VM plug-in. (Microsoft's VM uses a nonstandard native-method implementation.) In any event, I hate to use native methods. I usually avoid the problem as much as possible by putting most threads at NORM_PRIORITY and using scheduling mechanisms other than priority. (I'll discuss some of these in future installments of this series.)


There are typically two threading models supported by operating systems: cooperative and preemptive.

The cooperative multithreading model

In a cooperative system, a thread retains control of its processor until it decides to give it up (which might be never). The various threads have to cooperate with each other or all but one of the threads will be "starved" (meaning, never given a chance to run). Scheduling in most cooperative systems is done strictly by priority level. When the current thread gives up control, the highest-priority waiting thread gets control. (An exception to this rule is Windows 3.x, which uses a cooperative model but doesn't have much of a scheduler. The window that has the focus gets control.)

1 2 Page 1
Page 1 of 2
How to choose a low-code development platform