Test email components in your software

The basis of a maintainable and stable software system is the ability to easily unit test the system in an automated way using testing frameworks such as JUnit. Unfortunately, verifying correct delivery of a system's email messages and validating content in an automated fashion is not a trivial task due to the complexity of typical email servers. Such testing is possible but tricky and may involve complex administration tasks or integration with proprietary message stores—tasks not conducive to successful automated testing.

I had been thinking about this problem for some time when I came up with the idea of writing my own SMTP (Simple Mail Transfer Protocol) server designed from the ground up to be used solely as an aid to automated testing. The idea was to create a server that could respond appropriately to SMTP commands, but, rather than attempting to deliver email messages to the receiver or forward to another server as appropriate, it would simply cache the received headers and message body for later extraction and verification by a test client.

One of the main attractions of using Java for network-based applications is access to a range of quality networking libraries provided as standard in the Java development environment. Complex yet robust networking software can be written in a few lines of code. In Java, the javax.mail package may be used to communicate with both email servers for the delivery of SMTP-based email messages, and message stores using POP (Post Office Protocol) or IMAP (Internet Message Access Protocol). This article will concentrate on the former—SMTP to send email messages—rather than message-store protocols that a user agent would use to extract messages from the store.

The basic requirements for such a "dummy" server are:

1. The ability to manage (i.e., start and stop) the server from a Java program—specifically a JUnit test case
2. No server administration tasks, configuration, authentication
3. The server would not (and should not) try to deliver messages since we are simply using the server to unit test
4. The ability to extract delivered messages from the server itself, rather than connecting to multiple and potentially remote message stores to verify delivery and content

This article explains how to write this simple email server; see Resources to download its code.

Email standards

Before we start writing an SMTP server, an overview of the standards documents that we will use in building the dummy SMTP server is in order.

Most commercial email servers in the world today support the SMTP protocol, originally defined in RFC (Request for Comments) 821 in August 1982. As an historic side note, the word simple in Simple Mail Transfer Protocol is derived from the fact that SMTP itself was based on an earlier more complex protocol definition named, unsurprisingly, the Mail Transfer Protocol (MTP). RFC 821 itself is currently being superceded by the draft standard RFC 2821, published in April 2001. This new draft standard adds no new functionality or updates to existing functionality, so, for the remainder of this article, when referring to SMTP, I mean the protocol as defined in RFC 2821. Since we will not be implementing anything more complex than the basic set of SMTP commands, for all intents and purposes, use of this later draft standard should not affect our solution.

In the interest of completeness, I should mention some of the other standards documents that relate to the SMTP protocol:

• RFC 2822, Internet Message Format: Defines the syntax of email messages themselves, including the headers (supercedes RFC 822).
• RFC 2920, SMTP Service Extension for Command Pipelining: Defines an SMTP protocol extension that allows multiple commands to be sent within one TCP send operation. In the interests of simplicity, our dummy SMTP server will not support this extension.
• RFC 3030, SMTP Service Extensions for Transmission of Large and Binary MIME (Multipurpose Internet Mail Extensions) Messages: Defines an SMTP extension for attaching large binary objects to email messages.

Many other standards documents relate to SMTP, adequate coverage of which reaches beyond this article's scope. For further information, see the IETF (Internet Engineering Task Force) Website.

The SMTP protocol by example

SMTP is a relatively straightforward protocol to understand. All communication between the client and the server is plain text and, hence, more or less human readable. Here is an example of a simple exchange between a Java mail client and the dummy SMTP server. Once the client has connected to the server on the default SMTP port (25), the following conversation ensues:

1  S: 220 localhost Dumbster SMTP service ready
2  C: EHLO WKS-JKITCHEN
3  S: 250 OK
4  C: MAIL FROM:<sender@here.com>
5  S: 250 OK
6  C: RCPT TO:<receiver@there.com>
7  S: 250 OK
8  C: DATA
9  S: 354 Start mail input; end with <CRLF>.<CRLF>
10 C: 
11 C: .
12 S: 250 OK
13 C: QUIT

Whenever client-server conversations are shown in this article, the prefix S: will be used to indicate the server part of the conversation and C: to indicate the client part.

In the above client-server exchange, the client connects to the email server using a socket connection. In fact, the standards don't dictate how this communication should occur, but invariably, socket connections are used. At Line 1, the server responds using a 220 code indicating that it is ready to talk. Some information related to the email server itself is also returned—in this case, the host name and email server name (Dumbster, the project name I used during code development).

At Line 2, the client issues an EHLO command, which initiates the conversation, and at Line 3, the server responds with a 250 code indicating that the client may proceed.

At Line 4, the client issues a MAIL command indicating the sender's email address, and, at Line 5, the server replies with a 250 code indicating that the sender address was validated and may proceed.

At Line 6, the client issues a RCPT command to identify the message's recipient. Multiple recipients may be indicated using multiple consecutive RCPT commands. In this case, we have just one recipient. At Line 7, the server again responds with a 250 code indicating that the address is valid and the client may proceed.

At Line 8, the client issues a DATA command indicating that it is ready to start sending the message's headers and body.

At Line 9, the server responds with a 354 code indicating to the client that it may proceed. Since the client will most likely send multiple lines of text, the server also indicates how the client data should terminate: with a single point character on its own line of input. In this case, there is no message body—not likely to happen in the real world, but I wanted to keep this example short to concentrate on essentials. So at Lines 10 and 11, the DATA command terminates.

At Line 12, the server responds with a 250 code indicating that the message was correctly received.

At Line 13, the client sends a QUIT command, at which point the client-server exchange terminates.

Required toolset

To develop an SMTP server, the tools listed below are required. I have given specific versions, but you will most likely be able to swap these out for previous or later versions if required, as nothing depends on specific versions of the JDK or JavaMail implementation.

• Java 2 Platform, Standard Edition (J2SE) 1.4.1_02
• JavaMail 1.3
• JavaBeans Activation Framework (JAF) 1.0.2
• JUnit 3.8.1
• Apache Ant 1.5.3

Overview of implementation

To create a functioning dummy email server, we must implement a server that will interact in a standards-based fashion with clients that connect to it. The implementation needs to be able to send the correct response to SMTP commands received from the client and track the state of the interaction with the client to send the correct response or error. This involves implementing the set of SMTP commands to which the server must respond. These are:

• EHLO/HELO: identifies the client to SMTP server
• MAIL: used to initiate a mail transaction and identify the sender—actually a return path
• RCPT: identifies a single recipient's email address
• DATA: used to send headers and body text
• RSET: resets the mail transaction without dropping the connection
• VRFY: requests the server to verify the given email address's correctness, which our dummy server doesn't support
• EXPN: requests the server to verify that the given address is a mailing list and, if so, replies with a list of addresses that make up the list—not supported by our dummy server
• HELP: requests useful information from the server—our implementation returns a simple nondescript message
• NOOP: no operation command, often used by clients to determine if the connection is still open without affecting the current transaction's state
• QUIT: instructs the server that it may close the connection after sending a reply to the client

A great degree of depth is not required for the implementation of these commands. For example, for the RCPT command, the email address is always assumed correct.

The client-server conversation's state is preserved in the SMTP server. This helps generate the correct responses to commands received from the client. The following is a state table that summarizes the server's main internal states and how client input affects a change of state and the responses generated by the server. For simplicity, only the principle SMTP commands are shown, with no error conditions.

Server internal state machine

Equipped with the above state table, it is now relatively straightforward to implement the SMTP server. The server's main controller resides in the SimpleSmtpServer class. The current implementation is simple minded, and, although it runs in its own thread, it is not multithreaded, meaning that only a single request at a time may be handled. A simple enhancement would be to support multiple simultaneous clients. The basic steps of the main controller are as follows:

1. Set up a server socket to receive client socket connections
2. Wait for a client socket connection
3. Process client commands and send replies
4. Client closes socket connection
5. Repeat from Step 2

Setting up a server socket connection is easy in Java:

1 ServerSocket serverSocket = new ServerSocket(25);
2 serverSocket.setSoTimeout(500);

At Line 1 in the code above, a new instance of ServerSocket is created. The server will listen on port 25, which is the default port number defined in the SMTP standards. At Line 2, we set the maximum blocking time of an accept() method on the socket connection to 500 milliseconds. This ensures that we do not block indefinitely, thus allowing an outside process to cleanly shut down the email server in a timely fashion. Next, the server waits for client connections:

 3  while(!doStop) { // Repeat until stopped
4    Socket socket = null;
5    try {
6      socket = serverSocket.accept();
7    } catch(InterruptedIOException iioe) {
8      if (socket != null) { socket.close(); }
9      continue; // Time out no connection received
10   }

At Line 3, the server loops until requested to shut down: the doStop Boolean variable is set by a management method on the SimpleSmtpServer class named, unsurprisingly, stop(). At Line 6, a Socket instance is created by calling the accept() method on the ServerSocket instance. This call will block until a client connection is received or an InterruptedIOException is thrown, indicating that the timeout of 500 milliseconds defined at Line 2 above has expired. If the timeout expires, we continue from the start of the while loop, allowing the server to check if it has received a shutdown request in the meantime.

The server needs to get references to both the socket input stream containing client SMTP commands and the socket output stream allowing the server to respond as appropriate to client input:

11   BufferedReader input = new BufferedReader(
12      new InputStreamReader(socket.getInputStream()));
13   PrintWriter out = new PrintWriter(
14                        socket.getOutputStream());

The server's internal state is now initialized by setting the state to CONNECT and creating a SmtpRequest instance for the connect action:

15   SmtpState smtpState = SmtpState.CONNECT;
16   SmtpRequest smtpRequest = new SmtpRequest(
             SmtpActionType.CONNECT, "", smtpState);

Once the request has been created, it can be executed. Execution of a request in this context means we take the server's current internal state along with client input from the input stream to produce a new internal state and output to be placed on the output stream back to the client. See Lines 17 to 19 below:

17   SmtpResponse smtpResponse = smtpRequest.execute();
18   sendResponse(out, smtpResponse);
19   smtpState = smtpResponse.getNextState();

Once the initial connection request has been handled, the server processes the mail commands from the input stream and sends appropriate responses back to the client on the output stream: