Test-driven GUI development with FEST

Try a functional approach to testing Swing GUIs in test-driven environments

1 2 3 Page 2
Page 2 of 3

Introducing FEST

FEST (Fixtures for Easy Software Testing) is an open source library licensed under Apache 2.0 that makes it easy to create and maintain robust functional GUI tests. Although several open source projects have been devised for testing GUIs, FEST is distinguished by the following features:

  1. An easy-to-use Java API that exploits the concept of fluent interfaces to simplify coding. For example, FEST lets me rewrite the code from Listing 1 as follows:

    private FrameFixture naive;
    
      @BeforeMethod public void setUp() {
        naive = new FrameFixture(new Naive());
        naive.show();
      }
    
      @Test public void shouldChangeTextInTextFieldWhenClickingButton() {
        naive.button("byeButton").click();
        naive.label("messageLabel").requireText("Bye!");
      }
    
  2. Assertion methods that detail the state of GUI components (for example, requireText("Bye!")). Such assertion methods can be useful when practicing TDD because they read like a specification.
  3. Support for both JUnit 4 and TestNG.
  4. Screenshots of failing tests, which can be embedded in an HTML test report when using JUnit or TestNG. This configurable feature is useful when verifying that a test or group of tests failed because of an environment condition and not a programming error.
  5. A Groovy-based domain-specific language that simplifies GUI testing even further. (This feature is still under development and is considered experimental.)
Writing testable GUIs

An important aspect of test-driven development is writing software that is designed to be tested. Try the following suggestions for writing testable GUIs:

  • Separate model and view, moving as much code as possible away from the GUI.
  • Use a unique name for each GUI component to guarantee reliable component lookup.
  • Do not test default component behavior; for example, do not test that a button reacts to a mouse click -- that is the job of the Sun Microsystems Swing team!
  • Concentrate on testing the expected behavior of your GUIs.

Although FEST does provide some unique features, it does not reinvent the wheel. Instead of creating yet another mechanism for component lookup and user-event simulation, FEST builds on top of Abbot, a mature, open source project for GUI testing. In fact, FEST was born from a previous project I created with Yvonne Wang Price, TestNG-Abbot. The initial mission of TestNG-Abbot was to provide TestNG support to Abbot, but TestNG-Abbot quickly evolved from being a "glue" project into a nice API for GUI testing. Yvonne and I then decided to make the library independent of any test framework (JUnit or TestNG)] and changed its name to FEST to reflect its new nature.

Tests created with FEST are robust because they are not affected by changes in layout or component size. In addition, FEST provides features not available in other GUI testing libraries -- its simple but powerful API being the most important one.

FEST compared to Abbot and java.awt.Robot.
Figure 5. FEST by comparison

Test-driven GUI development with FEST

In the following sections you'll get to know FEST by walking through a testing example. You should download FEST now if you want to follow along with the examples. Figure 6 is a sketch of the example GUI to be tested. It represents a login dialog where the user enters her user name, password and domain name to log in to the system.

A Swing-based login GUI.
Figure 6. A Swing-based login GUI

The expected behavior of the dialog is as follows:

  1. The user enters her user name and password, both required.
  2. The user selects from the drop-down list the domain to which she wishes to connect.
  3. If any of the fields is left blank, a pop-up dialog box notifies the user that the missing information is required.

As I did in the first example, you will start with a failing test.

Listing 4. A FEST test that verifies an error message

// Omitted imports and package declaration

 1  public class LoginWindowTest {
 2
 3    private FrameFixture login;
 4
 5    @BeforeMethod public void setUp() {
 6      login = new FrameFixture(new LoginWindow());
 7      login.show();
 8    }
 9
10    @Test public void shouldShowErrorIfUsernameIsMissing() {
11      login.textBox("username").deleteText();
12      login.textBox("password").enterText("secret");
13      login.comboBox("domain").selectItem("USERS");
14      login.button("ok").click();
15      login.optionPane().requireErrorMessage().requireMessage("Please enter your username");
16    }
17
18    @AfterMethod public void tearDown() {
19      login.cleanUp();
20    }

Notes about the test

The test in Listing 4 uses FEST to invoke the GUI being tested, simulate user events and verify that the GUI works as expected. More specifically, the test does the following:

  • Uses a org.fest.swing.fixture.FrameFixture to manage and launch the window to test (lines 6 and 7).
  • Ensures that the text field where the user enters her user name is empty (Line 11).
  • Simulates a user entering the password "secret" in the appropriate text field (Line 12).
  • Simulates a user selecting a domain from the drop-down list (Line 13).
  • Simulates a user clicking the "OK" button.
  • Verifies that a pop-up window (a JOptionPane) is displayed showing an error message with the text "Please enter your username" (Line 15).

Two more important observations about the test shown in Listing 4:

  • FEST performs component lookup using the component's unique name, which guarantees that a GUI component always can be found, as long as it hasn't been removed from the GUI. It is possible to find components using custom criteria specified in a org.fest.swing.ComponentMatcher, but it may not be as reliable as using the component's name. For example, finding a component by its displayed text is problematic because such text could change over time (for instance, as result of changes in the user's preferences or by adding support for multiple languages).
  • It is necessary to release resources used by FEST (such as the keyboard, mouse and opened windows) following the execution of each test (as shown in Line 19). You can release used resources by calling the method cleanUp in org.fest.swing.fixture.FrameFixture, org.fest.swing.fixture.DialogFixture, or org.fest.swing.RobotFixture.

Creating and testing the GUI

Once you have finished writing the test, you create the GUI to test and verify that the test does in fact fail, as demonstrated in Figure 7.

Creating the login window and verifying that the test fails.
Figure 7. Creating the login window and verifying that the test fails (click to see the demo)

One important detail to notice is that specified component names must match the ones expected in the original test from Listing 4. This way, FEST can find those components easily and simulate user events on them. Listing 5 shows how you would specify the names of components in the login window using a GUI builder.

 

Listing 5. The GUI's login window

// Omitted additional code generated by GUI builder.

  jLabel1.setText("Username:");
  jLabel2.setText("Password:");
  jLabel3.setText("Domain:");
  jTextField1.setName("username");
  jPasswordField1.setName("password");
  jComboBox1.setName("domain");
  jButton1.setName("ok");

In Listing 6, you add the user-name validation to make the test pass.

 

Listing 6. Adding user-name validation

jButton1.addMouseListener(new java.awt.event.MouseAdapter() {
    public void mouseClicked(java.awt.event.MouseEvent evt) {
      jButton1MouseClicked(evt);
    }
  });

  private void jButton1MouseClicked(java.awt.event.MouseEvent evt) {
    String username = jTextField1.getText();
    if (isEmpty(username))
      JOptionPane.showMessageDialog(this, "Please enter your username", "Error", JOptionPane.ERROR_MESSAGE);
  }


Figure 8 further demonstrates the steps to add the user-name validation and see the test pass.

Adding username validation to make a test pass.
Figure 8. Adding user-name validation to make a test pass (click to see the demo)

As you see in Figure 8, the test passes and the user-name validation is complete. You're not quite finished with the login window, however. The requirements specify that you still need to implement the following behavior:

  • An error message to be displayed if the user does not enter her password
  • An error message to be displayed if the user does not choose the domain to which she wishes to connect
  • A successful login

I won't go over the steps to complete the sample GUI here, but you certainly could build in the additional features yourself. The important point to take away from this simple example is the confidence you will have in a GUI built using FEST. The TDD approach ensures that your GUI will work as expected when you are done with it. You'll also end up with a comprehensive test suite, which you can use to ensure that any future changes to the GUI (such as changing the component layout or adding some "makeup" to it) will not introduce bugs accidentally.

You've seen a fairly easy, successful example of test-driven GUI development using FEST. Next I'll show you how to handle a less straightforward testing scenario.

1 2 3 Page 2
Page 2 of 3