| About InfoWorld : Advertise : Subscribe : Contact Us : Awards : Events : Store |
 |
||||
|
||||
 | ||||
 |
|
|||
 | ||||
|
|
|
||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||
 NEWSBeat biz rivals by testing By Tim Fielden , InfoWorld Test Center September 15, 2000 BUSINESSES SPEND HUGE amounts of time and money developing software. But one of the dirty little secrets of the software world is that many firms do not put nearly the same effort into testing their applications.
The irony is that few stages of the development cycle are more important than testing. The fact is that good testing equals a better return on investment, higher levels of customer satisfaction, and, most important, better software. Bad testing -- or no testing -- doesn't. So let's start with the basics. If you want to put a solid test program in place, the first thing you must do is design an effective testing process. Many shops come up short here. Rather than building a comprehensive testing phase into their development cycles, some companies instead perform "throwaway" tests -- impromptu, ad-hoc routines that are neither documented nor repeatable. Granted, throwaway tests can be useful: According to the popular maxim, if you sell something, you should use it, so companies often use applications in their build state. This process, sometimes known as "blitz testing", is a form of throwaway testing. But if you are using throwaway testing as your main means of evaluating software quality, you are almost certainly missing a large portion of errors. After making a commitment to formalized testing, the first decision you have to make is whether your tests will be manual or automated. Obviously, automated testing is often cheaper, faster, easier, and more reliable. But some tests are better done by hand. Usability tests are a prime example. If you are developing a shopping cart, you will want to assess the application's look and feel, and only human beings can give you that kind of feedback. The next step is to decide whether your tests will be functional ("black-box" testing) or structural ("white-box" testing). These approaches can complement each other, but they are also very different methods that yield very different results. Black-box testing measures whether or not a program meets all required functional specifications. To that end, black-box tests look for faults of omission -- places where the specification has yet to be fulfilled. This information can be useful, but because the black-box method doesn't scrutinize a program's internal code structure, it is not necessarily thorough. After all, just because a piece of software meets its spec doesn't mean it will always do what it is supposed to do. White-box testing, on the other hand, allows you to look inside the software. This approach relies upon the tester's knowledge of the application to guide the creation of test data, which are then used to determine whether the application's structure or logic is faulty. But white-box testing is more expensive and labor-intensive than black-box testing. Moreover, it is only as accurate as the tester's knowledge of the code. There's no accounting for errors of omission. Of course, black-box and white-box tests are not meant to be mutually exclusive. Typically, you'll run both. Most companies use black-box tests once the specification has been approved and the code is being developed. White-box tests tend not to be run until after the black-box phase is over, because white-box tests cannot be planned or implemented without source code. Many successful testers go even further, blending the strengths of the structural and functional methods throughout the entire testing procedure (a technique known as "gray-box" testing). Let's suppose, for example, that you're testing a Java payment processing application and that you have the source code, but no information on how security will be implemented. Because you only have certain parts of the code, you might want to run structural tests on the bulk of the program, and functional tests on the security modules. The point is that, to truly test a piece of software, you have to use both structural and functional methods. Once you have established which approaches you are going to take at which stages of the project, the next step is to define exactly what those tests are going to do. For that, you must develop test cases -- collections of tests, cataloged by function or operation. Creating test cases is an inexact science with few hard and fast rules to live by, so many testers simply follow best practices. One of the most important best practices is to ensure that a test case scrutinizes the code for things that are invalid and unexpected. The goal of testing is to find things that are wrong, not right. Thus, a validity test might search for fields that only allow numeric data -- what happens when you enter characters in those fields? In any event, when you're building test cases, you have a number of different kinds of tests and tools at your disposal. There's the unit test, which developer use to verify their code at the smallest compilable level. You can also use tools like the stub -- a shell of a program that only performs limited functionality, but lets you compile applications that would fail because they call nonexistent programs. The simulator is similar. This tool provides simulation services, such as generating traffic to a Web site or automatically seeding data into an application the way a user would. Finally, you can perform tests with trusted components, which are applications that have already passed the testing phase and can therefore be introduced into your software without risk. When the code has passed your developer's scrutiny, it is time to move it into the hands of your testing organization. Ideally, those people will have a full suite of component tests through which they put the application through its paces. For example, you might want to test that all your software's pages are displayable, all links are working, and all data is correctly presented and returned. At this stage, you must also establish sound entrance and exit criteria. This information clearly states what condition code must be in to enter this phase of testing (for example, all required functionality must be implemented, and compilation errors must not exceed a certain level of severity), and what condition it must be in to exit (you could forbid outstanding fixes, say, or demand that all code must execute correctly). Once the testing has met the required criteria, many companies choose to move on to a round of integration testing. This is one of the most critical stages of the entire testing project, as it is the first stage in which all parts of the application are assembled and presented as a system. Deadlines are usually tight at this point of a project's life cycle, so again, it is important to establish entrance and exit criteria. If the code does not pass, you might have to repeat the component testing phase. This is also the point at which it is crucial to have manager buy-in: Any code changes here could have significant testing implications. If you find that previous tests were invalid, you will have to revisit those tests, and that is the kind of thing that management ought to know. But you are not yet finished. System testing, the next step, is the practice of using the system or application as an end-user would. Of course, fixing errors found at this stage is bound to be a costly, drawn-out task. Thus, to save time and money in the future, those errors should be documented and applied to existing tests. Finally, you should have a complete set of regression tests on hand. These are nothing more than a subset of component tests, but they help ensure that any changes made to the code after production does not hurt previous functionality. One common practice is to simply rerun your existing component test beds, along with any new tests created for code fix validation. If your old test plans prove that the new functionality has not changed an application's behaviors, then it is probably safe to assume that the program is ready for end-users. Does testing sound like one big pain? If so, try to bear in mind that it is all done in the name of higher-quality software. So be ruthless. Don't be afraid to expose errors and kick a project back to square one. After all, that's the whole point of the exercise.  Related article Building better apps for your business via thorough testing Tim Fielden is a senior analyst with the InfoWorld Test Center. Send him e-mail at tim_fielden@infoworld.com.
 RELATED SUBJECTS  Software DevelopmentSPONSORED WHITE PAPERS 
SPONSORED LINKS
|
|||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||
|
||||||||||
