Software testing practices to live by

Notice I don’t say "best" software testing practices

I’m often asked, “Tell us what are the best practices in software testing.” While I’d like to say I have the answer, just like a doctor treats patients, there is not a cure-all medicine for everyone. Software testing practices are different for each industry, for every company, and for every tester. After working with hundreds of clients and projects in different industries over the last ten years at XBOSoft, I’ve found there are common principles that lead to success.

I won’t call them “best” practices because what’s best for one company is not the best approach for another company, even in the same industry. There are just too many contextual differences to enumerate when applying the “best” practice to any particular situation. However, I have compiled a list of principles that provide a solid foundation when you embark down the road to better software quality.

1. Save time

We’ve all heard that we can never get time back. That’s true in life and also in software testing. We want to eliminate any unnecessary activity. In agile, some interpret this as “don’t write any test cases and don’t log defects.” Of course, this is not the case, but reduction is definitely in order. The question is where and when. Some training and experience is needed to know how to lighten traditional testing artifacts. You always want to ask questions: Why are we doing this? Is it because that’s what has always been done or that’s what someone told you to do? What result will we get and who will this be used by? When? Why? What will they know as background information? What information will be derived from each test result? How will this information be used? Every test activity has an opportunity cost. Is this more important than other activities at this moment? Asking and answering these questions will lead you down the path of saving time.

2. Be objective-oriented

That doesn’t mean the opposite of subjective, although that’s good too. Strive to be objective-oriented versus task- or action-oriented. You need to connect everything that you do with a why or an objective. Your objective may be to find important problems quickly in a certain function or feature. If so, starting off by writing test cases may not be the best way. On the other hand, if your objective is to “satisfy regulatory requirements” or “provide documentation so that tests can be automated,” you should clearly understand what level of granularity is needed. First, clearly understand your objective. Sometimes you may not know and have to ask. Maybe the person you ask won’t know either. That will lead to a meaningful discussion whereby you can evaluate your situation and find the most useful actions to take to achieve that objective if it is indeed worthwhile.

3. Emphasize test design

Effective and efficient software testing requires experience in various test techniques. Most software is not as simple as filling out a form; it has complicated business logic and could involve third-party plugins as well as various other parameters. To test effectively means more than going through every function, but using critical thinking and test design techniques with a deep understanding of user stories, both positive and negative. Don’t blindly apply techniques from one project or story to another. Think carefully through the stories and where there are likely to be problems in each story. Then apply the technique best suited for that problem.

4. Evaluate risk and review the timeline

In general, testing focuses on product functionality. In other words, make sure the product can do what it is supposed to and doesn’t do what it’s not supposed to do. Based on an understanding of the product’s working logic, determine the problems that are most likely to happen and have the biggest impact either on the project itself, to the user or client. Then focus your energy in these areas. After you have results, run through the same exercise again incorporating additional information that has been gleaned from the results, and then again determine where you think problems are most likely to occur. The goal is to gather information about the quality of the software from the tests as soon as possible. Think of it this way: You need to provide management information to make a go/no-go decision on release timing—now, tonight, this week, next week. Let that guide your approach.

5. Promote exploration

Rather than dive into test cases, use exploratory testing to learn and gather information quickly.

  1. Use exploratory testing as the basis and reasoning of where to develop test cases rather than writing test cases first, unless there is a clear and mandatory regulatory requirement.
  2. Let the results from the exploratory testing tell you where to put your effort and where formal test cases are needed.
  3. Exploratory testing can also tell you where different levels of granularity are needed in your approach. Rather than being tied to a strict process, let your process be a framework or approach.
  4. Exploratory testing helps you quickly gather important information as to where to automate, where to conduct performance testing and where your product has the most risk.

If you’re a test automation proponent, that’s great, but acknowledge that machines just can’t do some things better than people and sometimes the human eye can outsmart a machine. Test automation and manual testing are complements with some overlap, not substitutes.

6. Use heuristics

We need to ascertain when to use careful analysis and when to use rules of thumb or heuristics. Often, a good heuristic will get you 80 percent there with 20 percent of the effort. Given that you can often overestimate problems you are testing, think if a heuristic can be applied and if the heuristic is valid and reasonable before going off the deep end in analytical thought. One example is combinatorial testing. Great for big problems, but for smaller problems you can apply other “back of the envelope” techniques to help you move faster and solve the problem quickly. In many cases, you’ll learn something in the quicker approach and have time to go back and use what you learned for another solution round.

7. Work and communicate as a team

Talk to each other. Use other people’s work—their templates, approaches, and whatever you find useful. We’re often surprised that when people put their heads down to solve a problem or get something done, their neighbor may have already solved it or have a better technique. To facilitate this, use pair testing. It’s similar to the XP practice of pair programming but for testing. Two testers sitting side by side on the same computer or different computers but testing the same functionality of the same software. Talking to each other as they go through functions and what they find. You’d be surprised what you can learn from someone else by something as simple as sitting next to them and talking.

In summary, always ask what tests you are running and why. Is it the most important thing to get done today? This week? Are you using the right strategy and technique for your objective? Talk to each other and be mindful that every minute counts, not just in life but in software testing too!

Copyright © 2017 IDG Communications, Inc.