At times I find quite interesting things in topics that I don’t seem too particularly interested in. A recent example – again – comes from Let’s Test in May 2012. While at the conference, I read through the program and thought that I don’t need to learn anything new from recent trends in bug reporting. Preferring to work on Agile projects, I don’t think I will use a bug tracker too much in the future.
On the other hand, I knew that I had signed up for BBST Bug Advocacy in June. So I kept wondering what I will learn there, and whether it will be as work intensive as Foundations was. I was amazed at some things. This blog entry deals with my biggest learning: for building blocks for follow-up testing – something I think a good tester needs to learn regardless of their particular background.
When I find a bug in a program, I usually start with some follow-up testing activities. But how do you test around a problem? Well, easy, vary the variables. What’s a variable? Elisabeth Hendrickson once taught me, everything that can be variable. Cem Kaner explains four categories for variables:
- Vary my behavior
- Vary data
- Vary settings
- Vary configuration
Let’s take a look into each of these in some more detail.
Vary my behavior
After having replicated a problem, I am interested in exploiting the problem. What’s the worst case scenario that I can make this bug replicate in? A first approach to this can be to vary my own behavior. For example instead of hitting Ctrl-V I can use the right mouse button context menu, or I can use the edit menu, and hit the menu entry in a usual desktop application.
I can also try to type very slowly into a text editor, or I can use the mouse to navigate between entry fields instead of the tabulator key – or maybe use the tabulator key instead of the mouse.
With my variations I can trigger different circumstances. Think about asynchronous transfers in today’s AJAX websites. What is the fastest pace I can trigger events so that I run into a race condition? This could easily outline a problem in the security system or with transactions. Everything that makes the bug even worse in daily use is interesting for me in follow-up testing mode.
Similarly I can vary all the data that my program is using. I can vary input files, the data I use to replicate the bug. Can I trigger the same behavior when I use a different language? What about unicode characters? What about sentences that do not make sense?
I may also vary settings like configuration files, registry entries, cookies in the browser, or /etc/ files on Unix machines. Even on mobile phones, I can sometimes vary behavior by uninstalling the Dropbox application, or switching off some of the security settings, turning on cellular data, or wifi.
When it comes to vary the configuration I may want to use a different machine with slower or faster bandwidth to the Internet, slower or faster CPU, memory, different Operating System like Windows 7 vs. Windows Vista or XP. But also think about iOS 3.5 vs. iOS 5 and iOS 6. I might also change the particular driver I use for a printer, use the default mouse driver instead of the Logitech one, or use the trackpad.
I think these category ideas help to come up with more follow-up test ideas. If we complement these ideas with Elisabeth Hendrickson’s Test Heuristic Cheat Sheet we may also come up with a lot of different more concrete variations.
Happy follow-up testing.