Category Archives: Exploratory Testing

Pomodoro Testing

In the past week I was asked whether could test something. As of my curious nature, I took the challenge to learn something new. In this case it was an iPhone application. I own an iPhone myself, but I never tested an application on the iPhone purposefully – I tested some of them in terms of the banana software that seems to get shipped despite (or maybe just because?) Apple’s checks before an app reaches the App-Store. But I digress.

I planned a whole day – Thursday – for this. I knew I was going to explore the app in sessions influenced by the Bach’s session-based test management paper, and I knew that I had to take a closer look into the specialties of an iPhone application like memory management and stress testing. Prior to Thursday I had already taken a look into the application, and seen some of the features. My colleagues shared the current feature backlog with me covering the features that were already implemented.

Another thing I knew was the fact I was about to deal with some customers during the day in order to arrange some new consulting gigs. So, I was rather that I won’t be able to deal the whole day with testing of the application.

Taking all these considerations into the context of my testing approach, I felt that a test session of one hour was too long to test the application and deal with interruptions during the day. Additionally one hour seemed too long to cover one aspect of the application. I could easily go for multiple features in that one hour. A year back I had crossed the Pomodoro Technique to manage my time, and I decided that a combination of session-based test management and Pomodoro Technique looked promising to me. In the Pomodoro Technique a time slot is usually 25 minutes long, – called a Pomodoro – interrupted by 5 minutes of break for mails, fetch a new coffee or go on a body break. This seemed to be just the right time frame for my testing sessions. That’s when I coined the term Pomodoro Testing.

So far, so good. I took my Time Timer with me that day to cope with the session time frames. In the morning I took a first pomodoro to plan my work for that day. I quickly identified that I needed a brief overview of the application in order to learn more about the sessions for the day. So I planned a first session of 25 minutes of inspectional testing followed by a debriefing in order to lay out the additional sessions after I gathered more information. Having learned some things about iPhone application testing a few days before, I also planned sessions for special behavior on the iPhone. I also planned the work I had to follow-up on besides my testing activities for the day.

I started my first testing pomodoro following the mission to find out as much as possible about the application. I identified several areas of the application, that I would like to see more about. There were ten or so main categories which I wanted to conduct. In the first pomodoro I initially took some notes on paper, but collected them together on a mindmap in the debriefing. For each of the ten main categories I planned a pomodoro, and realized that this probably will be too much for a single day.

After the first two testing pomodoros I got more and more familiar with the application. I noticed some things in the settings, and some things in the UI which seemed confusing to me, but there was nothing serious for me to note down. The application was pretty straight forward.

On the afternoon I noticed though that I was able to pull in more than one session that I had planned into a single pomodoro. I think this was caused by me getting more and more familiar with some aspects of the application. I planned on a very granular level for this application, but still I made fewer progress in the beginning, and could easily deal with 2-3 topics that I initially planned for later.

At the end of the day I had collected many things in my mindmap which I then shared with the product owner and the whole team. I annotated bugs that I found and stuff that I found inconsistent with little icons to grasp quickly, and put some more lines about my major findings into that email.

Pomodoro Testing for me is the opposite of thread-based test management. In thread-based test management there are things of an application that may take more than a single session of focused testing. For an iPhone application it seemed right to me to limit the session length down further since there were not as many aspects I wanted to explore. In addition it seemed the right level to plan my testing activities for a whole day.

In reflection, I wouldn’t want to further cut down the session length than that. 25 minutes was pretty short for some things, and I also extended that time frame in one or two sessions to finish up what I was dealing with rather than stopping my curiosity forcefully.

Since I worked alone I didn’t do many debriefings over the day. I took one in order to plan additional sessions after the first pomodoro, and I talked to a colleague later for a complete debrief about my findings. That’s pretty much it. If I would have applied this in a team, I would experiment with debriefing times in order to achieve between one and two debriefings over the course of the whole day.

I think there are some smaller application – like mobile apps – where Pomodoro Testing is the right practice to apply in that context. For the majority of web applications and desktop application though this technique is probably too time constraint to serve the purpose.

Structured Exploratory Testing – an oxymoron?

Michael Bolton beat me on blogging about it. Though, I still want to throw in my pieces to the structure in Exploratory Testing debate. The source of the conversation was

On Twitter, Johan Jonasson reported today that he was about to attend a presentation called “Structured Testing vs Exploratory Testing”.

as Bolton writes. The implication of the talk is probably, that Exploratoy Testing and Structured Testing are opposite and mutually exclusive. Let’s see if this holds.

Continue reading Structured Exploratory Testing – an oxymoron?