Inspired from the EuroSTAR 2010 Exploratory Test Management roundtable I had an idea, which I would like to play with a little. Since it doesn’t seem as if I will get to it too soon, I decided to put it up on my blog, and maybe get some feedback from peers and early adopters who are eager to play around with the idea, and can provide me with feedback. Now, the idea is to collaboratively come up with test charters for all the Exploratory Testing sessions on your project.
Exploring a product for problems is one essential part of my understanding of “done” on any project. I love to do much test automation, but I also set time aside for exploratory testing. With Agile methodologies focusing on the team and on collaboration, I think there should be a way to plan your test sessions in the team. But how can we reach a common understanding of the charters for test sessions within a team? – whatever the used methodology may be.
At the roundtable someone (sorry, I forgot the name, but I think it was Henrik Andersson) explained that his team was debriefing and chartering their sessions together. During the debrief the team often came up with new charters for new sessions. Having everyone else on your team knowing what others did is a great thing.
But how do you charter together? That’s where my idea comes in. I would like to try out an approach similar to the way Agile teams estimate their work. The idea is that after getting a common understanding of the application (i.e. through a collaborative inspectional testing session, or through a first debrief, or through an already existing understanding of the application, or…) the team should have a good basis in order to come up with test charters.
The team gets together in a brainstorming session, where they will identify charters. They can use any aids for this. For example they could start to create a story map of the application, and see how each of the mapped items should be considered for a testing charter. The team might dive into the Product Backlog, in order to get an understanding of the application, and how to address risks. Or the team might use a risk analysis in order to define charters. In general the charters should not be constraint by any aid they use. It’s more important to come up with as many ideas as possible for charters. You can come up with short test sessions of 90 minutes, or come up with test threads consisting of multiple sessions. Since this brainstorming might take more time than worthwhile, you might want to timebox the brainstorming.
In a second step the team takes a look on their brainstorming results. The task then is to assign bring the charters in a priority for your project. The target is to have a basic understanding of the priorities of all the charters that you brainstormed. The charters are intended to be worked on in this priority order during the next few days. Don’t try to make this priority too perfect. You are going to change the priorities based on the debriefings in the next few days anyways. Depending on your context, you may end up re-prioritizing a lot, or just seldom. In either case you will have a better knowledge about your variations already before starting this. Try to come up with a good-enough priority list of the test charters for the next one or two days at least.
That’s it for the first chartering. This shouldn’t take too long. You should restrict it to at most one hour of time. I am also unsure whether coming up with durations for your test charters should be done in a separate step. Also, when and how you start to break down test threads into charters is a thing I would have to play around with at the first project that I might want to try this out.
Now, while working through your test charters in priority, you will get new knowledge. Since humans are not that good in guessing the future, you will find new priorities and changing priorities during the next few debriefings. Spend some time (i.e. five minutes) to reorganize your open test charters after the debrief, and come up with new charters, and prioritize them – similar to the way you did the initial test charter backlog. Over time you will get knowledge about your project, and adapt to changing risks.
That’s it. I wrote this idea (not in this extent, though) to Chris McMahon as a proposal for the second Writing-About-Testing conference in May, but he understood it as a proposal for test estimation. Though you might use your charter backlog as an estimation technique as well (maybe I will write about this at some future point), it’s primarily benefit comes from the test planning activity.
So, if you dare to try this idea out before I can, drop me a comment and tell me what worked for you, and what didn’t.