Category Archives: Testing

Software Testing

CAST 2011: Build a deeper community of practice – How to organize a peer conference

At CAST 2011 Paul Holland held a presentation on how to organize a peer conference on software testing. Paul gave myself some great pointers for the upcoming organization of the GATE workshop.

Continue reading CAST 2011: Build a deeper community of practice – How to organize a peer conference

German Agile Testing and Exloratory (GATE) Workshop

Maik Nogens and myself are organizing the German Agile Testing and Exloratory (GATE) Workshop. We are proud to announce the call for contributions for this workshop. It will be held in Hamburg, Germany on October 1st 2011.

GATE will be a low budget, non-profit peer workshop. This means that we might split expenses for the location and commodities equally among the participants (probably below € 100). Every participant will take care of the remaining costs for her- or himself, e.g. travel, lunch.

As the main goal we identified the following elevator pitch:

For ambitioned testers who want to learn established approaches and practices in the craft of software testing the GATE workshop provides a platform for an equal knowledge exchange. In contrast to traditional conference formats the GATE workshop provides a practical, low budget, controversial experience on software testing.

That said, we are interested in contributions such as

  • realistic experience reports
  • controversial testing techniques
  • testing in practice (Testing Dojos, Hands-on testing)
  • options for distributed software testing

Feel free to contact us if you are unsure. We will be most glad to provide you feedback.

The language of the workshop will be dependent on the workshop participants. It might be German all day, but if we got international contribution, we might decide to hold the workshop in English.

If you want to attend the first German Agile Testing and Exploratory Workshop, come up with a contribution, and send it to us until September 5th. Further details on the attendees, the program, and travel information will be provided later.

If you are interested in the format, there have been quite a few of these peer workshops in the testing space recently. Here are some pointers:

Hope to meet you in the nicest city in the world. Hope you are as excited as we are.

Testing Dojos from the Back of the Room

Last week I organized together with my colleague Meike Mertsch a Testing Dojo for a client of ours. The testers there had attended some Testing Dojos in the past, but with some drawbacks. As part of the clarification process the testers picked an internal application – their billing system. I knew that a billing system can be quite extensive to test within a setting of a single hour. So, I asked for an introduction in the morning, in order to make up my mind on the mission for that dojo. One thing I noticed in the morning was the applicability of the 4Cs scheme for the dojo.

The 4Cs are described in Sharon Bowman’s book Training from the back of the room. The 4Cs stand for Connections, Concept, Concrete Practice, and Conclusion. My colleague Stefan Roock also explained me that he sometimes exchanges Concrete Practice with Concept in his classes. I found that we could apply the same pattern to the billing system testing dojo.

Connections

Since the billing system is large in nature, we needed to make a connection to the known stuff of that billing system. The teams had rarely to do with the billing system. So the tacit knowledge was hard to grasp, and it was hard to keep the lessons learned for that system. I knew we needed to bring up a connection to that past knowledge. We picked to have a group brainstorming to bring back the few elements of the billing system that stuck in the heads of the testers in the beginning. We spent 10 minutes on that.

Concrete Practice

We had overall roughly two hours. I had planned to run a combination of Weekend Testing sessions in pairs of two or three and Testing Dojos intervened by short debriefings about the findings. I ran a similar setting back at WeekNight Testing live in March 2011. We scheduled two to three 20 minute sessions of testing with 5 minutes of debriefing and pair switching. The twenty minutes seemed to be enough to run one or two tests in the billing system. So, it was a good fit.

Concept

We mixed this up with the concept the testers should learn more about. For the billing system there was a RESTful interface. This was introduced and documented in the company’s wiki, but just one tester appeared to know about this. So, we derived a mission which included the REST API for testing the billing system. We had print-outs of the documentation with us, so every pair could get a grip on it. This worked surprisingly well.

Conclusion

For the conclusion we sat together for the final half hour and talked about ideas for improvement in the billing system. To get the discussion started I asked the participants to apply wishful thinking, and come up with anything they would like to see there. Despite stuff like influence on time and running bills for single accounts, there were also concerns about traceability like logfiles and progress report.

After that Meike came up with an action list. She asked for things that the participants can apply right away as well as for volunteers to carry out the actions and follow-up on them. We identified two points which two of the participants put on their agenda.

Reflection

So, the 4C model worked surprisingly well with this sort of Testing Dojo. It was absolutely necessary to make the connection to the system in the beginning. The attendees knew a lot about the system. We had to bring this together. Together with the complexity of the billing system, we would have been lost if we had skipped that part.

The debriefings after each session were also necessary with the pair switch to spread not only the knowledge but also bring several learning insights based upon different machines. From the discussions in the morning I got the notion that previous Testing Dojos did not come with a mission as well as debriefings. Since the Exploratory Test Management Roungtable at last year’s EuroSTAR conference I consider the mission and the debriefing the most mandatory element of any session-based test management implementation. The experiences at this client underlined this impression.

On the PageObject Pattern

Recently I started to think more in depth about the PageObject pattern – at least how some folks called it initially. One problem I noticed in discussion with colleagues and clients is the lack of pattern description for this so called pattern. I decided to explore this problem in more depth, and try to come up with a constructive solution to that problem.

Continue reading On the PageObject Pattern

EuroPLoP from a first-timers perspective

Last week I had the pleasure to attend EuroPLoP. I submitted a pattern that I started back on October 1st at the inaugrual AA-FTT pattern writing workshop. Back then I called it Essential Examples, whilst through several round of shepherding and workshops I ended up with the name One clear purpose currently.

From the conference, I had several impressions as well as some insights which I would love to have gotten earlier. I decided to write these down for the next first-timers in the future to consider – even before submitting a pattern in first place.

Continue reading EuroPLoP from a first-timers perspective

The Landscape of Testing

In the past few days I had the opportunity to stay with Diana Larsen. She explained a bit about her current studies on Human System Dynamics, and introduced some tools from this school of thought to my colleagues and me.

One of the interesting tools was the Landscape Diagram which you can find explained in more depth here. The basic concept aligns around two axis (on a meta level a lot of consulting tools do). The y-axis relates to the level of the agreement within the team on the approach to take. This might relate to the agreement on using a particular process like Scrum, or on introducing a particular tool in the software development process like FitNesse or JUnit. The x-axis aligns the certainty level with this decision. This might relate to the familiarity with the process or tool introduced. On the lower end, there is high agreement, while on the upper end the team is far from agreement. On the left end the certainty with the decision is very high, while the right end represents a great level of uncertainty.

Now, this Landscape Diagram reminded me of Stacey diagrams which I knew from the CSM course I took a few months back. In the Landscape diagram there are just like in the Stacey diagram some interesting areas. When the level of agreement and the degree of certainty is high on the lower left corner, we speak of a high degree of order and organization. In here rely domains like accounting, and taxes where a high degree of regulation is the way to go. Though turning in your tax statement might horrify you each year, you probably want to have that kind of order to support your community.

In the upper right corner, in the area of low agreement and high uncertainty lies the area of chaos. If you don’t agree to anything, and you are new to the field, then you are basically cowboy-coding.

Between these two extremes lies the field of self-organization. With some agreement and some certainty your team will self-organize. But this also holds for low certainty if you agree on what to do, and for low agreement if you have a high certainty or familiarity in what you are doing.

Interestingly you can move from chaos downwards into more ordered fields by providing structure thereby creating certainty and agreement. You can move in the other direction by opening thought process to new ideas. I think Jerry Weinberg refers to this in some of his books (i.e. Exploring Requirement – Quality Before Design) with asking “Do I have too few ideas? Or too many? Or just the right amount?” You can either amplify your ideas and generate more perspectives when you got too few of them, or remove ideas when you have too many of them.

But what has this to do with testing you ask? Well, let’s take a closer look on the four schools of thought in testing like Bret Pettichord defines them.

First, there is the Analytical school which

sees testing as rigorous and technical with many proponents in academia.

This is the school of testing which motivated analytical approaches to proof the software is right theoretically. I think this school of thought has been abandoned in the 80s, but it provided rather ordered techniques to achieve this. So, I would consider it in the lower left corner of the Landscape diagram.

The Standard school

sees testing as a way to measure progress with emphasis on cost and repeatable standards.

Here we have a high degree of certainty based on measured progress and emphasis on cost and repeatable standards, which means a high agreement on what we are supposed to do. Also in the lower left area of the Landscape diagram.

Let’s check the Quality school. It

emphasizes process, policing developers and acting as the gatekeeper.

From my observation this also means high certainty by providing policies and processes, and a high degree of agreement in the gatekeeper role. Lower left organized area.

The Context-driven school

emphasizes people, seeking bugs that stakeholders care about.

This is interesting to the degree that people bring in a new perspective here. People are sometimes hard to predict and this results in a lower degree of certainty in our process. On the other hand the agreement to bugs that stakeholders care about provides the prerequisite for a self-organizing team. If you add some agreement practices like Specification by Example you support this process as well. The structures of Exploratory Testing provide another agreement that helps the self-organizing process. So, the context-driven school appears to me to be the only one of the four mentioned here that does actually emphasize the self-organization of teams.

The bigger problem for me comes when noticing that the highly organized field is de- and un-skilling highly educated people in self-organization (which I actually consider a key property of being human). This de- and un-skilling more than often leads to de-motivation, and over time ends up in a high degree of uncertainty and agreements by trying to trick your boss on the crap he proposed. From the point of view of the managers though they keep themselves in the belief that they got a highly organized process in place. If you don’t believe me, take a look for effects like watermelon reporting in which you have a green status when viewed from the outside of the system, but you won’t know whether your system is red or rotten until you peek inside.

Now, compare this to a self-organized way to test. Here people got some simple rules on how to act like focus on stakeholder value, and support your team, in order to achieve the same goals. To some people this looks scary and chaotic. The Landscape diagram explains that it’s a bit more chaotic, but not completely chaotic when applied correctly. The problem often comes with the correct application. If you end up in a chaotic system, you should first of all notice where you are, and then stop to pretend that you are organized. Provide some simple rules to overcome the chaotic state and achieve a basic level of either certainty or agreement to reach for the self-organizing area of the landscape. And yes, this might mean that you got to do some management work. Hooray!

Now, get out of your chair and see how your team is agreeing and which level of certainty it currently has, and take some action on it if necessary.

The World Quality Report

On one of my newsfeed recently an article turned up claiming that cloud computing pushes the demand for quality forward. The referenced study is nothing less than the World Quality Report.

I was about to rant about this, and the flaws I see starting with the fact that with Capgemini, Sogeti and HP at least two cloud computing companies have paid for this study and report. But I figured that for the majority of my readers wouldn’t be necessary at all. You are probably aware that Quality is value to some person (Quality Software Management – Volume 1 Systems Thinking, Gerald M. Weinberg, Dorset House), and that the number three obstacle to innovation is the single solution belief – the belief that modern psychology has all the right answers (Becoming a Technical Leader, Gerald M. Weinberg, Dorset House).

So, go ahead, read the report for yourself, and leave me any flaws and/or supportive statements in the comments.