The Landscape of Testing

In the past few days I had the opportunity to stay with Diana Larsen. She explained a bit about her current studies on Human System Dynamics, and introduced some tools from this school of thought to my colleagues and me.

One of the interesting tools was the Landscape Diagram which you can find explained in more depth here. The basic concept aligns around two axis (on a meta level a lot of consulting tools do). The y-axis relates to the level of the agreement within the team on the approach to take. This might relate to the agreement on using a particular process like Scrum, or on introducing a particular tool in the software development process like FitNesse or JUnit. The x-axis aligns the certainty level with this decision. This might relate to the familiarity with the process or tool introduced. On the lower end, there is high agreement, while on the upper end the team is far from agreement. On the left end the certainty with the decision is very high, while the right end represents a great level of uncertainty.

Now, this Landscape Diagram reminded me of Stacey diagrams which I knew from the CSM course I took a few months back. In the Landscape diagram there are just like in the Stacey diagram some interesting areas. When the level of agreement and the degree of certainty is high on the lower left corner, we speak of a high degree of order and organization. In here rely domains like accounting, and taxes where a high degree of regulation is the way to go. Though turning in your tax statement might horrify you each year, you probably want to have that kind of order to support your community.

In the upper right corner, in the area of low agreement and high uncertainty lies the area of chaos. If you don’t agree to anything, and you are new to the field, then you are basically cowboy-coding.

Between these two extremes lies the field of self-organization. With some agreement and some certainty your team will self-organize. But this also holds for low certainty if you agree on what to do, and for low agreement if you have a high certainty or familiarity in what you are doing.

Interestingly you can move from chaos downwards into more ordered fields by providing structure thereby creating certainty and agreement. You can move in the other direction by opening thought process to new ideas. I think Jerry Weinberg refers to this in some of his books (i.e. Exploring Requirement – Quality Before Design) with asking “Do I have too few ideas? Or too many? Or just the right amount?” You can either amplify your ideas and generate more perspectives when you got too few of them, or remove ideas when you have too many of them.

But what has this to do with testing you ask? Well, let’s take a closer look on the four schools of thought in testing like Bret Pettichord defines them.

First, there is the Analytical school which

sees testing as rigorous and technical with many proponents in academia.

This is the school of testing which motivated analytical approaches to proof the software is right theoretically. I think this school of thought has been abandoned in the 80s, but it provided rather ordered techniques to achieve this. So, I would consider it in the lower left corner of the Landscape diagram.

The Standard school

sees testing as a way to measure progress with emphasis on cost and repeatable standards.

Here we have a high degree of certainty based on measured progress and emphasis on cost and repeatable standards, which means a high agreement on what we are supposed to do. Also in the lower left area of the Landscape diagram.

Let’s check the Quality school. It

emphasizes process, policing developers and acting as the gatekeeper.

From my observation this also means high certainty by providing policies and processes, and a high degree of agreement in the gatekeeper role. Lower left organized area.

The Context-driven school

emphasizes people, seeking bugs that stakeholders care about.

This is interesting to the degree that people bring in a new perspective here. People are sometimes hard to predict and this results in a lower degree of certainty in our process. On the other hand the agreement to bugs that stakeholders care about provides the prerequisite for a self-organizing team. If you add some agreement practices like Specification by Example you support this process as well. The structures of Exploratory Testing provide another agreement that helps the self-organizing process. So, the context-driven school appears to me to be the only one of the four mentioned here that does actually emphasize the self-organization of teams.

The bigger problem for me comes when noticing that the highly organized field is de- and un-skilling highly educated people in self-organization (which I actually consider a key property of being human). This de- and un-skilling more than often leads to de-motivation, and over time ends up in a high degree of uncertainty and agreements by trying to trick your boss on the crap he proposed. From the point of view of the managers though they keep themselves in the belief that they got a highly organized process in place. If you don’t believe me, take a look for effects like watermelon reporting in which you have a green status when viewed from the outside of the system, but you won’t know whether your system is red or rotten until you peek inside.

Now, compare this to a self-organized way to test. Here people got some simple rules on how to act like focus on stakeholder value, and support your team, in order to achieve the same goals. To some people this looks scary and chaotic. The Landscape diagram explains that it’s a bit more chaotic, but not completely chaotic when applied correctly. The problem often comes with the correct application. If you end up in a chaotic system, you should first of all notice where you are, and then stop to pretend that you are organized. Provide some simple rules to overcome the chaotic state and achieve a basic level of either certainty or agreement to reach for the self-organizing area of the landscape. And yes, this might mean that you got to do some management work. Hooray!

Now, get out of your chair and see how your team is agreeing and which level of certainty it currently has, and take some action on it if necessary.