At CAST 2011 Michael Bolton gave the first keynote presentation. Michael spoke on questions around context-driven thinking and the valuable and problematic aspects of modeling testing around schools of thought.

Michael apologized Cem Kaner, who could not attend unfortunately. Michael hoped to cover the same ground as Cem’s original talk was supped to be about. Michael referred to the CBC series on ideas in the past year. The series is called How to think about science. Bolton cited a discussion between Boyle and Hobbs in that series on how lungs deal with oxygen. The instrumentation of these experiments were called a matter of fact.

The experiments used a sophisticated setup. They protocoled everything they did. The experiments had to be reproducible, so every step was protocoled. The air pump experiments unfortunately were cursed by the unreliable tools fallacy.

Bolton said that these experiments could not proof that they can be repeated in other locations besides the lab setup in London. In general we don’t have enough data in the human history to claim that the laws of nature will be the same in a gazillion years form now. Testers know about this, since testing something does not ultimately mean that you proved it to work.

Bolton referred to Jerry Weinberg’s statement that decisions about quality are always political and emotional, and therefore unrational. Michael described the idea of schools of thought based on that. People have different values, that’s what makes quality statements political and emotional. Bolton referred to the endless debates about the idea of the four schools of software testing between James Bach and Cem Kaner, which Douglas Hoffman and James Bach are going to run on the second of the conference.

Bolton talked about the ISTQB movement and that he had a problem with how their marketing works. He referred to several occasions where he talked to ISTQB people, and they ended up walking out of that room when opposed to his arguments.

Bolton said that context-driven does not mean everything goes, but rather that context is considered foremost. He referred to a discussion between Jerry Weinberg and James Bach. Weinberg asked Bach what to do if you don’t have enough time to find every bug in a program. Actually, you never have the time to find any problem, but that’s not our goal as a tester. Instead we set up ourselves to find every meaningful problem with the software.

“It’s not what we know that is the problem, but it’s the thing that ain’t so.” Bolton cited from DeMarco. The value of any practice, or approach, or tool, or heuristic, the value of anything that we do, depends on its context. There are good practices in context, but there are no best practices. Bolton referred to the principles behind the context-driven testing school, pointing out the uncertainty there.

Bolton pointed out that differences between being context-driven, context-oblivious, context-specific and context-imperial. Context-oblivious testing is not aware about the complexity behind testing. Context-specific testing applies for medical software, where you can act context-driven, but also context-specific. Most of the other schools in software testing try to change the context for testing. This is what Bolton called context-imperial.

Bolton spoke about the nature of heuristics being fallible devices. Skepticism keeps testers from the fallacy to believe that every heuristic applies all the time. A focus on repeatability is so common in the Agile world that it inhibits discovery. Testing is much more subject to adaptability. If there are different contexts, we have to adopt to them. Bolton suggested to rather question how to run more tests compared to what most books on testing suggest. Thereby we can increase the number of important bugs we can find. Testers should learn how to observe people. We must also learn how to respond compassionately to all these busy people around us.

Bolton concluded his keynote with upcoming challenges for testers. How does software interact with people socially? Bolton summarized a definition of social science. Social science is about studying humans using quality and quantity measurements. They treat values and ethics rather than issues. They accept biases instead of ignoring them. Bolton thinks testers need to go into the direction of testing as social science.

We can also learn a lot on how natural languages adopt to different contexts. Natural languages are an expression of culture, and different culture respond to different contexts. Mankind can respond in 6000 spoken languages to different languages. We have to be cautious about certain practices in how and why they became practices. Understanding the context of different practices makes us context-aware.

Bolton concluded with various fields testers should take a closer into. These didn’t appear to differ from the fields he referred to at last year’s keynote at the Agile Testing Days. Bolton gave some examples of safety language. Safety language refers to pre-mature judgements in our communication. Instead of saying “No user would ever do that” Bolton and Bach teach in Rapid Software Testing that it really means “No user which I have though of and which I like would do that on purpose”.

Bolton finished with a call for testers bringing everything they came with to the table to support diversity in testing.