Testing Challenges

A few months back I came up with the idea of Testing Dojos. Since then I spread the idea on Testing Dojos. The question I got most often when speaking about Testing Dojos is about where to get testing missions from. We have had the same problem when coming up with a mission for an upcoming Weekend Testing session regularly.

I also remember when I got back from the Software Craftsmanship 2009 conference I was inspired by Micah Martin’s coding kata, and asked for a testing challenge on one of my mailing lists. That’s how I first got in closer contact with Michael Bolton, since he followed-up on my request for a testing challenge by providing my with the Wason challenge which James Bach and he use in their Rapid Software Testing courses.

I annoyed my colleagues in the weeks after that with this particular challenge. Later I realized how hard these testing challenges are to find on the web. At some point a few weeks back I decided to register a new domain and simply put up testing challenges that I came across. Now, as of today I have put up the sessions we ran in Weekend Testing in the European chapter and some of other chapters as well. So far, this has been hard work, and with a critical eye on my schedule I would really love to crowd-source filling up the page with more testing challenges. You can find what we have so far on testing-challenges.org, and I hope this will be of great value in various circumstances in the months and years to come.

So far, I have restricted the access to the wiki for editing to users with a login, and the only way to sign up for a login is get in contact with me. That said, if you are interested in contributing, and classify and categorize the existing challenges, or improve the existing challenge with additional information like the original originator, drop me a line, and I can set an account up for you.

On another note, I also launched Testing Dojo.org at the same time. It is basically a write-up based upon my Testing Dojo article in the December’s issue of the Methods & Tools magazine, and is similar in its nature to Coding Dojo.org. Since I noticed many spam edits when taking a look at Coding Dojo.org, I also restricted editing on Testing Dojo.org to signed up users. You may need to contact me for this, but I’ll be glad to set up an account for you. Also notice that I included a list of public Testing Dojos which I hope to extend in the future. Until now it just references the Finland group meetings.

My Top 5 blog entries

Darren McMillan asked me today about my TOP 5 blog entries which I would recommend handing out. So, I sat down today and skimmed over my blog entries from the past. I was realy delighted and got many insights about stuff that I had nearly forgotten about. So, in the end I decided to share them with my readers as well here. Though, I had problems limiting my blog entries to just five, but I tried to categorize them as good as possible.

Continue reading My Top 5 blog entries

Why is it so hard to teach “what we started to call ATDD”?

Four months have nearly past since I started my new job at it-agile GmbH. Lots of things have happened since then. I got to know many teams, I learned lots about design, architecture, test-driven development, and also about testing. This blog entry is about the experiences I made since September in teaching ATDD, – I deliberately name it ATDD since I haven’t found a more suitable name, but I know that name should be replaced with something different – and what I plan to work on in the next year.

Continue reading Why is it so hard to teach “what we started to call ATDD”?

Active vs. passive testing

Good software deserves active testing. The activity of testing does not stop with the work of understanding what the software does. It must be completed by the work of criticism, the work of judging. The undemanding tester fails to satisfy this requirement, probably even more than he fails to analyze and interpret. He not only makes no effort to understand; he also dismissed a product simply by putting it aside and forgetting it. Worse than faintly praising it, he damns it by giving it no critical consideration whatever.

Huh? Harsh words. Let’s discuss them in the light of testing vs. checking.

Continue reading Active vs. passive testing

Software Testing Apprentices

Yesterday, Jason Gorman called for action. He made his readers aware that the state of the art of software development is poor – dramatically poor. If we continue to work and educate the generations to come on software development, we are surely set up to continue to decline our craft even further than we did. Gorman explains the educational system to be tremendously poor for software programmers, that we won’t survive in a few years from now – assuming that Moore’s Law continues to apply. Read his full call to action in his blog post Time To Look Seriously At Software Developer Apprenticeships.

Some of the points look compelling to me, considering where I have come from, and where I consider to be heading towards. After dropping out of university in 2006 holding my German diploma in my hands, I got my first job shortly after my father had died of lunge cancer at the age 58. (I interviewed with my employer a few days before he died.) That said, I didn’t know a thing about software testing, never attended a software testing course – not that there was one, nor that I would have been interested in the topic at that time – back in university. Within the first two weeks I mostly sat down to read about the product they were building, intended to get some knowledge about it in my head from the large documentation.

But it simply didn’t stick. All the time I was asking myself, how this would turn out. I needed something to play around with. So, I got introduced to the shell scripts they had built to test the software part I should be working on. Within some week I was able to run some of these tests on my own. Shortly after that I was working on the project to extend the behavior. I excelled at it, pulling in work from other colleagues when I found myself finishing my assigned tasks before the assigned timeframe.

One year later, we were working for our first client. In the meantime I had taken over technical responsibility for the tests we were running for the customization that we built at that time. My boss’s boss called me into his office, and offered me to lead a group of five starting from one week from then. I had stayed one and a half year with that company at that time, and was offered a leadership position for some of their testers. I took the job.

Two months later I attended that first formal software testing training… and I found it rather boring.

That said, how do software testers start to learn about their craft? I don’t know how all of them learn about the craft of software testing, yet I know how I learned it. I read in the evenings, and at the breakfast table. I worked on my PC at home trying to get more knowledge about the area I was confronted with, trying to find out new ways to “test this”, immediately trying them out the next day. I was mentored by my superior, I mentored some of our new colleagues during the whole four and a half year that I worked with that company. I asked testers to build something, I explained underlying concepts, and helped them reach an understanding of what I consider software testing to be. I helped to make them grow as well. Having the same lack of formal training in software testing, I think this is what I would call apprenticeship, isn’t it?

I wasn’t all too sure about this up until earlier this week. Having taking a testing challenge from Matt Heusser in early 2009, I had become a black-belt tester in the Miagi-Do school, turning later into a black-belt instructor. Over the past year I have worked with three testers through a challenge, and helped two of them go beyond their current level of expertise and skill.

One of them was Michael Larsen, the TESTHEAD. By far he exceeded my own expectations. He worked through the black-box software testing course, became a mentor in this course on his own. He also worked at a local boyscout club as a boyscout leader. During the past year he has produced the weekly podcast on “This Week in Software Testing” together with Thomas Ponnet and Matt Heusser. He also first reviewed some of our draft chapters in a book on how to reduce the cost of testing. Later he turned in as a chapter author himself, taking the opportunity to contribute to something that is surely becoming meaningful.

Now, Michael just announced earlier this week, that he is going to switch jobs. He is currently running a blog series on how he prepares himself for the new job, and let’s everyone else participate in his learning. Reminds me largely on the apprenticeship blogs I saw from 8thLight apprentices, Eden apprentices, or ObjectMentor apprentices.

But the interesting thing about Michael is, that he got this new job by steadily working on his passion for testing, continuing to grow, and pushing himself to new limits. But you should definitely read his blog entry on how he got where he is.

That said, apprenticeships don’t seem to be a new idea for the software testing world. Lacking formal training, and disbelieving in the syllabus of the certification programs out there, software testers have built a high reputation based on their apprenticeship programs, and we know how to run them. We already do Software Craftsmanship to educate our peers. Let’s continue this, and intensify it even further. Maybe that could be a nice goal for 2011.

XP Days Germany 2010: Scrum Norris

At the XP Days Germany 2010 I presented a Pecha Kucha about Scrum Norris, initially a blog post merely collecting Chuck Norris for Agile software development initiated by some of my colleagues from it-agile. Here is the translated transcription of that talk with the pictures included as well. As I missed to mention this during my talk, take this write-up as an over-sketched situation, and see if you can find Chuck Norris in your project as well. Oh, and take it with humor if you find out that you’re Chuck Norris in your project.

Continue reading XP Days Germany 2010: Scrum Norris

EuroSTAR: Exploratory Test Management Roundtable

At the final day of the EuroSTAR 2010 there was an Exploratory Test Management Roundtable facilitated by James Lyndsay.

At the roundtable we found out that the kick-off and the debriefing part of test sessions are essential. During the debrief test managers get insights in how the tester managed their time and focus. By trying to find out how much time the tester spent on preparing tests, to setup the system, and to actually test, the test manager gets to know whether test data setup or an automated installer could help to speed up the tester. Beyond this, costs and value as well as scope and benefit of testing activities are relevant. During the debriefing the test manager can find out how much value the testing brings, and whether testing costs could be reduced by reducing test missions, and bringing up new charters, or by extending them, if there are lots of areas with too few attention and focus so far.

One of the main points discussed handled the introduction of new testers to test charters and how much advice to give them. The whole discussion reminded me on situational leadership theory. In the beginning new testers need more clear advice and guidance. Over time the leader has to find the point when he can give more freedom to the tester, and less advisory is needed. So, while in the beginning the leader might pair up with the new tester, over time, the tester should be able to do more and more work on her own. At this point the leader can delegate more work to the tester.

From the discussion at the round-table I took two things that I want to investigate over the next few months. First, I would like to try out ideas to collaboratively bring up test charters. The identification of test charters reminded me largely on how Agile teams estimate their work using planning poker or Magic Estimating. Bringing up charters as a team in a collaborative environment seems to be a worthwhile idea to me, and I would love to explore this thing that I got in my mind since then further.

The second idea I got in mind, is how test automation and exploratory test session can be used in combination. How can we track test automation work and exploratory test sessions on the taskboard, and provide the feedback to the whole team? There are stories on the web from teams who track test sessions as tasks on the taskboard as well. I would love to play with some combinations, and see what works, and what doesn’t, and find out how to help teams make the trade-off more explicit.

EuroSTAR: The Danish Alliance

At the EuroSTAR conference a small group of people met at two of the evenings to share more of their knowledge and passion for software testing. Initiated by Shmuel Gershon, motivated by the Rebell Alliance conference at a conference, we met, ate, drank, and talked about testing.

Shmuel did an awesome job to get us together there, and he took the responsibility to file the videos from the lightning talks on youtube.com. Check out his blog entry which has them all, including a German guy who says something about feedback, responsibility, and craftsmanship. But you definitely want to check out the other talks, which are even greater than that.

Inspectional Testing is about time

Stefan Kläner commented on my last blog entry saying:

But an inspection for me is a somewhat regular process. Like state authorities would inspect restaurants to make sure they work according to certain standards, or an internal check if your company fulfills all FDA requirements before an FDA audit.
So in my opinion inspection would be a repetitive task. To stay at your windshield-example (which I like), you would inspect it in the morning, to check if it is frozen or not. Obviously you wouldn’t inspect in in the summer, that’s why I wrote “somewhat regular” as it is usely triggered by an event.

In order to answer the question, I have to get back to my initial inspiration, which is the book “How to read a book”.

Continue reading Inspectional Testing is about time

Inspectional Testing

Over the course of the past week, I noticed a name for something several Exploratory testers already do. I crossed the idea first while reviewing one of the chapters in an upcoming book on how to reduce the cost of testing. The chapter was written by Michael Kelly, and discussed Session-based Test Management or “Managing Testing based on sessions” as Carsten Feilberg recently pointed out at EuroSTAR.

The idea is simple. Challenged by Michael Bolton, I came up with the description, that Inspectional Testing is like scratching the ice from the windshield of your car in the winter. If you have lots of time, you scratch all windows free completely. If you don’t have enough time, you know that you should scratch everything free before starting to drive, but you can also select to scratch just some of the ice, so that you see enough to start driving. You risk a car crash at that time, but over time the ice will melt while driving. So, depending on the time you feel comfortable with to take for your testing activities, you start testing just enough. After the first charter, you reflect on your first time-boxed test session, and come up with additional tests as you see fit. You see clearer than before at this point in time, but you may want to dive into some topics in more depth. This is like letting the ice melt while you drive in your car.

Continue reading Inspectional Testing